My son and I just spent 2 hours fighting Malistaire pretty much by ourselves, because the 3 other kids who teleported in with us were useless. One left after 10 minutes. The second left after 30 minutes because he ran out of mana. And the third (who teleported in just before the Malistaire fight) wouldn't cast any spells because he was afraid of running out of mana.
My son and I had Malistaire down to 1,000 health when our router disconnected. After I rebooted it, we tried to teleport back to the kid who was still there, and it said the instance was full. But the instance obviously was NOT full, because he was the only one in there, and my son and I had just left maybe 30 seconds earlier.
That is your routers fault. You obviously like the game or you wouldn't care enough to spend that much time trying to kill a boss.
It takes games a while to realize you are not "still there". It isn't only this game but many games. I used to play WOW, and i would get booted once in a while, to have to log in with another character, and see that my first character was still "online" when I wasn't with him.
It happens. I don't know if it can be fixed or not, but none of the other games its happened to me in ever fixed it.
But in this case, it is your routers fault. The game didn't boot you, your hardware did.
the way to get around the full instance bug is really simple, first start the instance over on your own and once your back inside teleport back to your friend
and wasting 2 hours on a fight is meaningless compaired to the amount of gameplay hours you put in just to get up to that point.
I also did what DarthNexus described and it worked. I was in MB on Katz Lab at the time (much more minor, I know) and I teleported out to get mana. When I went to go back in, I got the instance error. I was really upset because I had friends in there and we were almost done. I figured, "well, I've got nothing to lose--either I'm going to start over, or I'm going to get back in." So I went through the "front door" of the dungeon, then teleported to my friend. It worked.
I had another friend ask for help with the Death Oni later on. I ported to him, he ported out to regain health, and tried to come back. He got the same error message. I told him about the above work-around. He did it, and it worked.
So it's replicable. Until a solution is found to the full instance error, this seems to be the best work-around.