I find that as long as I have "set jamba master" as part of my isboxer switch, I can easily fix this problem by switching to someone else and then back to the master I want. I also make sure that I always have the team panel only show up on the master and the party leader to always be master so that I can easily see that the toon I'm on knows YEAH IM THE MASTER before any annoying follow behavior can manifest.