More than likely, your current master thinks that the slave is its master and that is why it follows the slave.

When you switch toons, you need to make sure you tell all characters in your Jamba team that the new toon is the Jamba master.

Easiest way to test this is to
1) invite all your toons to a party
2) make sure the team option to promote toon to party leader is on (push settings to all toons)
3) switch to your first toon; does it get promoted to party leader?
4) switch to your second toon; does it get promoted to party leader?
5) ... etc; cycle through all your toons makeing sure they all become party leader. If they don't become party leader, Jamba is not getting the message that the master has changed...