Out of curiosity, do you have Click-To-Move enabled 24/7 on your slaves? As an example, in my setup the only time click-to-move gets enabled is when I press W to interact with my leader's targeted NPC/corpse/mob (in the on-keydown action). It gets unset in the on-keyreleased action and every time I swap characters just to make sure it's off.