Could be lag, if you switch targets quickly during casting, the slaves haven't had a signal that your main has switched. You could stop casting for a second when you switch targets, that normally helps for me. I have this problem occasionally even with FTL setup with IS.