I find it happens when I am in combat (or entering combat) and I hit my:
/target party1
/follow
/assist

And I only noticed it happenening after I selected [x]Attack on Assist. I have not tried to undo that selection to see if it is relevant. Also, I play on two machines and see it happen to followers on both.