Its been a while since I have boxed SWTOR, but from what I remember is that any combat action you do breaks follow.
I know I was spamming the follow key (which I bound to one of my side mouse buttons) all the time.