Either its lag or there is a buff on the master.

If you follow the slave and don't have problems, I would bet its a lag issue.

That said I used to have this problem a lot before I had decent hardware, it only occurred on mounts (pre-flying) and carrot and a stick on the slaves only was enough to cure it.