Are you broadcasting your mouse?
It sounds like everyone is hitting the dismount button, when the main does.