You can't move a group with arrow keys .....

When you turn you will have to get inaccuraces of 1 degree and if you go a long distance even 1 degree apart they will end up very far away from each other.

Autofollow is the only possibility (unless you want to move guys one at a time lol).