Most people seem to have a few combinations for follow.

A) An auto follow key.
B) A method to break auto follow.
C) A way to move the "master or hot character" only.
D) A way to move the "slaves or alts" only.

It seems you're attempting D, but to move them individually instead of as a group.

The easiest solution might be changing the keybinds.
You don't need them to be Alt + Up Arrow.
You could use Alt + Arrow Keys, Ctrl + Arrow Keys, Shift + Arrow Keys, Alt + WASD keys, Ctrl + WASD keys.
For that matter, you could bind Alt + Ctrl + F7 to a movement if you want.
The trick will be remembering your keybinds.