Follow rarely breaks for me randomly, but when my internet connection was getting a lot of use (like a torrent that isn't throttled properly) and I tried to run through the common in Ironforge, I had follow break about 3 times. I have a similar setup to you, 1 pc (Vista 64, Q6600, 4GB, 8800GT 512MB), main on a 21inch and 4 slaves on a 19inch.

I have a few addons loaded on both main and slaves and haven't had any real issue apart from that one time. I hope you work out a way to minimise your follow problem. There are a few multiboxer addons that will tell you if follow breaks on your slaves; you might find an addon useful if follow is breaking frequently for you.