You can do it on the same PC, but you have to have the game launching the Direct3D instance on the desired video card, and then the window is restricted to the monitors driven by that card (consider it just like using 2 PCs, your windows are restricted to the PC they are launched on). There is (currently) no efficient way to transfer the buffers stored for rendering by card 1 to card 2 (or vice versa) and therefore it is always going to be slow if you move it to the other card. So yeah, "that's the way it is".
However, someone recently pointed this out to me:
http://www.matrox.com/graphics/en/products/gxm/ which I believe would allow you to run 4 monitors off 1 card if you had 2 of those dual head bad boys, one for each output on the video card. That'd give you the monitor count you're looking for on one PC, and not drop your framerate like a rock if you move the windows around on all of them.
If you want to go 2 PCs, what you are thinking of would be fine.