You can use different brands (AMD & nVidia) videocards, and you can also use different models of the same brand (8800GTX & 9600GT). However, if you use two of the same, you have to use the same drivers - like nVidia 172.69. You *cannot* use SLI/Crossfire with different brands, but most people hooking up multiple monitors with multiple videocards shouldn't be thinking about SLI anyway.

If I were to build a new and relatively inexpensive 5-box machine with 3 monitors, I'd do something like this:
q9450 (cheapest 45nm quad core with more cache than q9300)
8gb ddr2 memory (ram is cheap, so $100 goes from 4gb to 8gb)
evga 780i or asus P5E3 premium (either board for 3x pci-x16 slots, even if 2 of them run slower than 16x.. asus if using ddr3 and/or pci sound card)
3x 8800gt/9600gt or 3x hd4850 (wow isn't demanding, and either of these cards will be fine for the lcd resolution)
3x dell/samsung 20"/22" 1680x1050 or 3x dell/samsung 24" 1920x1200 LCDs (depending on money available)
g15 & x-keys pro.
vista 64
keyclone

Lot of options for hard drives, cases, power supplies.. but the above would be perfect for the input/output/processing.