http://www.wowwiki.com/Formulas:Mob_XP#Group_Experience

Its not going to be a huge difference, a 5 person groups gives a 40% bonus while a 3 person group gives a 16% bonus - remember tho the xp is first split across all 3-4, the group bonus is supposed to help mitigate the xp split, I believe for absolute fastest leveling boosting just one character is the fastest, although adding more doesn't slow you down significantly; probably around the order of 20 or 30% longer with 4 alts instead of 1, since you are killing elites in instances stuff so fast most of the XP comes from mob XP/hour (x3 xp in RAF) instead of group bonus.

4 can instance in outlands the early instances easy enough, your mage can probably solo ramparts/slave pens. The latter instances will be very hard though, especially the 70 ones. Just for that I'd suggest picking up a 5th, so you can always do any instance (or heroic, once geared). Keep in mind its a ton easier having as many of the same classes as possible. Just fewer buttons to push and fewer timing issues to worry about.