I'm running 5 aswell but have been considering reducing it to three for the same reasons as the op, aswell as the scenarios..

One thing i'm sort of curious about though, how many from same guild is required for the guild to earn gold from a dungeon on a weekly basis? If 3 is enough that's it for me, then i'll go with three!