I'm slowly migrating my setup from a mid-tower setup to a rackmount. I've got my main in a 4U case. I agree that 1U is WAY too loud, and really 2U should be the minimum height for the fan noise and cooling. 3U would be more ideal for the slaves and a 4U for your main. I'm going to end up with a 19U height, which comes out to be about 40" tall, but reasonably quiet.

I remember at my last job I went from tower servers to a 1U rack setup. Mind you my office is where the servers were. That was a HUGE mistake. Only go 1U if you can remote mount them in a closet or another room.