You need to increase your RAM and upgrade your video card. 4 gigs is really low. I'd try and upgrade to 12 to 16 gigs of memory and a nvidia 570 video card or 580gtx if you have the money.

Not sure how old your Xeon is, but is it one or two physical processers? I'd guess 1 with 4 cores, and 4 HT cores.
You might be able to throw another processor in there and get a really nice boost as well. But first the memory and video card.