Two areas stand out to me, video and RAM. Very cheap to drop in another 6GB in that setup, either another 3x2GB set or even swapping to a 3x4GB set should be ~$100. Video wise something like an Nvidia 570 would be the way to go IMO. If you can afford it, the 580 is amazing (I switched from a 295GTX which had very similar performance to your 260 thanks to SLI issues and the improvement was quite tangible on a 2560x1600 screen). I have never found GPUs to be particularly rewarding to overclock personally. If you want to overclock anything, the CPU is a good candidate and relatively straightforward to get up to 4GHz from the standard 2.66 on air, espeically if you drop in memory around 1866MHz into it.

Cheers