Whenever you load an instance, you have to wait for the slowest box to load. If you are ok with that and it runs WoW well enough, then leave it be.

You can take this as far as you want to go but at some point you get where the marginal cost becomes overwhelmingly high.

Get the basics, play with it and if it is what you want then you can always upgrade later, especially given you have a hodgepodge of systems as it is.