OK I did some testing last night.

System. Athon X2 4600+
4Gb Ram (stupid motherboard allows 2.8Gb)
Radeon X1950XT (512Mb) - 24" monitor 1920x1200
Radeon HD2400 Pro - 19" CRT + 17" LCD

I have my WoWs all running on the 24" in the following setup 1x 1520x1200 on the left and 4x 380x200 stacked vertically on the right.
I am running Keyclone 1.8e because 1.8f seems to shift around the hotkey for instances and my macro for leader ends up not being the focused instance.

Screen Layout
[img]index.php?page=RGalleryImageWrapper&itemID=61&type =original[/img]

So when I first swap 1 & 2 it take about 1 - 2 seconds. Swapping back takes ~ 0.2 secs
After the warm up phase (i do a full sequence swap as it does seem to cache something

Swapping 1 - any = 0.2 secs
Swapping 2 - 3 takes ~0.8 secs, is the time to swap any of the small windows with another small window

I think this is due to the background sorting of windows as swapping 2 and 3 will have to swap 2 and 1 as well so that 1 ends up in the 3 position.