This isn't going to help you any, but my latency is a fairly consistent 30-60ms.
So the issue is somewhere, either on your end... or somewhere between your system and Blizzard's system.