Once the word starts to spread on your server or battle group (well, BG might be harder to crack) then you'll get less whispers from your own faction, the other faction tend to be constantly bemused though...some will /cheer some will use /spit (hate that emote). But anyway, you'll become immune to it soon enough and it actually becomes funny after a while.

I always make a point of explaining to people that ask what I'm doing and tend to ignore the 'WTF' brigade unless they can be bothered to string a sentence together

I also mentioned what I was doing on the realm forums so people understood I wasn't a bot. Mixed reaction but mostly positive and did help reduce the whispers.