If it bothers you, report the guy.
I generally don't care what some random guy thinks about me, so don't report them often.

There are also addons to block whispers, I believe originally made for the gold spammers.
You could force the guy to level to 4th or 9th (or whatever), before he's allowed to whisper to you.