you could turn off the focus-follows-mouse until the next release...

personally, it only happens once a night, so there is something to how you release the button (very aggravating though)