I think that's currently expected behavior. For now, I've been told the dialog box goes away after a few minutes if you don't click on it. But yah, I'm pretty sure that's expected behavior.