I have the same kind of issue, mine is fixed by just pushing alt.

What I believe happens for me is that when I alt-tab out of the main window, the 4 slaves don't stop pushing alt. So when I come back, anything i push won't work on the slaves until I press alt again.

Maybe this is your issue too?