If you want to remap via when using videofx, then you actually need to turn off the passthrough, and instead setup a Key Map (lets call this VideoFX Remap) with some mapped keys in it, with the hotkeys and remaps you want. You then create another mapped key in a different Key Map (like Always On), which will be loaded. This last mapped key, has two steps. Step 1, load the Key Map VideoFX Remap on the current window. Step 2, unload the Key Map VideoFX Remap on the current window. You set this to execute a step on "press OR release". You then have this execute on mouse enter/exit on your videofx viewer.

So, when you mouse into the viewer, it loads the VideoFX Remap keymap, and all your hotkeys and remaps are active. You mouse out of the viewer, and it unloads it and you are back to normal.