I have keymap enable/disable and broadcast enable/disable keys in my Always On keymap which do pretty much exactly what you're talking about, though for the entire group instead of a single slot; any time I add a new keymap (e.g. for another class or spec) I add it to the enable/disable keymaps so it gets toggled with everything else. It should be fairly trivial to right-click -> Make Copy of the standard enable/disable mapped keys, change the keybinds for them and change their actions to enable -> self and disable -> all of: other to do what you're after.
If that doesn't make complete sense, I'm sure I can do a test setup and post the config file for you to use as an example. Actually, I've been meaning to do something like that for a while for those times when I want to park everyone else and just "play" one character for a few minutes.
Connect With Us