When you go into keybindings look in the top right corner of the keybinding window, it has an option for GENERAL or THIS CHARACTER ONLY. Try setting up your non-bongos character's keybindings using the THIS CHARACTER ONLY tab. Hopefully this will retain your keybindings for your non-bongos character.

EDIT: it actually says "Character specific key bindings" and it is a checkbox.