But my question is when I controlling the main client, and someone whispers me, when I reply how can I stop the clones from also replying to the person who whispered me ?
Just hit the Keyclone "override" button to temporarily disable broadcasting so that only your main replies.

If you're using Jamba then the bonus is that even if the whisper was to a drone, you can reply from the main and the sender will see the reply as if it came directly from the drone.

I don't have any keys in any of the various lists that KC lets you maintain - my setup relies entirely on simple key broadcasting - so I'll leave it to others to share their experience on that