From Dual-Boxing Wiki

Software video capturing

A list of software(free and commercial) can be found here:
http://en.wikipedia.org/wiki/List_of_screencasting_software
CamStudio is quite useful.

Software video capturing requires a lot of CPU power and will decrease the game's performance. You can try to set the priorities of each window manually in the task manager to get better results.

Hardware video capturing

• You can use a digital camera to capture your screen and use software like VirtualDub to encode it correctly.
• You can use a hardware video capturing card and capture a computer's video output with that: http://en.wikipedia.org/wiki/Video_capture_card