After looting a few times and NOT switching windows (happens about once per 5-8 mins) it would start following one of my other toons even when follow after combat was turned off (happened with and without it on).

This problem does not occur without this addon actually running/active.