I'm pretty sure I was close to enough for them to follow after combat so I doubt that is the issue. It also happens during combat, almost like they are headed towards a location other than where I told them to go.

I didn't know about being able to set the click to move to on and off so I mapped those keys and am going to see if that does anything for me. It was set to off prior to me messing with it.