I will update this as i play some more but it was just breaking with my latency at 255 and my frame rate at 20... i am not sure if those numbers make sense of my follow breaking or not.

EDIT: My follow was more like a huge debuff and i could have probably gotten across the map faster walking... my latency was at 212 and my frame rate was 36.

EDIT: Again more fail follow and it was 122 and 21 for Latency and FR

EDIT: Again at 133 and 29

EDIT: Again at 235 and 28