Basically you lose follow or some of your guys just plain disappear if they aren't on the same quest line.