You won't lose the links, but you will lose any ungranted levels. I found this out the hard way when i moved my level 45 shaman and then at 60 could only grant 7 levels from each.