Purge will only work on those if your target has a purgable buff. If not it wont do anything. In fact you wont cast it at all.

This makes purge a bit unreliable spell to remove grounding totem since you most likely (should) have purged down your target before burning you NS/EM. Rank 1 shock could be an alternative if you cant remove it with purge.