I only drop totems for boss fights, but then again, I have a pally tank.