Are you starting your casters off at max range? I do it without tremor totem and he doesnt get more than 1 hit off on another before the fear breaks. I do a quick taunt and pull him back as I face him on the dps and back up. You can also bubble and drop it immediately or try using a pvp trinket.