Create and ATG for your mages. Put them in it, in the order you want them to nova. Create a keymap, that doesn't have a keybind to activate it. Call it something like "Frost Nova Rotation". In that, create an action that outputs the hotkey you have set for frost nova in game, and sends it to all mages. Tick the round robin box. Set it to not pass for however long you want the interval between frost novas to be.

Now, create a two step keymap. In step one, select the option "kep mapped actions" and choose "do mapped key acton". Select "Frost Nova Rotation". Now in step 2, create a regular keystroke action, say arcane explosion is bound to 3. So have it output 3 to all mages.

Now, you have a DPS rotation, that is calling another keymap, and that keymap is doing a round robin with your mages frost nova. Say you set the round robin do not pass to 3s. The first time you hit the hotkey, 1 mage will frost nova. Then all mages will AE and will AE until 3s has passed, then another mage will frost nova, then all mages will AE.