Ok lets do some testing:

All same toon all dalaran: reload with


No Addons: 2.21s
Only ACP+ISboxer 2.57s
Basic 6s
Slave 7.19s
AH/Crafting 7.61s
Solo 7.93s
Master 11s
Raid 8.88s

It appears that the issue lies somewhere in the addons that I've declared "Basic". Will continue testing.

Basic: Arkinventory, arkinventory rules, bartender, clique, friendsnomore, grid, grid custom layouts, gridindicatorsideicons, griddungeonrole,isboxer, jamba, jamba talk, jamba taxi, jamba toon, Minimap button frame, mysales, omnicc, omniccconfig, quartz, reagent restocker, SUF, SUF options, swatter, TSM, tsm item tracker, tsm mailing.

Narrowing this down is going to take a while and losing any of these will be suboptimal.