I don't know about the lower damage. I think the only place I would be sacrificing damage would be the gems. Every where else, I would just be losing crit.