I wouldn't give up on it. I did it last night in Normal and quickly identified how ridiculous the movement required on that fight was. It really reduces the amount of options you can get on some fights. It also seems like some of the dungeons are much more difficult than others for us, with that in mind there were dungeons in Legion too that I actively avoided if I could for Mythic keys just because it wasn't worth the hassle. I'd say your team is fine if you did everything else without concern.
The curve keeps getting higher and higher for us, but honestly playing just 1 is not even the same satisfaction.