It's been a while, but I have mine setup that way for AV (so i only see my party healthbars)

I think steps were:
uncheck the "hide party frames in raid" (general options or Party options if I recall)
disable raid bars display (if you don't want them)