I've been trying to figure out how to do this the last week. What I ended up with was similar to what you are doing, but just a tad different. I didn't want to specify the specific order of the party in the event that I drop one character out or have someone else in the group, so I put together 5 macros.

These use modifiers to cast on the correct targets. Put the correct macro on the character that needs it based on where they are located in the party setup on your main screen.

Code:
Player = nomod
Party1 = Ctrl
Party2 = Alt
Party3 = Ctrl Alt
Party4 = Ctrl Shift

Player
/cast [nomod:ctrl,nomod:alt,nomod:shift,target=Player][mod:ctrl,nomod:alt,nomod:shift,target=Party1][nomod:ctrl,mod:alt,nomod:shift,target=Party2][mod:ctrl,mod:alt,nomod:shift,target=Party3][mod:ctrl,nomod:alt,mod:shift,target=Party4] Spellname

Party1
/cast [nomod:ctrl,nomod:alt,nomod:shift,target=Party1][mod:ctrl,nomod:alt,nomod:shift,target=Player][nomod:ctrl,mod:alt,nomod:shift,target=Party2][mod:ctrl,mod:alt,nomod:shift,target=Party3][mod:ctrl,nomod:alt,mod:shift,target=Party4] Spellname

Party2
/cast [nomod:ctrl,nomod:alt,nomod:shift,target=Party1][mod:ctrl,nomod:alt,nomod:shift,target=Party2][nomod:ctrl,mod:alt,nomod:shift,target=Player][mod:ctrl,mod:alt,nomod:shift,target=Party3][mod:ctrl,nomod:alt,mod:shift,target=Party4] Spellname

Party3
/cast [nomod:ctrl,nomod:alt,nomod:shift,target=Party1][mod:ctrl,nomod:alt,nomod:shift,target=Party2][nomod:ctrl,mod:alt,nomod:shift,target=Party3][mod:ctrl,mod:alt,nomod:shift,target=Player][mod:ctrl,nomod:alt,mod:shift,target=Party4] Spellname

Party4
/cast [nomod:ctrl,nomod:alt,nomod:shift,target=Party1][mod:ctrl,nomod:alt,nomod:shift,target=Party2][nomod:ctrl,mod:alt,nomod:shift,target=Party3][mod:ctrl,mod:alt,nomod:shift,target=Party4][mod:ctrl,nomod:alt,mod:shift,target=Player] Spellname
Once that is set up, then I use a template to send the command
Code:
	<ApplyTemplate VButton 1297 678 28 31 "Ctrl Oem4" "Ctrl NumPad1" "w2,w3,w4,w5">		//Player 1 - Party buttons
        
<Template VButton>
	<If MouseIsOverWindowRect WoW1 %1% %2% %3% %4%>
		<SendLabel %6%><Key %5%>
<EndTemplate>
The next obvious question is "what if my main character is in window 4? Or window2?"... Such is the next problem... I just flipped my labels around so they target the correct windows when i reference the label.

Code:
<Label w1 192.168.1.3 SendWinM WoW1>
<Label w2 192.168.1.4 SendWinM WoW5>
<Label w3 192.168.1.3 SendWinM WoW2>
<Label w4 192.168.1.4 SendWinM WoW4>
<Label w5 192.168.1.4 SendWinM WoW3>
The last part of the problem is that doing this requires 5 macros for any spell you want to cast like to be prepared for everything. If you don't have many party targeted abilities to use, then this method works out fine. The other method would be to turn this into a "/target" instead of a "/cast" and then add an additional command after the /target to cast the correct spell which requires two clicks.

If the party membership changes, it requires a switching the macro out on each character and changing the label around in the code, which is the least amount of work I've found when dealing with switching in a party.