Directories, Symlinks, Memory Usage (Mac OSX)

[combhua]

TLDR version: Symlinked, same/diff directories showed no noticeable difference in memory use whether it was 1 or 5 instances of wow. Why?

I (try to) 5-box on one Mac Pro with the following specs:

- 2 Quad Core 2.8 GHz (8 cores total)
- 4 GB ram
- nvidia geforce 8800 GT 512 MB
- 1 stock 320 GB (I think Seagate Barracuda 7200.10)
- maxfps: 30, maxfpsbk: 10
- sound off
- lowest video settings except main has max view distance
(amazing to think these are standard specs on his board these days /feels bad for his old windows machine)

Outside of cities, it works modestly well. It's rare to lose a follower (I use Jamba follow strobing - excellent stuff). I do find I have more difficulty if I turn up settings. Porting into Dalaran is a nightmare however. I almost always disconnect someone if I do all 5 at the same time (the main having a load of addons may contribute to this). /follow through Dalaran is impossible.

So I started digging around and have been using the method of symlinking the directories for a while now. Still disappointed with the results, I wondered about the expected efficacy here. To me, it makes sense that the OS would recognize the same files are being requested and only pull them from the HD once, only needing one copy, right? Well, one test led to another and I'd appreciate another perspective on these results.

How I set up the tests
I wanted to measure memory usage under running a varying number of WoW instances:
- from 1 directory, no symlinks used
- from different directories, no symlinks used (yes, 5 Data directories take a long time to duplicate)
- from different directories, all using symlinks (that is, all 5 point to a Data directory not being otherwise loaded in the test)
- from 1 directory, symlinked to another, otherwise unused directory, in Dalaran

For each of the above conditions, the machine was rebooted and no extraneous software was loaded. In the first three, the toons were standing in the same spot with the same view at the mailbox in Warsong Hold, Borean Tundra. The Dalaran test was conducted with the toons standing on the sewer grate at the entrance to the Horde area and the camera at max distance directly overhead.

What I found was that there was very little difference between conditions (except Dalaran) and little to no reduction in memory used per instance of wow as additional instances were started. At 5 instances and ~1500 MB of ram used for Wow, there was a less than 10 MB difference between setups. I also monitored what was labeled as "Virtual Memory", "Private Memory" and "Shared Memory". Again, those had differences of around 10 MB. Free memory exhibited similar behavior with a slightly larger variance. The Dalaran setup obviously used a significantly higher amount of memory in all cases.

Perhaps I lack a deeper understanding of the innards of how Mac OS reports memory info to Activity Monitor (assuming via top). There were 4 available columns related to memory and I recorded those in the spreadsheet. You can see the everything else in the screenshots of activity monitor if you're curious.

The simpler-to-read spreadsheet does a better job of lining up the relevant numbers for comparison. Each 'sheet' is a condition of running 1, 2, 3, 4 or 5 instances of WoW. Vertically on the sheet, you'll find the four differences in directory usage and the readings of memory usage for each instance as well as a total.

Reading this, I'm surprised by the lack of difference between symlinking and not. I thought (based on my readings at this board) that the symlink results would be best from and the 5-data-directory setup would have the worst memory. While I never expected 5 symlinked copies to use 1/5th of the memory, I was shocked to see it used more than 4/5ths the memory and debatably no difference. At first, I thought it was because my 5 instances pretty much maxed the memory anyway, so that's why I tried 1-5 instances. However, those provided the same results.

Can anyone explain this? I hope this sparks some solid, discussion.

* This is by no means a very scientific experiment. There were way too few samples and not enough controls. Also the person planning the tests (me) does not have extensive knowledge of the behind-the-scenes bits of memory management. So take it with a grain of salt.

Thanks for your time

What this does not show is the more realistic performance of actually running around while random textures and models are retrieved from the harddrive as I didn't have at good method for testing that. Any ideas are welcome.

[pewshamanper]

i know this is not relevant but I love your computer i want it!!

but i am 3 boxing with 15 fps on all guys with highest detail on maine
with 2.4 ghz duel core
imac 20" 2007 model 3g of ram

[user]

To be honest i didnt read it all.. but then i did read it and it was all over my head. Any way i think its a lack of ram. I run 2.8Ghz dual core 4Gb ram 256video

Im not exactly concerned with performance at this time being lvl 40 and running instances but i reallly hope RAM comes alot cheaper when I get to lvl 80. Im sure there alot apple wizards here, somewhere.

edit: so i read again and yeah interesting. sure you got 512 video card?

[MultiMacBox23]

Smy-linking is directing each wow as to where to look for the data files. Whether you have is symlinked or separate directories it is not going to effect the memory. a symlink "Symbolic Link" is a virtual folder that points to a folder in a different directory. we are dealing with hard drive space/location NOT memory usage.

what has performed the best for me is symlinking. i have tried all the other combinations. with the symlink the hard drive is not bouncing all over the place trying to pick up the same piece of database information (players, objections, textures, etc.....) from 5 different directories in 5 different physical locations. With symlink all 5 wow's are trying to grab it from the same directory through the symlink redirecting each request to a shared directory. (EDIT: I have not tried 5 separate hard drives, with wow separated to each one.)

You are trying to compare memory usage for 5 programs (WoW) with different ways that they store their database information. Of course you can't measure any real difference. Whether you locate your data base in 1 or 5 locations, each Wow is going to only load the same amount of it in memory each time.

you should be testing loading times/disk access times to see differences between the different methods, that is the benefit of symlinking.

With OSX, it will use any unused memory present in the system as a cache for disk access. it will dynamically allocate and deallocate it, and you wont see it happening in the monitor.

Wired = System/OS vital memory (non-touchable. its "wired" in and cant be unallocated)
Active = Used by applications
Inactive = WAS used by an application that was closed (it keeps this memory ready to be used if the app is reopened)
Free = Free, not used


Hope that clears a few things up.



also, second graphics card = no improvement in frame rate, only benefit is more than 2 monitors.

memory is your friend. about 8gigs is the point of diminishing returns.

in my tests, distance has more of an impact than detail/texture quality. i run my main window fullscreen on my 30 everything maxed. and the other 4 split evenly on the second 30", and the other 2 20" screens are for things like safari to lookup wowhead/thottbot, widgets, watch a movie, etc. I have my settings at 60fps main, and 15 slaves. i average 25-40fps on main screen (this is after wotlk it used to be better during Vanila wow & TBC) and still rocksolid 15fps on slaves.

[user]

Brilliant MultiMac!!! even tho its not my post

[combhua]

TLDR version: Symlinking saves space. I cannot see it improving performance for the reasons listed below.

Maybe I misread things, but a lot of posts on these boards implied significant performance boost when symlinking. One of the ways this would be evident would be what I tested. Apparently that's not the case. Aside from about 3-4 months, I've played wow since f&f beta pretty hardcore, so I have a pretty strong sense of 'typical' and 'atypical' performance.

First, if there's a way to objectively test and quantify results for the improvement in load times between all five clients, let me know and I'd love to test that and share the results.

I would think though, that if symlinking improves the seek and load time of 5 instances of wow because it's only having to find the file once, then I would guess that one would only see a relatively minor increase in load times, not a linear (or worse) increase. Additionally, if the clients are indeed sharing information, then it makes little sense for them to load 5 separate copies of the same files. Here's why:

There's two scenarios. Either 1) The WoW client is managing the file io and loading textures/etc into memory or 2) The WoW client passes the request on to the OS, which manages the file io and loading into memory.

1) In the first case, the WoW client would know the seek location for that bit of data and either share it with the other clients, or not. 1a) If it shared the info (which I doubt bc of security), then there's little rational for it to not point to the memory location of the textures instead of the physical drive. 1b) If it did not share this info with other clients, then each client is indeed seeking and locating its own data independently of one another.

2) In the second case, the OS is aware of the seek location and managing the passing of memory on to the wow client. The OS would likely handle requests by loading a textture, etc into ram and then when the same requests comes in from client #2, it would provide the same handle to the client, not reload the file and provide another pointer.

If neither the client nor the OS provide any sort of advantageous pointer/cache sharing, then symlinking has virtually no effect. If you're aware of the measures Blizzard takes to protect the reading of data from their client, it makes a lot of sense that the wow clients do not share data between one another.

I would love to be wrong about this so that our hardware doesn't have to scale linearly to maintain performance. If someone has ideas for objective tests that can be repeated and quantified, let me know.

[MultiMacBox23]

#1 Answer = 1b. Each wow client has no clue what the other is doing, as it is originaly intented to only run 1 client. They didnt build WoW to multibox, we just force it to.

#2 Yup/correct

Currently with 5 clients ran at the ram time, in an area thatcan get my framerates to dip below what it usually floats at, i'm at 45% total CPU usage (averaged of all 8 cores).

The only 2 things that can help me now are faster GPU & possibly faster data xfer between system --> gfx card(s)

waiting to see how "Snow" leopard, and its OpenCL ability to offload computations to GFX cards will effect performance, since my second 8800gt is barely getting tapped at all.
from what i can tell my second card is displaying the wow-clients / and other system windows on the other monitors, but none of the accelerated abilities of the card are being used.

my simple/lazy way of testing that is that FPS with 1 card vs 2 cards = less than 5fps difference, and even that can get lost in the FPS up/down fluctuations. i uncapped the FPS limit on the main,
noted it. then installed the second card, and moved the second 30" from the second port first card, to first port second card. re-noted fps.

-ray

[hertome]

Hi everyone,

i tried to symlink my wow directories but failed, i cant get the shell script to work i always get an error message.
I got an MacPo with 1 proc. and 4GB Ram.

But that is not what i wanted to ask... Is there a way to move the swap-file to another hdd under osx 10.5?

I recently got an SSD and it gives me an incredible perfomance boost and i wanted to move the SWAP-File to
the SSD for faster acces.

Thanks in advance.

Greetings

Hertome