Arma3 and the /LARGEADDRESSAWARE flag (memory allocation > 2GB)

You can see that Task manager only shows 1.5gb in use, but that's only the Private Working Set, so that's only physical memory allocated. Then you see the total commit charge in Performance Monitor and you see it's at 6.6gb and the big red bar shows 43% of my total commit charge in use. So ArmA 3 does use at least 4gb of memory and does fill the entire 32 bit addressing space.

To confirm this with 100% certainty, you can use Process Explorer (look for Virtual Memory) or VMMap (look for "Total") tools. Both tools are official Microsoft ones from Mark Russinovich and helped me not once to debug memory problems.

If i understand this: http://forums.bistudio.com/showthrea...=1#post2481817 correctly,

the recommendation for a modern system with 64bit OS and 8+GB RAM is:

"dont use maxmem and let arma 3 change internal limits automatically" ???

Yes, that should work very well in most cases (there might be some corner cases of course).

---------- Post added at 09:54 ---------- Previous post was at 09:46 ----------

@ramius,

reaching the 2GB border with altis map should really not be a problem.

Empty mission starts with 1.1GB (client), small missions starts with ~1.3GB, and dependend of the mission and player number it

normally slowly grows to above 2GB (max. seen by me 2.4GB). I use processexplorer from sysinternals for measuring.

Probably the easiest thing is to put a soldier on Altis near Kavala, set high object view distance and look around you in a circle. That will force the engine to load the objects and terrain into memory and based on the view distance it shouldn't be hard to reach and break the 2GB barrier. You can also jump through several places on the island which will force loading of even more objects into memory.

@Dwarden,

yes, thats exactly what i have to assume now.

I tested yesterday with both tbb 4.0 and tbb 4.2, more than 1 hour each (maxmem not used/set).

Both runs with perfect FPS normally between 40 and 50 (vsync on).

Max. memory ~2.2 for each of both tests (BECTI 0.9 on altis).

Also, there is no important difference between 4.0 and 4.2, maybe 4.2 is a tick better (just a feeling).

The difference i observed in the last days (problems disappear), seems to be only caused by improvements/changes in the engine (DEV branch).

PRELIMINARY CONCLUSIONS:

tbb4malloc_bi.dll works well enough, currently

dont use maxmem without a need (64bit OS, enough physical RAM)

Makes sense, I'm glad there are no anomalies :).

Tested vanilla TBB 4.0 and TBB4.2 on our dedicated server last night. Three players, 180 AI's in approx 140 groups and a lot of waypoints in Kavala. Server starts with usage of 800mb anf 50 FPS. Players walking in towards the AI in Kavala let's the servers FPS drop to 25-40 FPS and memory usage quickly goes up to 1015MB but seems to stay at this level. No noticable difference between TBB4 and TBB4.2.

Also, there is no important difference between 4.0 and 4.2, maybe 4.2 is a tick better (just a feeling).

The difference i observed in the last days (problems disappear), seems to be only caused by improvements/changes in the engine (DEV branch).

As my conclusion I will confirm the above statement for dedicated servers.

P.S. I would be glad to help with more testing on my Win08 dedicated server!

To confirm this with 100% certainty, you can use Process Explorer (look for Virtual Memory) or VMMap (look for "Total") tools. Both tools are official Microsoft ones from Mark Russinovich and helped me not once to debug memory problems.

Without ArmA 3 loaded:

http://s13.postimg.org/hycur1qmv/Commit_Without_Arm_A3.png (145 kB)

With ArmA 3 loaded at the main menu:

http://s13.postimg.org/9qax61yqf/Commit_Arm_A3_Load.png (177 kB)

ArmA 3 at editor with Altis loaded:

http://s13.postimg.org/sjwq31wyf/Commit_Arm_A3_Editor.png (170 kB)

ArmA 3 while flying around an empty map:

http://s13.postimg.org/x38yoketz/Commit_Arm_A3_Flying.png (163 kB)

You can see that commit goes from 1.8gb to 6.0gb which is a 4.2gb increase.

I know that Dwarden, but thank you for clarifying it some more. I'm not trying to figure out what uses the memory. What I'm trying to get at though is that the game, utilizes 4gb very easily and probably would utilize much more if given the addressing space. I think what we are running into is a situation where the data chunks being streamed are just too large or too numerous to try to be streamed through the file mapping API. With a larger addressing space, e.x. 64 bit addressing, you could pre-load a lot more data and then stream in smaller bits of it using the same non addressing memory mapping you use now or just paging it and calling it when needed using the normal virtual memory system.

Well there's something there that is causing the stutters and it's much more pronounced the faster you are traveling or rather the more data you are forcing through the pipeline. The only thing I can think of is that the non addressed data is trying to be read into virtual memory much faster than it can handle. It's the only link in the chain that I can see causing that issue, unless there's some sort of bug in the malloc or memory manager.

Another thing I noticed is CPU spikes of 5-10% during large stutters which signals to me that large amounts of I/O traffic is being processed as on an empty map with no AI or ballistics or scripts to calculate, there shouldn't really be anything spiking the CPU usage except file operations. The stuttering got so bad that at times I thought the game was going to crash. This is on an SSD with 8gb of ram and a Phenom II 965 @ 3.8ghz.

PCI-E 2.0 GTX 480

Gigabyte GA-MA770-S3 Rev 2.0 AMD 770 chipset

Phenom II x4 965BE C3 revision @ 3.8ghz

Windows 7 64 Home Premium

Stuttering also for me with max viewdistance and objects over 8000m with or without this dll.

With viewdistance no problem but with viewobjects things gona crazy, especially with jet on altis..

My game stop responding and restart after 5 seconds and so on.. also without change direction or view.. for what hw are made those specs? titan x 3 ? ram usage was always over 2-2,5 gb and gpu ram around 1.8

3930k@4.5

32g@2400

asus rampage iv extreme x79

Ocz revodrive 3 x2 pcie 2x

Sli gtx 680 pcie 3x

Win 8.1 pro

Maybe the game need more speed to read objects..

Tonight ill try restore ramdisk also here but my hdd isnt really slow :p

Without ArmA 3 loaded:

http://s13.postimg.org/hycur1qmv/Commit_Without_Arm_A3.png (145 kB)

With ArmA 3 loaded at the main menu:

http://s13.postimg.org/9qax61yqf/Commit_Arm_A3_Load.png (177 kB)

ArmA 3 at editor with Altis loaded:

http://s13.postimg.org/sjwq31wyf/Commit_Arm_A3_Editor.png (170 kB)

ArmA 3 while flying around an empty map:

http://s13.postimg.org/x38yoketz/Commit_Arm_A3_Flying.png (163 kB)

You can see that commit goes from 1.8gb to 6.0gb which is a 4.2gb increase.

Systemwide commit memory is inconclusive, that's what I was trying to say ;-) For example, the system itself may reserve memory for e.g. drivers, especially Realtek ones, or some other applications (Steam UI, TrackIR, ...)

Using 4.2 GB of address space can't obviously happen for 32bit process even with LAA - only 4GB is possible even on 64 bit system.

Can you repeat the tests with the VMMap or Process Explorer, and look at virtual memory of the process itself?

@Windies + @Dwarden, thanks for your interesting questions and clarification.

Your discussion was induction for me, to read this:

http://www.bistudio.com/english/company/developers-blog/85-breaking-the-32-bit-barrier

again and this time without to overlook some important details :rolleyes:

@DarkWanderer,

please read the document linked above (if you did'nt already), this cleared something for me :)

(CreateFileMapping forces the OS to pre-occupy RAM without consuming virtual addresses from arma process,

later the arma process can access this RAM in smal windows, via MapViewOfFile)

Greets,

Fred41

File mappings do not consume virtual memory of the process, yes. But they do consume system's total memory (system commit). Hence, measuring system commit is senseless. The results Windies has provided confirm that - because by them it seems like ArmA process eats 4.2GB of virtual space, which is not possible.

Limits for 32-bit process VM are:

• Without LAA or on "default" 32-bit OS: 2GB
  
• 32-bit OS with 3GB flag and LAA: 3GB
  
• 64-bit OS with LAA: 4GB
  

Basically, that's what Dwarden said:

what you most likely see is the engine utilizing mapping memory w/o address via OS API

I'd replace "most likely" by "definitely" :) To see effect of LAA flag, VM and only VM matters.

... continued later ...

... ups ...

@DarkWanderer, it was never my intention to "insult" you in any way, or to be "sarcastic" (really, why should i?).

Maybe i misunderstood your last post (this is possible because i am not very good in english).

No, it's me who must apologize here. After reading some especially "adequate" stuff, I've misread your phrase in a bad way :) My bad, sorry.

I got the impression, that you mean the following:

The large values for commited system memory (observed by Windies) are only caused by " for e.g. drivers, especially Realtek ones, or some other applications (Steam UI, TrackIR, ...)"

Because of that i assumed, that you didnt know about the linked document.

Thats all. Sorry again.

If you follow this thread, you could notice, that i did some wrong assumtions, related to the reason for the observed problems.

At least for me, such misstakes are normality and not really a problem.

The title of this thread is a few days old and i can't change him now, but i think we should not disrupt this very interesting thread, just because the title is obsolate?

Greets,

Fred41

Well, actually I've had a mention of memory files in that post, but removed it because was not sure about whether it actually takes virtual memory or not. Sure the discussion should continue. It just looked to me as if you tried to support Windies's point of view by pointing to this link.

In any case, the misunderstanding was due to my fault. I apologize.

-----------------------------------------------------------------------------------

Back on the track of discussion, I think it'll be interesting to check to which extent ArmA engine can utilize this "extra" mapping memory. Here Windies's observations are actually coming in handy - an indirect way to check such usage is to subtract the arma3.exe VM size from the total change of system commit (provided only ArmA 3 maps big files).

I've seen mentions that in some cases memory mapped files work slower that the regular file reads. As far as I know, Lock On/DCS series utilise regular file reads, not memory mapping (but still have 4GB file size limit - because of internal 32-bit counters :P). May this be the reason of stutters in some cases?

... thats what i am asking me currently too.

How can we determine, what amount of memory is currently "pre-occupied" via CreateFileMapping?

(hooking kernel32.dll was my first idea, but there should be a more reputable way :)

Back on the track of discussion, I think it'll be interesting to check to which extent ArmA engine can utilize this "extra" mapping memory. Here Windies's observations are actually coming in handy - an indirect way to check such usage is to subtract the arma3.exe VM size from the total change of system commit (provided only ArmA 3 maps big files).

I've seen mentions that in some cases memory mapped files work slower that the regular file reads. As far as I know, Lock On/DCS series utilise regular file reads, not memory mapping (but still have 4GB file size limit - because of internal 32-bit counters :P). May this be the reason of stutters in some cases?

It's a best guess scenario. I'm no programmer by any means, but I understand some things from taking programming courses in high school and college. The way they do their memory management is weird because they would run out of addressing space very quickly if they did it in a normal way, I.E. Pre-loading all assets and paging non essential data and streaming from the page file. I'm sure pre-loading all assets would run into the 6-8gb range, so most likely what they do is then use the file mapping API to keep data ready to stream into memory on demand. This way they can run normal addressing for the process, which is why you only see around 2gb usage for the process but you see a commit charge of 6gb or more.

Also you have to remember that was a short flight, if you look at the screenshots and look at my max commit, I did some earlier testing and the commit charge almost hit 7gb. This is all with an empty map, no assets, just me in an aircraft flying around. I will test it out with a battle going on and see what the commit charge is like.

If this was drivers, why would commit charge increase gradually as I go through different stages of the game, I.E. main menu, editor, then loading into the game? I'm sure some of it is for the drivers, but I highly doubt you can explain it all away with that.

Well, as we found out already, it was not drivers etc... It was files mapped to memory. I've included it in my post initially, but later deleted - because I was not sure about their actual behavior.

While this is certainly not a good way to measure the LAA effects, you've managed to find a good approximation for measuring the total amount of RAM which can be potentially used by the process. I guess, it's a good starting point to look for other optimizations. Maybe there's a way to tweak OS file mapping behavior. I'll see if I can find anything on the topic.

Here's a screenshot of a multi-player server and the current/max commit and process virtual usage for comparison to an empty map in the editor.

http://s10.postimg.org/8u2j84wex/Commit_Arm_A3_Multiplayer.png (161 kB)

It peaked around 7.4gb commit and the process virtual size was almost 2.7gb. That's around 5.6gb commit just for ArmA 3 when subtracting the base 1.8gb commit charge.

... thats interesting.

Just a question, you wrote "screenshot of a multi-player server" and a client is showed of the desktop and in the process explorer window.

Are there both server and client on the same system, or is it only client connected to a remote server?

Here's a screenshot of a multi-player server and the current/max commit and process virtual usage for comparison to an empty map in the editor.

http://s10.postimg.org/8u2j84wex/Commit_Arm_A3_Multiplayer.png (161 kB)

It peaked around 7.4gb commit and the process virtual size was almost 2.7gb. That's around 5.6gb commit just for ArmA 3 when subtracting the base 1.8gb commit charge.

It's me in a windowed client measuring my memory usage while I'm connected to a server. I don't have a way to measure server performance right now, though I would be very interested in seeing the memory usage of the dedicated server.

... you could start a dedicated arma server on the same system as your arma client, thats my current testing setup ...

If you do so, an entry in server.cfg, like this:

localClient[] = {127.0.0.1};

helps maxing out the network bandwidth.

But server performance is impacted a bit, because this two processes have to share the available CPU cores and physical RAM.

It's me in a windowed client measuring my memory usage while I'm connected to a server. I don't have a way to measure server performance right now, though I would be very interested in seeing the memory usage of the dedicated server.

If I ran them both on the same machine though I would have no conclusive way of measuring the amount of memory each program uses.

BTW DarkWanderer, DCS went 64 bit A LONG time ago, like back when DCS A-10C came out. In fact I don't even think they maintain a 32 bit binary anymore and they're system reqs list a 64 bit OS as a minimum requirement.

... yes, thats true too.

I checked the microsoft api's, but didn't found a reasonable way to determine the ammount of system memory, occupied indirectly via CreateFileMapping by a specific process.

But i think, it could make sense to monitor the (hard) pagefaults counter for the arma (server) process, to verify if a performance problem (lag) is related to memory allocation/paging.

Maybe i will soon add a counter PF/sec to the monitored processvalues in ArmaServerMonitor.

Greets,

Fred41

If I ran them both on the same machine though I would have no conclusive way of measuring the amount of memory each program uses.