RAM Management - Serious Question

[I give up 152]

I raised the subject couple of times and I am sorry but I have to raise it again because its has a severe impact in game performance.

 

As a matter of demonstration and also in a attempt to avoid useless discussions, I made a video that clearly demonstrates what I have been saying.

In this case (trying not to make the video to long) I stopped on 13 GB of RAM loaded, but it can load much more.

 

Also the most weird is if we close the scenario and open a new one, the RAM loaded in previous one is still here and keeps loading more.

 

Only when we change back the textures from Ultra to High is when the RAM gets "flushed".

Why this happen? Is there a way to avoid it? Because it has a nasty impact in performance.

 

https://www.youtube.com/watch?v=pA8WVJBCEiI

[djotacon 190]

...

[St. Jimmy 272]

Have you tried if this happens with page file enabled? Would be interesting to hear.

[I give up 152]

Have you tried if this happens with page file enabled? Would be interesting to hear.

Thats not the point, besides the game only starts to use pagefile when you are out of ram. 

Also, everything that goes to pagefile performs even worse, no matter if is a SSD or whatever it will perform worse than the slowest RAM module.

The point is, no freakin game should use this insane load of ram.

And my issue is not because the ram that the game is using, (I have 32 GB), is because the freakin engine after 4 GB (approximately) is unable to use it properly and under acceptable timings.

Can we have this fixed?

[Greenfist 1863]

It would be interesting to see your process list from task manager with all the memory columns visible.

[St. Jimmy 272]

So -maxMem parameter doesn't prevent this happening?

So it's loading textures in the RAM or something related. I've tested the same thing couple of times but I've only 8GB of RAM so I've only got the RAM usage somewhere above 3GB (maybe 3,5GB) when I flew with the camera 20mins around the Altis.

Yeah could you post Resource Manager stuff how much RAM is shared, commited etc? That would be very interesting.

[I give up 152]

So -maxMem parameter doesn't prevent this happening?

So it's loading textures in the RAM or something related. I've tested the same thing couple of times but I've only 8GB of RAM so I've only got the RAM usage somewhere above 3GB (maybe 3,5GB) when I flew with the camera 20mins around the Altis.

Yeah could you post Resource Manager stuff how much RAM is shared, commited etc? That would be very interesting.

-maxMem parameter is related with arma3.exe, only. Since the memory load is not related with it, the parameter does not affect at all.

 

That's most likely because you dont have GPU power enough.

Also, with 8GB of RAM you are forced to have pagefile enabled, would be interesting to see your pagefile load, repeating the steps and having the graphics in Ultra.

 

About resources or processes, having the pagefile enabled we see the virtual memory usage in the Kernel process, means if we have loaded 8 GB in to pagefile the Kernel process is using 8 GB (approximately).

With pagefile disable there is no resource or process showing the RAM loaded, loads directly in to RAM, God knows how.

 

And yes, is the textures that are being loaded, means more quality more RAM loaded (in your case more pagefile loaded).

 

Besides, I am tired of repeating the same over the last year and having useless debates with people that do not have similar hardware, consequently having no clue of what is being said. 

That's why I have made the made, its for developers, I am quite sure that they know what I am saying.

[St. Jimmy 272]

I've been thinking to upgrade my RAM to 16GB so I'm bit interested what happens in Arma 3.

You can read some of my results from this post with page file naturally system managed with the 8GB of RAM.

https://forums.bistudio.com/topic/186732-amd-cpu-bottleneck/page-3#entry2967561

[I give up 152]

I've been thinking to upgrade my RAM to 16GB so I'm bit interested what happens in Arma 3.

You can read some of my results from this post with page file naturally system managed with the 8GB of RAM.

https://forums.bistudio.com/topic/186732-amd-cpu-bottleneck/page-3#entry2967561

Afterburner reads pagefile.

And your readings are quite conclusive.

Pagefile = 7747MB usage

RAM = 3316MB usage

7747 + 3316 = 11063 MB of memory loaded and in use.

 

I dont see where is the surprise when I am showing 12.5GB of memory (RAM) loaded and in use.

Is basically the same, in your case textures are being loaded to pagefile (because you dont have RAM enough)m in my case textures are being loaded to RAM (because I have RAM enough).

 

And this is just insane, there is no chance for a game like this to work properly with such load in to memory.

There is no way for the memory to be refreshed (something that is continually required) under acceptable timings, there is no way for having an acceptable performance, only by itself is one major reason for CPU/GPU bottleneck.

[Freghar 73]

I raised the subject couple of times

On the bug / feedback tracker?

 

because its has a severe impact in game performance.

Any proof of that? Normally, high memory usage indicates heavy caching = less waiting for disk I/O or page faults. At the very least, it's not an indication of "performance" as in "FPS".

 

Also the most weird is if we close the scenario and open a new one, the RAM loaded in previous one is still here and keeps loading more.

That's not weird at all - the memory is not actually loaded by the .exe, it's cached by the system. ARMA actually uses the pagefile API to make the system do that (instead of using the memory locking API or actually being a 64bit app), read more on https://www.bistudio.com/blog/breaking-the-32-bit-barrier .

 

Thats not the point, besides the game only starts to use pagefile when you are out of ram.

Not true, see above.

 

Also, everything that goes to pagefile performs even worse, no matter if is a SSD or whatever it will perform worse than the slowest RAM module.

That's why I have my pagefile on a ramdisk and can definitely see faster mission load times than without pagefile (still on rotational HDD here).

 

The point is, no freakin game should use this insane load of ram.

And my issue is not because the ram that the game is using, (I have 32 GB), is because the freakin engine after 4 GB (approximately) is unable to use it properly and under acceptable timings.

Any game can use any amount of ram it wants to get the job done - if you selected Ultra everything, expect Ultra resource usage.

Timings (as in memory timings) have absolutely nothing to do with this - the game cannot possibly use all the RAM at once. Actually, frequent memory access would be a much bigger problem than lots of RAM used.

 

-maxMem parameter is related with arma3.exe, only. Since the memory load is not related with it, the parameter does not affect at all.

 

That's most likely because you dont have GPU power enough.

Also, with 8GB of RAM you are forced to have pagefile enabled, would be interesting to see your pagefile load, repeating the steps and having the graphics in Ultra.

I used to run with 2GB of RAM on win7 with pagefile disabled. :)

My personal findings can be found in that other thread, I repeated them with everything set to max distance and Ultra (even under the post processing tab), after a game restart (to be sure):

• red = read/written pages per second (unit = 1 page)
• blue = free memory (unit = 10^-7 bytes, eg. 800 = 8GB)
• green = page file usage (unit = 10^2 %, eg. 100 = 10%)

The first graph is from me trying the Zeus 4+1 bootcamp, so I could fly around. Turns out the mission had some overcast which cannot be disabled via Zeus, so I loaded an editor mission with a littlebird and spent a few minutes flying (second graph), with full 12km distance. Both high and low altitude. I even spawned several full squads of units fighting.

As you can see, with my 16GB RAM (6GB of which is dedicated to ramdisk with pagefile), I cannot reproduce the issue. I was able to get (in the worst case) 3.4GB of RAM usage and only few % of pagefile above that (6-8% change out of 6GB). Similarly to my previous graphs - you can see Windows clearing up RAM and putting the pages in the pagefile, at a rate of ~256 pages/sec (red line).

These readings are from the native measurement tool that comes with windows and uses kernel hooks to get the data. You cannot get more exact than this.

 

About resources or processes, having the pagefile enabled we see the virtual memory usage in the Kernel process, means if we have loaded 8 GB in to pagefile the Kernel process is using 8 GB (approximately).

Where exactly do you see that? I'm no Windows expert (more of a Unix one), but that's nonsense - there's no reason for "paged out" pages to be showing as the virtual memory - the former is a level higher than the latter.

It would make sense with pagefile disabled, though, as Windows has to somehow simulate the paging API even when there's no paging file.

 

And this is just insane, there is no chance for a game like this to work properly with such load in to memory.

There is no way for the memory to be refreshed (something that is continually required) under acceptable timings, there is no way for having an acceptable performance, only by itself is one major reason for CPU/GPU bottleneck.

Eh, what?

Besides, I am tired of repeating the same over the last year and having useless debates with people that do not have similar hardware, consequently having no clue of what is being said.

I am also tired of people, who pretend to know low level memory signalling and try to somehow tie it to used memory as shown by the OS. Do you even know how page tables work on x86?

Besides, even if the high memory usage is real, there's probably nothing BI can do about it - it's the Windows kernel doing its own thing. Who knows what that extra usage is? It might be page cache, eg. essentially free memory used for write-through caching disk contents (IIRC Windows calls this SuperFetch) and MSI counting that incorrectly as "used". True, it might be ARMA textures loaded "just in case" in the background.

The point is - you need a pretty reliable reproducer, a proof that what MSI shows is a correct value, find out what the usage actually is and prove that it has a negative effect on anything, .. before developers start investigating your issue, I imagine.

[ruPal 143]

 

That's why I have my pagefile on a ramdisk

 

How did you do that? Ram disk is initialized after pagefile, so I couldn't move it there. I tried to move it in Windows but... you have to restart Windows and then "Ram disk is initialized after pagefile". May you please give me instructions on that.

Btw, I opened a ticket on feedback tracker about memory leak. If you have small page file then game crashes with virtual memory errors. It was a while a ago, but BI answered that they don't know what cause that problem.

[old_painless 182]

Freghar, could you share the name of "the native measurement tool that comes with windows and uses kernel hooks to get the data" ?

 

Cheers,

OP

[Freghar 73]

How did you do that? Ram disk is initialized after pagefile, so I couldn't move it there. I tried to move it in Windows but... you have to restart Windows and then "Ram disk is initialized after pagefile". May you please give me instructions on that.

I use SoftPerfect RAM disk software, which allows me to use a "boot" ramdisk, apparently set up before Windows initializes paging files. I have never investigated why it makes repeated mission load times faster for me, there are multiple possibilities, but no time to dig deeper, though. :(

 

Btw, I opened a ticket on feedback tracker about memory leak. If you have small page file then game crashes with virtual memory errors. It was a while a ago, but BI answered that they don't know what cause that problem.

As long as it's reproducible, great. The more pressure we can make for using 64bit, the better. :ph34r:

bratwurste's "issue" might also be a real one, the thing is there's a lot of presumptions about what causes performance issues and what doesn't and that high usage might actually be by design, to speed things up. If it is real and not imagined by MSI - would be useful to know *what* that number measures.

 

Freghar, could you share the name of "the native measurement tool that comes with windows and uses kernel hooks to get the data" ?

 

Cheers,

OP

It's mentioned in the other thread, perfmon.msc. It's unfortunately localized to the language of Windows, which is why I posted text translation of the legend instead of including it in the screenshot.

I was really surprised what you can measure - we have "perf" on Linux which can do similar things, but I never really expected something like that from Windows. :)

[old_painless 182]

Thanks - I actually knew that application from Windows Server. Nice to see that it has been carried over to humble Windows 10 Home :-)

 

Btw, theres a huge amount of I guess "hardcore" system tools for Windows - one site that has always supplied quality is sysinternals.com. I think there were bought by Microsoft at one time

 

Edit: They were: https://technet.microsoft.com/en-us/sysinternals/bb545021.aspx

[Freghar 73]

Okay, so I managed to (mostly) localize perfmon to English by copying a few files around and did a few more measurements,

("Měřítko" means multiplier.)

Apparently, Windows has one additional "layer" between allocated ("reserved") and actually used (COW'd pages) memory - "commited" memory, defined as

Committed Bytes is the amount of committed virtual memory, in bytes. Committed memory is the physical memory which has space reserved on the disk paging file(s).

The amount of Commited memory did indeed go up to ~15GB, however note that this is not memory that's actually used (and also that it was at 9GB before Arma started, probably the ramdisk).

Am I reading something wrong?

edit:

https://support.microsoft.com/en-us/kb/2160852

This shows how many bytes were allocated by processes and to which the operating system has committed a RAM page frame or a page slot in the pagefile (or perhaps both). As Committed Bytes grows greater than the available RAM, paging will increase, and the pagefile size that is being used will also increase. At some point, paging activity starts to significantly affect performance.

So it's unclear to me whether the commited memory is used (but paged out) or just "more reserved". In any case, (RAM total - RAM available) + pagefile usage are still below what's considered commited, so I'm inclined to believe the space is not actually used.

[I give up 152]

@ freghar

 

Are you speaking about ARMA 3 or something else?

Why you are referring pagefile countless times? Where in my video you see pagefile load? It shows only RAM.

 

Do you know the impact on performance when you are loading tons of textures at one time directly in to RAM, do you know impact of that in CPU and GPU? If you dont, I suggest you to look at video. Or do you think that the RAM performance is  not a crucial factor in this game? And I am not speaking about RAM tmings, latency or even speed. Do you know what is Dynamic Random Access Memory? Or Memory Refresh. Do you know why Memory (physical and virtual) have different llimits (in matters of usage) in 32 and 64 bit systems and / or applications?

 

Pagefile on a RAMDISK? Hilarious. So you have RAM enough (it seems) and you prefer to use it as pagefile (as virtual memory instead of using it as "true memory"?. And you are saying system is faster on that way? Just hilarious.

 

I am perfectly aware of "32 bit breaking barrier" and that according to the notes released is only related with virtual memory, I have my opinion about it but this not the proper place to speak about.

 

Understand this.

 

When the game is loading and using 10 GB of virtual memory (pagefile) is quite clear where, how and with is being used, we just need to look at Kernel to understand all that. And that is what "32 bit breaking barrrier" is about, if was not it never in this life a 32 bit application could load 10 GB of virtual memory.

 

Now when the game is loading and using 10 GB physical memory (RAM) we dont have any trace about where, how and with is being used. NOTHING. 

We know that the RAM are being used, we know that the RAM has been loaded by the game, but we dont have a clue how and where is being managed and / or used. I my opinion is not being managed at all since the system, processes, resources do not have it in use, its a like is being used by a ghost.

 

But honeslty, I perfectly understand your position, It has become quite clear when you said that you can load (play) the game with 2 GB of RAM having the pagefile disabled..

 

Btw, is not like "any game can use the amount of RAM it wants", there are limits, in first place dictated by the system and application x32 or x64 and in second place is dictated by the hardware or its capacities. Do you understand what I am saying?

 

Edit. (because on that wall of text I have missed it)

You have superfetch enabled?

[Freghar 73]

Are you speaking about ARMA 3 or something else?

Yes, of course.

 

Why you are referring pagefile countless times? Where in my video you see pagefile load? It shows only RAM.

Because that's how Arma 3 memory management seems to work, regardless of whether you have it enabled or not - it uses the file mapping API and the OS deals with it somehow, whether it has a paging file or not. I also tend to be skeptical about programs showing one RAM value without saying what it actually measures - allocated virtual memory or pages with data?

 

Do you know the impact on performance when you are loading tons of textures at one time directly in to RAM, do you know impact of that in CPU and GPU?

If you dont, I suggest you to look at video. Or do you think that the RAM performance is not a crucial factor in this game?

And I am not speaking about RAM tmings, latency or even speed. Do you know what is Dynamic Random Access Memory?

Or Memory Refresh.

Do you know why Memory (physical and virtual) have different llimits (in matters of usage) in 32 and 64 bit systems and / or applications?

I don't exactly know the impact, I haven't measured it and I presume you haven't either. Game shuttering isn't a valid measurement of RAM usage - one would have to record the shuttering separately and match it against RAM access data, collected at much finer granularity than I do (and at MMU level). Then, one would have to look at the cause for the shuttering (nr. of draw calls?) and prove that RAM speed is indeed the limiting factor. It might very well not be.

Yes, I know that DRAM/SDRAM is. I have soldered a few. I also am aware of what is memory refresh and that it becomes less significant with higher memory usage (ironically). ;)

And yes, I know the difference between 32 and 64bit virtual addressing space.

 

Pagefile on a RAMDISK? Hilarious. So you have RAM enough (it seems) and you prefer to use it as pagefile (as virtual memory instead of using it as "true memory"?. And you are saying system is faster on that way? Just hilarious.

You seem to misunderstand what virtual memory is - it's not pagefile, it's just an abstraction above RAM+pagefile, so that the application doesn't have to deal with where its data is stored (simply said).

And yes, for some weird reason, I don't get as much shuttering ingame - hard to say why. It may be some specifics of Windows memory management or how Arma works with the file mapping API. It's not magic and probably has some explanation, but I don't have enough time, interest or source code to dig deeper.

Without paging file, I get http://i.imgur.com/Kk9KiDY.jpgfor a few seconds when I quickly look around. With the pagefile, I get it only once per mission.

You're kind of right that it doesn't make much sense as (on Linux, but Windows may do it similarly) you can only use "swap" (paging file) for anonymous memory, not file-backed, kind of due to how page tables work. But again - the MMU just calls the page fault handler, which differs per OS.

 

I am perfectly aware of "32 bit breaking barrier" and that according to the notes released is only related with virtual memory, I have my opinion about it but this not the proper place to speak about.

Well, it's kind of related to everything due to how memory-mapped files work as they're (presumably) the reason behind this topic.

 

When the game is loading and using 10 GB of virtual memory (pagefile) is quite clear where, how and with is being used, we just need to look at Kernel to understand all that. And that is what "32 bit breaking barrrier" is about, if was not it never in this life a 32 bit application could load 10 GB of virtual memory.

The game is not loading 10GB of virtual memory as it has only (up to) 4GB. Similarly, as the pagefile can only contain paged out anonymous memory for the process, it can never have more than (up to) 4GB of data for that one 32bit process (unless Windows does mm in some very weird way).

 

Now when the game is loading and using 10 GB physical memory (RAM) we dont have any trace about where, how and with is being used. NOTHING.

Right, because it's not. IF indeed 10GB of physical pages is used, it's not loaded by the game. The game can only map small chunks of files into its memory at a time as it needs to fit the (2-3GB) free virtual memory addresses. When it unmaps the chunk(s), it's up to the OS what it does with the data.

 

We know that the RAM are being used, we know that the RAM has been loaded by the game, but we dont have a clue how and where is being managed and / or used. I my opinion is not being managed at all since the system, processes, resources do not have it in use, its a like is being used by a ghost.

We don't know the RAM is being used, we know that some program displayed a nice big green number without telling us what it is.

We don't have a clue because we (you?) haven't investigated enough into what the memory is actually used for.

 

But honeslty, I perfectly understand your position, It has become quite clear when you said that you can load (play) the game with 2 GB of RAM having the pagefile disabled..

I didn't say I can load (play) the game, I said

I used to run with 2GB of RAM on win7 with pagefile disabled. :)

implying that I used to run (Windows) with pagefile disabled. Actually, I was running Operation Flashpoint back then and without Windows Aero and other visual mess, 2GB was very much enough.

 

Btw, is not like "any game can use the amount of RAM it wants", there are limits, in first place dictated by the system and application x32 or x64 and in second place is dictated by the hardware or its capacities. Do you understand what I am saying?

It was more of a philosophical statement, as in - any game can use whatever hardware resources it wants, but its commercial success will be dictated by how many people can actually run it. As in - the game knows best how much RAM/VRAM it needs for the textures and if you want the best ones it has, you'll need to meet the RAM requirements (or suffer slowdowns).

 

You have superfetch enabled?

No, I don't. Page cache seems to be working without it and SuperFetch (according to Microsoft) seems to be just boot-time preloader + some algorithm that tracks what you do and pre-loads files into page cache based on time of day and that kind of stuff. As I use Windows only for games, I don't start it regularly enough for such heuristic to have a meaningful effect.

PS: I've done some additional measurements, both with pagefile disabled, the first one with my 6GB ramdisk still active, the second one without it (~16GB RAM free initially). To make things interesting, I added per-process statistics (arma3.exe) for both (cuts into the first one as I added the counters, the second graph has them already set up).

As you can see, even without the ramdisk and pagefile, only about ~3GB of memory was really used. The rest is reclaimable for other purposes by the OS and is "used" only for optimization purposes (which is a good thing). And even accounting for this, only about 8GB of physical pages was actually used overall, incl. processes other than Arma.

So no, I cannot reproduce your "issue" no matter how hard I try.

[I give up 152]

Not going to answer to this nonsense statements. I refuse to be trolled.

[heavygunner 179]

Bratwurste always refuses to acknowledge things if you have actual proof.

Just like in here: https://forums.bistudio.com/topic/156993-arma-3-cpu-vs-ram-performance-comparison-1600-2133-up-to-15-fps-gain/?p=2895044

[I give up 152]

Bratwurste always refuses to acknowledge things if you have actual proof.

Just like in here: https://forums.bistudio.com/topic/156993-arma-3-cpu-vs-ram-performance-comparison-1600-2133-up-to-15-fps-gain/?p=2895044

One more expert. even making fool of himself.

In the thread linked, I was saying that a SSD (just because pagefile) would drastically increase performance in this game, and some were saying that was false and was only about RAM speed.

This one in this thread keeps talking about about pagefile (he has it on RAMDISK as a matter of performance for ARMA 3) and how it is important for the game.

Freaking clueless trolls.

[Freghar 73]

Bratwurste always refuses to acknowledge things if you have actual proof.

Just like in here: https://forums.bistudio.com/topic/156993-arma-3-cpu-vs-ram-performance-comparison-1600-2133-up-to-15-fps-gain/?p=2895044

Thanks for the link. I kind of figured out he was a troll few posts ago, but I still wanted to learn a bit more about Windows memory management and this was a nice opportunity. Turns out it's pretty similar in basic concepts to how we do things in the Linux kernel.

The funny thing is that I actually did the benchmarks without pagefile or ramdisk in the end, both of which he seems to recommend. :)

FTR - SSD would help initially, because the mapped files aren't yet in the page cache. Assuming the page cache is big enough (MSDN seems to mention some problems Linux doesn't have thanks to the "swappiness" tunable :) ). The interesting thing is that NT kernel apparently shows page cache as "free" memory. Go figure.

[I give up 152]

Thanks for the link. I kind of figured out he was a troll few posts ago, but I still wanted to learn a bit more about Windows memory management and this was a nice opportunity. Turns out it's pretty similar in basic concepts to how we do things in the Linux kernel.

The funny thing is that I actually did the benchmarks without pagefile or ramdisk in the end, both of which he seems to recommend. :)

FTR - SSD would help initially, because the mapped files aren't yet in the page cache. Assuming the page cache is big enough (MSDN seems to mention some problems Linux doesn't have thanks to the "swappiness" tunable :) ). The interesting thing is that NT kernel apparently shows page cache as "free" memory. Go figure.

Yes, and what is your issue?

Like I said I am perfectly aware about how the game operates with pagefile and how it can be helpful having it enabled in the fast way we can.

 

Now, in this thread where do you see me referring any issue with pagefile?

The issue that I am reporting is purely related with RAM load and management.

 

Did you watched? Did you see how the RAM load went from 12GB to 6GB soon I changed the graphics settings from Ultra to High? Obviously not.

And dont worry about measuring tools, I just made it simple so people like you could understand.

 

Now. and since you keep referring pagefile (not virtual memory, according to you), where it says in game requirements or any released note, that I need to have it managed by the system?

Where it says that I cant have it set with a limited size (the minimum required by the system).

Where it says that I need pagefile to play this game?

Now, go troll elsewhere, because this thread is about physical memory, not virtual.

[I give up 152]

https://www.youtube.com/watch?v=bg2Y8T4-F1Q

 

AA 4x

FXAA High

Aniso High

HBAO+ Standard

Atoc Arrowhead trees+grass

[Freghar 73]

To repeat the main point again - it's not about seeing a big "RAM: 12345 MB" number, screaming "TOO MUCH!!!", it's about understanding what that number refers to, what it actually measures. If it is even correct in the first place. You would do that by using some other system performance measurement tools that tell you know where your RAM is spent.

Aside from perfmon, I was able to quickly google out http://www.microsoft.com/whdc/DevTools/Debugging/default.mspxand http://www.microsoft.com/whdc/DevTools/tools/DrvVerifier.mspx , for example.

If you just keep pointing to the big obvious number, you'll get nowhere.

[St. Jimmy 272]

Could you post this kind of picture from resource manager?

And didn't you say you're not using page file? Well then you should get load in your RAM. It has been pointed many times in these years that if you've low or disabled page file and you don't have enough RAM, the game crashes because of "out of memory". The interesting part in here is that you seem to actually maybe see the RAM load, when usually nobody hasn't seen any big RAM loads. But usually those issues has been posted by people that have 8GB or less and you can get the RAM load to 6GB pretty easily so I guess there isn't "enough free room".

@Freghar

Why are you using RAM disk for page file? I've read there's no real point to do that. You could just disable it and have the same effect or is there some other wizardy going on?