Jump to content
Sign in to follow this  
galzohar

Windows File Caching and More RAM

Recommended Posts

I've seen many times mentions of ArmA 3 nowdays using the windows file caching in order to more effectively stream data from the hard drive and use more than 4GB of RAM even though the application is limited to 4GB, since it is not using that RAM directly but is rather using it through the OS file caching system.

However, I haven't seen many mentions of whether this is actually effective enough to the point where having enough RAM on your system will remove the need for an SSD/RAMDISK due to this file caching, or whether the file caching is not effective enough regardless of how much RAM you have and you still need an SSD/RAMDISK to remove stutters for streaming new data from your HDD.

The reason I ask is because I just happened to upgrade my GPU to a GTX 770, but could not play smoothly with high settings because whenever I drive around Altis with those settings (3000 object view distance and very high settings, which work find while standing still), my FPS drops to a crawl (3-7 FPS), probably due to getting stuck loading too much data all at once from the HDD. Teleporting myself back and repeating the test on the same mission (without exiting or resetting) gave the same results, probably meaning the data just wasn't saved in RAM. I was wondering if more RAM would make windows cache more data in RAM for faster loading, or if an SSD/RAMDISK is required.

I currently have 4GB of RAM, and due to my old motherboard (EX58-UD3R, i7-920) upgrading it is a bit complex and might be a waste of money as I'll probably have to just buy new RAM all over again when I replace my entire PC 1-3 years up ahead. However adding 2GB is much cheaper than buying an SSD and even 12GB (my MB is triple channel) can have a similar price to an SSD with potentially better performance if file caching is indeed used effectively by ArmA 3 nowadays.

Thanks for any help!

Edited by galzohar

Share this post


Link to post
Share on other sites

From my understanding, ARMA only loads assets in RAM when it needs them, and doesn't cache to RAM. I do believe it stores assets in the page file and loads them into RAM when they are needed. I could be wrong, but I THINK that's how it is.

Share this post


Link to post
Share on other sites
From my understanding, ARMA only loads assets in RAM when it needs them, and doesn't cache to RAM. I do believe it stores assets in the page file and loads them into RAM when they are needed. I could be wrong, but I THINK that's how it is.

Memory access doesn't work like that.

ArmA, nor any application running on Windows, can't decide whether it uses page file or actual RAM. That is up to the OS. The process (ArmA) doesn't know where its actual memory address resides.

Share this post


Link to post
Share on other sites
Memory access doesn't work like that.

ArmA, nor any application running on Windows, can't decide whether it uses page file or actual RAM. That is up to the OS. The process (ArmA) doesn't know where its actual memory address resides.

Ahh ok. Well, either way it seems assets are being cached into the page file, which can cause the game to run slow when it needs assets that aren't yet loaded into RAM. I suppose a way around this would be to move the page file onto a RAM disk, but I'm not sure if that would affect the stability of anything. Also, since the page file gets loaded before the RAMdisk would, the page file would have to be created everytime the computer booted back up.

Share this post


Link to post
Share on other sites

Pagefile on ram disk is useless. Imagine the Pagefile as a virtual Memory using HD because you don't have enough. You won't say "I have no memory left so I put it on my memory".

Pagefile must be needed also for deep System sleeping (when memory is not supported by power anymore, it looses it's content)

Share this post


Link to post
Share on other sites

Can we stay on topic please? The title is clearly about Windows file caching, not page file.

Share this post


Link to post
Share on other sites

I would buy a small SSD 120GB, or even 64GB. They are pretty cheap now. I have a system with 16gigs of RAM, and I normally keep steam games on of a 2 drive 2TB raid 0. But arma 3 is one of those games I moved and is sitting off an SSD. It did wonders to get rid of the stutter. Adding RAM would help caching if you cross the same area back and forth. But if it has to fetch off of the drives it would have to wait on the latency of your hard drives. This causes the stutter. An SSD access to the data is lot quicker than a mechanical driver.

I like flying in arma 3. So I was always covering new terrain. I wasn't happy until I moved it to the SSD even with 16 gigs of RAM. I even made a ram drive one day. It wasn't that noticeably faster than the SSD.

Share this post


Link to post
Share on other sites

Well, it seems like the game IS reading the same data over and over. Especially in my test where I was driving through the same area again and again each time with different graphic settings, without restarting the mission. If the objects/textures were cached properly, or if the files from which they are read were cached by windows, then I would not have had FPS drops as the game had to load them while I was driving through the town.

If you still have stutters with 16GB without SSD, though, then I guess windows file caching just isn't good enough for this purpose, or just isn't used properly by ArmA 3. After all, 16GB would be enough to cache 100% of the game files in RAM and still have more than enough for the game itself, windows and other random software. If this really is the case (caching not solving the issue regardless of how much RAM you have), it means an SSD really is necessary...

I've seen a post for ArmA 2 which was checking a lot of statistics for HDD action while playing, and it seems like the maximum spikes in MB/s read is mostly lower than the MB/s a reasonable SSD can read with random access of 4kb blocks, which explains why a ramdisk didn't improve much - As an SSD is already fast enough, the much greater speed of the ramdisk is just not that noticeable as it's simply not needed, at least in most cases that were tested in that test. When that test was made, though, ArmA 2 did not use any of window's file caching, which is why I needed some answers for it using current ArmA 3 version.

Edited by galzohar

Share this post


Link to post
Share on other sites

Just wondering would these have any bearing on caching ?

GPU_MaxFramesAhead

GPU_DetectedFramesAhead

Share this post


Link to post
Share on other sites
Just wondering would these have any bearing on caching ?

GPU_MaxFramesAhead

GPU_DetectedFramesAhead

No, they're related to how many frames the GPU can render ahead.

Share this post


Link to post
Share on other sites

@galzohar, i think both, your 4GB and your slow HDD is bottlenecking ...

Arma is currently not using windows system file caching mechanism.

Instead, it uses windows file mapping API, to store some last recently used data in memory mapped pagefile.

This means, for optimal performance, a arma system should have enough physical RAM, to avoid IO traffic to/from pagefile

and it should have fast IO to provide fast file reading (especially for files from arma addons folder).

Default setting, maxmem=2048, results in around 1.7GB extra memory/pagefile usage (outside of arma process address space).

I would say at least 6GB physical RAM (more is better) is required, to avoid pagefile IO in most scenarios.

So my suggestion: 1. upgrade to 12GB (triple channel) and 2. buy a small, cheap SSD (64GB) to place armas addon folder here.

(I hope your OS is already 64bit?)

Greets,

Fred41

Edited by Fred41

Share this post


Link to post
Share on other sites

I would do the reverse. Buy an SSD first and if you can afford it buy RAM. New RAM probably won't be as portable to another system if you ever buy a new one. You can easily move an SSD to a future system.

You can even move you page-file to the SSD though lots of writes would cause a cheap SSD to fail sooner.

Just to see how much disk access normally occurs, I opened up "Resource Monitor" to watch the activity on my SSD drive. Arma and a couple of other games are the only things loaded on that SSD. It is different from my OS/boot SSD. I loaded up stratis in the editor, got in pawnee and just flew around. It reads quite a bit from the add-on folder particularly as you enter different areas. It quieted down after flying around a bit, but I can still see it hitting the SSD with bursts on occasion. Again I have 16gigs of RAM.

Share this post


Link to post
Share on other sites

Lot's of games need more than 4GB RAM. Not sure how your MB works, but is it the performance hit that big if not running triple channel? More RAM always was and should still be, better than faster timings or double/triple or other marketing stuff. Adding 4GB more for a total of 8GB shouldn't be that expensive and will bring some life to that system.

FYI, I get BSOD on some games due to lack of RAM if I disable page file with only 8GB RAM.

PS: Runs relatively okish with very high settings, 3000km for object distance and 5040x1050 on 8GB RAM and 7950OC, 2500k OC (30FPS more or less), in editor while driving around Altis and the big city to the west. So yes, you NEED more RAM.

Share this post


Link to post
Share on other sites
Lot's of games need more than 4GB RAM. Not sure how your MB works, but is it the performance hit that big if not running triple channel? More RAM always was and should still be, better than faster timings or double/triple or other marketing stuff. Adding 4GB more for a total of 8GB shouldn't be that expensive and will bring some life to that system.

FYI, I get BSOD on some games due to lack of RAM if I disable page file with only 8GB RAM.

PS: Runs relatively okish with very high settings, 3000km for object distance and 5040x1050 on 8GB RAM and 7950OC, 2500k OC (30FPS more or less), in editor while driving around Altis and the big city to the west. So yes, you NEED more RAM.

That's because there are other applications running in the background that require memory too, and you're practically starving them by disabling page file ;)

Share this post


Link to post
Share on other sites

Nope, only the games and Steam or U Play and MSI Afterburner. 4GB RAM is not enough for a gaming PC anymore.

Share this post


Link to post
Share on other sites

Arma is currently not using windows system file caching mechanism.

Well, that sucks and pretty much sums this thread up. An SSD is a must. More RAM might help, but no matter how much RAM you have, without file caching there's no way you'll get good performance without an SSD.

Share this post


Link to post
Share on other sites
Well, that sucks and pretty much sums this thread up. An SSD is a must. More RAM might help, but no matter how much RAM you have, without file caching there's no way you'll get good performance without an SSD.

Its true arma3 doesn't use pagefile. I have 8GB RAM, arma3 still runs well when I disable the pagefile. And you will much different when teleport yourself on the map between an installation on HDD and SSD. So you have to buy a SSD if you want to obtain smooth game experience.

Share this post


Link to post
Share on other sites
Its true arma3 doesn't use pagefile. I have 8GB RAM, arma3 still runs well when I disable the pagefile. And you will much different when teleport yourself on the map between an installation on HDD and SSD. So you have to buy a SSD if you want to obtain smooth game experience.

That's because ArmA 3 only uses 3 GB at most since it's a 32 bit executable and that ArmA 3 doesn't decide whether it uses page file or not. That is up to the OS. If you disable the page file... well, of course the OS will not allocate any memory on it nor put anything on that said memory.

I feel like a lot of people on this forum don't understand how memory management works on OS level nor on process level.

Share this post


Link to post
Share on other sites
That's because ArmA 3 only uses 3 GB at most since it's a 32 bit executable and that ArmA 3 doesn't decide whether it uses page file or not. That is up to the OS. If you disable the page file... well, of course the OS will not allocate any memory on it nor put anything on that said memory.

I feel like a lot of people on this forum don't understand how memory management works on OS level nor on process level.

You do realize the topic is about windows file caching and not about how the game manages its own RAM, right?

If you don't know the difference, please read around, because the questions you are answering are not the questions that were being asked.

Share this post


Link to post
Share on other sites
You do realize the topic is about windows file caching and not about how the game manages its own RAM, right?

If you don't know the difference, please read around, because the questions you are answering are not the questions that were being asked.

No, but if you read closely, you can see Lao Fei Mao talking about page file.

Also all data on Windows is read from the system cache, which in turn reads from the disk. User space applications can only turn off caching and/or enable immediate flushing to disk. In the latter case, the system cache still caches the data.

Share this post


Link to post
Share on other sites

Yes. The real question though is if/why ArmA turns off the caching. I don't know of anyone who could come up with a reason.

And just because someone went off-topic talking about page file which is completely irrelevant to the topic doesn't mean everyone else have to join in and derail the thread. Especially if they actually understand anything being written here (as the original comments about the swap file obviously came from people who didn't really understand what the topic was about).

Share this post


Link to post
Share on other sites

@galzohar, first:

A pagefile is not completely irrelevant, especially on a 4GB system (like yours).

In respect to your real question, arma doesn't turn off file caching completly.

As i wrote, arma just doesn't use the mechanism provided by windows, it uses it's own implementation, based on 'section objects' (file mapping api).

One important advantage is, that it saves some costly memory copy operations by allowing the IO reads directly to memory inside armas adress space

and bypassing the extra sytem buffers. A second advantage is, that arma always 'know', what data are in the buffers and what not.

One disadvantage is, that armas internal buffers are consuming valuable virtual address space (on 32bit windows limitted to 2/3GB or 4GB) and therefore very frequently remapping is required.

The only way to avoid this remapping, would be a move to 64bit (8TB adress space on windows) for arma ...

Greets,

Fred41

Share this post


Link to post
Share on other sites
@galzohar, first:

A pagefile is not completely irrelevant, especially on a 4GB system (like yours).

What I meant is that it was irrelevant to the topic of windows file caching in ArmA, and irrelevant to the game needing to read a lot from the hard drive due to not saving the data in RAM even if RAM is available. Pagefile is used when you don't have enough RAM and thus RAM needs to be dumped to the hard drive, which is the opposite scenario to the one being discussed.

In respect to your real question, arma doesn't turn off file caching completly.

As i wrote, arma just doesn't use the mechanism provided by windows, it uses it's own implementation, based on 'section objects' (file mapping api).

One important advantage is, that it saves some costly memory copy operations by allowing the IO reads directly to memory inside armas adress space

and bypassing the extra sytem buffers. A second advantage is, that arma always 'know', what data are in the buffers and what not.

One disadvantage is, that armas internal buffers are consuming valuable virtual address space (on 32bit windows limitted to 2/3GB or 4GB) and therefore very frequently remapping is required.

The only way to avoid this remapping, would be a move to 64bit (8TB adress space on windows) for arma ...

Greets,

Fred41

This is interesting, I wonder if it's really the case (as in it actually does save the memory copies, and it makes sense that it does). However this still doesn't explain why they wouldn't enable windows file caching on top of it, so that whatever ArmA's system doesn't cache, will still be possible to cache by windows using the 64-bit address space of the OS, as the windows caching would be completely external to the ArmA 3 application and should be transparent from the ArmA application point of view.

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
Sign in to follow this  

×