ArmA 2 I/O analysis results

[liquidpinky 11]

My pressing questions is will a 4GB RAM disk work or I have to put the ENTIRE game 8GB onto a RAM disk to see performance. 16 GB of RAM with 4X4GB config cost the same as a smaller capacity SSD.

4gb is okay, you just have to be selective of what you put in it.

Just run qwertz test with Process Monitor to help decide what you would prefer for your own gaming scenarios.

I have it currently down to 3gb of files and the spare 1gb as a pagefile.

That was taken by analysing one of qwertz earlier tests then deciding what would make the game smooth and sorting out things that wouldn't effect graphical performance like sound files etc.

I do recommend putting the UI files in there though as map and menu loading is a treat now.

[reilar 10]

Hi guys,

I have fixed my tool - you can download it here:

http://www.mediafire.com/?sharekey=574bc4cb113cc15690a82c7bb0fad7ade04e75f6e8ebb871

Should work with everyone's data now - let me see the results !

Yes, now it works perfectly:)

I've now moved whole my ArmA 2 installation folder onto a ram drive and logged a couple of minutes flying in a Harrier over Chernagorsk.

This is the analysis of it using your excelent tool:

http://www.mediafire.com/?kmrzzzytcni

Still get some stutter though even if it's better than without a ram drive.

[.kju 3240]

How about having people post a screenshot of their ingame video settings.

After all this should have a visible and noticeable effect no matter what.

[Leon86 13]

How about having people post a screenshot of their ingame video settings.

After all this should have a visible and noticeable effect no matter what.

http://forums.bistudio.com/showpost.php?p=1467819&postcount=56

I notice an effect in my 2 video's

[GLeek 10]

Yes, now it works perfectly:)

I've now moved whole my ArmA 2 installation folder onto a ram drive and logged a couple of minutes flying in a Harrier over Chernagorsk.

This is the analysis of it using your excelent tool:

http://www.mediafire.com/?kmrzzzytcni

Still get some stutter though even if it's better than without a ram drive.

your entire arma2 folder ? 16GB you have installed ?

my mobo can manage maximum 4x2GB @ 667mhz. my overclock impose 720mhz x9 . grrrrr!!!

i will still buy 4 more gig , because ramdrive rocks

[Leon86 13]

Click my sig. ;)

Liquidpinky, could you maybe change your guide? The shell link extentions are completely unneeded. You can just create a folder named AddOns on your ramdisk (say R:\AddOns) and start arma with adding -mod=R:\ to the shortcut.

Dead3yz explains it in more detail in this topic:

http://www.armaholic.com/forums.php?m=posts&p=54150

plus, he has some more performance tips

[reilar 10]

your entire arma2 folder ? 16GB you have installed ?

I have 12GB istalled and using 9GB for ram drive. ArmA 2 uses about 8.5GB.

[.kju 3240]

Ram drive with corsair 1600 ... 12 gb (1680mhz) cl 9 9 9 22:

Insane values :O

[qwertz 10]

Yes, now it works perfectly:)

I've now moved whole my ArmA 2 installation folder onto a ram drive and logged a couple of minutes flying in a Harrier over Chernagorsk.

This is the analysis of it using your excelent tool:

http://www.mediafire.com/?kmrzzzytcni

Still get some stutter though even if it's better than without a ram drive.

By the way, in my own tests, I have found that flying around is actually LEAST stressing in terms of I/O - I guess that's because you are high up and staying on a quite low LOD. Most stressing is running/driving around in towns/cities.

I have taken the freedom to post your results below - hope that's fine with you.

Edited October 20, 2009 by qwertz

[BCA Cat Toaster 10]

@1longtime

As you can see from the screen I posted in my earlier posting here the Samsung 256GB PBJ-22 SSD is not bad performer at all, it´s just two or three months on the market. I don´t think I have problems with my SSD, it´s just awesome but I´ve never played ArmA2 on a physical harddrive so I can´t compare that at all but for sure my experience would be 20 times worse. It depends where you come from, what you are used to and which settings you use so I might complain from an already high level and shut up at all. ;-) It´s just not as smooth as I want it to have on my rig where everything else of course runs fine on the highest detail level.

However we can see now, how BIG the differences are between a classic HDD, an already Highend VelociRAID 0, some SSDs and RAM when it comes to ArmA-I/O´s and those numbers are REALLY impressive. A game shouldn´t be limited by Harddrive-Speed in that manner.

[shataan 1]

Why can`t peeps with less ram for the moment use a ramdrive for their swapfile? Will that at least help a bit?

[Leon86 13]

I dont think moving the swapfile is a good idea. If you dont have a lot of ram just put structures.pbo, chernarus.pbo and buildings.pbo on it. And maybe plants2_tree.pbo if you still have space left. These files get more I/O than all other files combined it seems.

more results btw. this time playing domination online:

http://www.mediafire.com/?woiz5cgn4ao

[oktane 0]

Arma does not use file caching - period. This means all the tools that are advertising to optimize the windows cache are useless for our purpose. On the other hand, it means that Arma is really 100% streaming the content. Because as a 32 bit app it can only address 2GB max (and I have yet to see it use more than 1GB), with >9GB of game assets, it means that there must be a LOT of disk activity.

From the disassembly, the CreateFileW call is using the dwFlagsAndAttributes 0x60000080 when it creates the file handles. Which means these are set if I grok that page correctly:

FILE_ATTRIBUTE_NORMAL

FILE_FLAG_NO_BUFFERING

FILE_FLAG_OVERLAPPED

The actual call used is at 0090CECD in the 59323 beta. I spent a lot of time (wasted) attempting to trace back where the flag is being set from, it gets passed as ptr's to a few funcs and I simply lack the skills to track it down. If we could find it, and set it to 0x40000080 instead and see how it functions then. (my guess would be worse) Another note would be, if they disabled this, they did it for a reason.. they are smart guys, likely because they have an internal cache. So by saying it doesn't use a cache may start a little fud.

Here is a detailed page about disabling buffering. http://msdn.microsoft.com/en-us/library/cc644950%28VS.85%29.aspx

I will now check how each interation of the engine did it.. OFP thru Arma, see if it was a recent change or an old one.. if its old we may be able to have them relook into windows own caching, since at least on the latest OS's, there may be benefits.

edit:

OFP Resistance does not use it. Its flags are hard coded above the CreateFile call, to:

push 10000000h ; dwFlagsAndAttributes, which means FILE_FLAG_RANDOM_ACCESS to optimize caching/buffering.

Arma Does use FILE_FLAG_NO_BUFFERING, just like Arma2.

So the change may have occurred in Arma development, or OFP Elite development. I'm not digging out a dev kit and patching the xbe to check the latter. :)

edit:

This tidbit helps explain some reasoning, as the assets grew:

The amount of I/O performance improvement that file data caching offers depends on the size of the file data block being read or written. When large blocks of file data are read and written, it is more likely that disk reads and writes will be necessary to finish the I/O operation. I/O performance will be increasingly impaired as more of this kind of I/O operation occurs.

In these situations, caching can be turned off. This is done at the time the file is opened by passing FILE_FLAG_NO_BUFFERING as a value for the dwFlagsAndAttributes parameter of CreateFile. When caching is disabled, all read and write operations directly access the physical disk. However, the file metadata may still be cached. To flush the metadata to disk, use the FlushFileBuffers function.

--

I have a readyboost drive that I use, 16gb, on my Win7 x64 and it is very active when Arma2 is loading and running. I think that may be using some alternate cache method. (the readyboost feature, not arma)

Edited October 21, 2009 by oktane

[qwertz 10]

From the disassembly, the CreateFileW call is using the dwFlagsAndAttributes 0x60000080 when it creates the file handles. Which means these are set if I grok that page correctly:

FILE_ATTRIBUTE_NORMAL

FILE_FLAG_NO_BUFFERING

FILE_FLAG_OVERLAPPED

The actual call used is at 0090CECD in the 59323 beta. I spent a lot of time (wasted) attempting to trace back where the flag is being set from, it gets passed as ptr's to a few funcs and I simply lack the skills to track it down. If we could find it, and set it to 0x40000080 instead and see how it functions then. (my guess would be worse) Another note would be, if they disabled this, they did it for a reason.. they are smart guys, likely because they have an internal cache. So by saying it doesn't use a cache may start a little fud.

Here is a detailed page about disabling buffering. http://msdn.microsoft.com/en-us/library/cc644950%28VS.85%29.aspx

I will now check how each interation of the engine did it.. OFP thru Arma, see if it was a recent change or an old one.. if its old we may be able to have them relook into windows own caching, since at least on the latest OS's, there may be benefits.

edit:

OFP Resistance does not use it. Its flags are hard coded above the CreateFile call, to:

push 10000000h ; dwFlagsAndAttributes, which means FILE_FLAG_RANDOM_ACCESS to optimize caching/buffering.

Arma Does use FILE_FLAG_NO_BUFFERING, just like Arma2.

So the change may have occurred in Arma development, or OFP Elite development. I'm not digging out a dev kit and patching the xbe to check the latter. :)

edit:

This tidbit helps explain some reasoning, as the assets grew:

The amount of I/O performance improvement that file data caching offers depends on the size of the file data block being read or written. When large blocks of file data are read and written, it is more likely that disk reads and writes will be necessary to finish the I/O operation. I/O performance will be increasingly impaired as more of this kind of I/O operation occurs.

In these situations, caching can be turned off. This is done at the time the file is opened by passing FILE_FLAG_NO_BUFFERING as a value for the dwFlagsAndAttributes parameter of CreateFile. When caching is disabled, all read and write operations directly access the physical disk. However, the file metadata may still be cached. To flush the metadata to disk, use the FlushFileBuffers function.

--

I have a readyboost drive that I use, 16gb, on my Win7 x64 and it is very active when Arma2 is loading and running. I think that may be using some alternate cache method. (the readyboost feature, not arma)

Hi oktane,

that's great insight! You might be on the right track here. I guess that the amount and size of data blocks have just gotten too large to be efficiently handled by the Windoze system cache. On top of that, with arma2.exe still being a 32bit app, it can only address 2GB of data including program code and therefore has not much address space left for any user level caching (which would cause the same probs as using the Windoze cache in the first place, thats why I doubt they did this), so it needs to constantly access the data storage volume to stream the content.

A way to fix this (besides putting the data on a RAMdisk so that effectively cache=disk) would be a 64bit ArmA on a PC with 6GB+ RAM that comes with its own user-mode cache.

Wait a minute - why wait for BIS - Are you a programmer? How about hacking a system-level driver that allocates a lot of system RAM for its own proprietary cache in 64bit, prefetches the stuff and then captures ArmA's unbuffered I/O's and redirects it to that buffer? Sorry if that is BS, I am not a coder :D

---------- Post added at 02:10 AM ---------- Previous post was at 02:06 AM ----------

By the way, new version of the tool is out (see my signature). Just a minor error fix.

Edited October 21, 2009 by qwertz

[wasserkool 1]

So a 4GB RAM drive will still yield noticable improvement?

[qwertz 10]

Ram drive with corsair 1600 ... 12 gb (1680mhz) cl 9 9 9 22:

Insane values :O

I can beat that lol - Mushkin 12GB (1600Mhz) CL 7-7-7-20 1T triple channel :p

By the way - too fast for HDTune (talk about "through the roof" performance)...:bounce3:

[froggyluv 2130]

So a 4GB RAM drive will still yield noticable improvement?

I'm only using a 1.5 GB Ramdrive and have noticed increased smoothness in terms of less pop up, flickering and the like

-Buildings.pbo

-Chernarus.pbo

-Structures.pbo

-Plants2_Clutter.pbo

-Plants2_Plant.pbo

-UI.pbo

are all I could fit. If I made it 150 mb larger I could add Plant_trees.pbo but I daren't try :butbut:

[shataan 1]

I am on win XP 32 bit, and have 8 gigs of Corsair Dominator @ 1600. WinXP only recognises 3 gigs I think, is it possible to make a ram drive still? Thx.

[qwertz 10]

I am on win XP 32 bit, and have 8 gigs of Corsair Dominator @ 1600. WinXP only recognises 3 gigs I think, is it possible to make a ram drive still? Thx.

Hey Shataan, no offense meant, but the answer is very close, if you have a look :)

http://forums.bistudio.com/showthread.php?t=88388

[tcp 10]

Yes, Shataan, as has been discussed previously, the RAM disk software will utilize all RAM above the 4086MB mark. However, only some software will also let you use RAM below that mark.

[lt_darkman 0]

I am on win XP 32 bit, and have 8 gigs of Corsair Dominator

Why have 8GB RAM in a 32-bit OS?

P

[alan.rio 18]

I have 4 gig of ram, with a dual boot system (XP 32 / Win7 64). As has been said XP isthe one I should use, however how much ram do you think I could assign to the RAMdisk? Is it worth my while to bother?

I'm planning on buying another 4gig but I won't be able to afford it for a while yet.

[whisper 0]

From the disassembly, the CreateFileW call is using the dwFlagsAndAttributes 0x60000080 when it creates the file handles. Which means these are set if I grok that page correctly:

FILE_ATTRIBUTE_NORMAL

FILE_FLAG_NO_BUFFERING

FILE_FLAG_OVERLAPPED

The actual call used is at 0090CECD in the 59323 beta. I spent a lot of time (wasted) attempting to trace back where the flag is being set from, it gets passed as ptr's to a few funcs and I simply lack the skills to track it down. If we could find it, and set it to 0x40000080 instead and see how it functions then. (my guess would be worse) Another note would be, if they disabled this, they did it for a reason.. they are smart guys, likely because they have an internal cache. So by saying it doesn't use a cache may start a little fud.

Here is a detailed page about disabling buffering. http://msdn.microsoft.com/en-us/library/cc644950%28VS.85%29.aspx

I will now check how each interation of the engine did it.. OFP thru Arma, see if it was a recent change or an old one.. if its old we may be able to have them relook into windows own caching, since at least on the latest OS's, there may be benefits.

edit:

OFP Resistance does not use it. Its flags are hard coded above the CreateFile call, to:

push 10000000h ; dwFlagsAndAttributes, which means FILE_FLAG_RANDOM_ACCESS to optimize caching/buffering.

Arma Does use FILE_FLAG_NO_BUFFERING, just like Arma2.

So the change may have occurred in Arma development, or OFP Elite development. I'm not digging out a dev kit and patching the xbe to check the latter. :)

edit:

This tidbit helps explain some reasoning, as the assets grew:

The amount of I/O performance improvement that file data caching offers depends on the size of the file data block being read or written. When large blocks of file data are read and written, it is more likely that disk reads and writes will be necessary to finish the I/O operation. I/O performance will be increasingly impaired as more of this kind of I/O operation occurs.

In these situations, caching can be turned off. This is done at the time the file is opened by passing FILE_FLAG_NO_BUFFERING as a value for the dwFlagsAndAttributes parameter of CreateFile. When caching is disabled, all read and write operations directly access the physical disk. However, the file metadata may still be cached. To flush the metadata to disk, use the FlushFileBuffers function.

--

I have a readyboost drive that I use, 16gb, on my Win7 x64 and it is very active when Arma2 is loading and running. I think that may be using some alternate cache method. (the readyboost feature, not arma)

That's quite logical once you see the size of the .pbo we're talking about, by the time ArmA1 was being developped, we didn't have access to so large amounts of RAM as we do now.

Thing is, why not caching internally (through game engine itself) what is being used? Ie, what has already been read from disk and extracted for usage (this tree texture, that house P3D, etc....) should stay in memory (making it less than the whole pbo). Or is there such variety that all can't stay in RAM together and/or there's always a new item not seen before that must be read from HDD?

My suggestion in this case would be to do a drastic data optimization pass to find what can be lowered (really lowered) in terms of size amongst the data, and make this lower quality things available through Graphical Option for players with lower performances rigs. Currently, everyone is loading the maximum quality textures and stuff at some point (closer for lower end rigs, but still loading it), I think. And it seems there is no internal cache, or it's not usefull. So you're hitting both low end rigs (through high quality things being loaded) and high en rigs (through constant HDD access). Or am I completely off? :) BI input would be very welcome

(sry for my non-technical terms, I'm no programmer, just using logic here :) )

Edited October 21, 2009 by whisper

[cjph 0]

I am on win XP 32 bit, and have 8 gigs of Corsair Dominator @ 1600. WinXP only recognises 3 gigs I think, is it possible to make a ram drive still? Thx.

Hi

I've been using 4Gb as a RAMdrive out of the 6Gb total on my XP32 system, and it seems to work ok.

[evilfury 0]

Using ramdrive option dramaticaly increase my windows load time. I have to wait 1min before it reach desktop. Anyone else experience this problem?