ArmA 2 I/O analysis results

[Leon86 13]

I've been playing around with the process monitor tool some more. Even though arma2.exe is only using ~700mb there is a lot of activity in the page file.

http://www.mediafire.com/?sharekey=798fb57f39f7f5edaf924764f9977b1de04e75f6e8ebb871

I tried to improve performance by creating an extra page file on the ramdisk, this didn't really work because windows uses these files 50/50 doing 2 writing operations to the ramdisk and 2 to the normal pagefile alternately. Moving the pagefile to the ramdrive would probably improve performance. still a bad idea though.

[liquidpinky 11]

I've been playing around with the process monitor tool some more. Even though arma2.exe is only using ~700mb there is a lot of activity in the page file.

http://www.mediafire.com/?sharekey=798fb57f39f7f5edaf924764f9977b1de04e75f6e8ebb871

I tried to improve performance by creating an extra page file on the ramdisk, this didn't really work because windows uses these files 50/50 doing 2 writing operations to the ramdisk and 2 to the normal pagefile alternately. Moving the pagefile to the ramdrive would probably improve performance. still a bad idea though.

Well actually, now you mention it.

I did a few runs last night with pagefile on the RamDisk only and the game ran miles smoother and I had no memory creeping up problems.

I have to try on my non beta version in MP tonight just to completely sure of it though.

I am also upping my RamDisk to 5gb out of my 8 as I have barely went over 80% Ram usage while playing, so I have some room to spare I think.

Means pagefile increase and more game files in there too.

[yanquis 10]

Using ramdrive option dramaticaly increase my windows load time. I have to wait 1min before it reach desktop. Anyone else experience this problem?

yes, but i dont typically shut down or restart unless forced to anyway

oh btw, im sure its just a result of the ramdisk being loaded. i imagine there is some way to disable it at windows startup.

---------- Post added at 02:18 PM ---------- Previous post was at 02:14 PM ----------

Well actually, now you mention it.

I did a few runs last night with pagefile on the RamDisk only and the game ran miles smoother and I had no memory creeping up problems.

I have to try on my non beta version in MP tonight just to completely sure of it though.

I am also upping my RamDisk to 5gb out of my 8 as I have barely went over 80% Ram usage while playing, so I have some room to spare I think.

Means pagefile increase and more game files in there too.

could you update the guide or otherwise share what files you use? and i am lost about this pagefile stuff. is the memory creeping up thing what might cause the game to CTD with an error message about (something to do with memory)? because ive gotten that a couple times...

i have finally got (fingers crossed) this game running pretty much perfectly on my system. a bit ironic since tmrw the computer store will hopefully be getting the 5870 in.

[.kju 3240]

Did some testing with the ArmA2Mark.

Hardware:

Q6600, ATI 4890, Asus P5Q SE, 4x 2GB DDR2-800 A-Data Extreme, Intel-M 80GB SSD.

First row: Game 100% on the SSD. 4.5 GB swap file on the Ram drive (5GB in total).

Second row: 5GB of the game on the SSD, rest on the Ram drive. 1 GB swap file on the Ram drive (5GB in total).

Third row: Like second row - just replace all plants2*, buildings* and structures with PROPER tweaks.

Ingame settings:

• Very high for texture detail, video memory, AF, terrain detail, object detail,
  
• Low AA (2x)
  
• Shadows and PP off.
  
• 1280x1024@32b
  
• Window mode
  

ArmAIIMark end scores:

1932 1966 1974: 1st run.

1928 1966 1974: 2nd run.

2000 2057 2149: Objects details to very low.

2115 2121 2208: In addition video memory and texture details to normal.

2783 2840 2918: In addition terrain detail to very low (aka clutter off).

Intel M:

Ram drive:

Conclusion:

• FPS wise little difference having a big swap file on the Ram drive or the
  files there itself (based on the Intel-M SSD).
  
• In regards to stutter ArmA2Mark might not be a good tool to judge that.
  
• It does very extreme and fast camera moves on purpose. Even more no
  preloading for the camera beaming and camera is certainly not the same as
  playing yourself.
  
• I'd say for ArmA2Mark little to none reduction of stutters, yet see point above.
  
• Will move the engine files (exe, dlls, dta folder) to Ram drive as well.
  
• Will test the difference between low, normal, high and very high clutter.
  Seems to be the biggest FPS killer for my system.
  

Edited October 21, 2009 by kju

[yanquis 10]

well its definitely not about fps, its just about real gameplay, how quickly the game becomes playable from load (as soon as the GPU can render the textures if using RAMDISK), piloting choppers at low altitude (supersmooth, but again aesthetic problems with GPU renderpop in my experience), etc. if fps is just a # when it comes to arma 2, arma 2 mark is just a blotch on a screen as far as im concerned.

if you have a shitty HDD (i dont know how low a score this actually is, but on the windows experience index its the main bottleneck on my system by almost like 2 points with a 5.9 score), the difference in gameplay is huge and u can clearly see it on arma 2 mark. the textures in all the test runs are actually on the screen for one thing (ha).

SSD and velociraptor people, i dont know if the effect is so stunning. but qwertz said he had an SSD setup and still the effect for him was huge, so who knows. of course he loaded everything onto ramdisk not just some textures like i have to.

[TimRiceSE 10]

I only have 4gb of RAM so I cant really afford to have so much in a RAM drive, with too large a RAM Drive i get huge performance slowdown ofcourse, however, I tried two tihngs...

putting almost all the game pbo's on a 8gb USB flash drive - this actually reduced game load time quite a lot, and went a long way to reduce stuttering, a lot smoother than just using a HDD

and then

using 1.5gb RAMDisk for structures, plants2_tree and buildings2 pbo's, and the rest of the pbos on the USB flash drive. Very very smooth gameplay...

[yanquis 10]

hmmm so someone FINALLY at least TRIED the usb thing, and it does work. well thats a simple $40 (for 16GB) solution that just requires plugging a stick into your computer externally.

[liquidpinky 11]

well its definitely not about fps, its just about real gameplay, how quickly the game becomes playable from load (as soon as the GPU can render the textures if using RAMDISK), piloting choppers at low altitude (supersmooth, but again aesthetic problems with GPU renderpop in my experience), etc. if fps is just a # when it comes to arma 2, arma 2 mark is just a blotch on a screen as far as im concerned.

if you have a shitty HDD (i dont know how low a score this actually is, but on the windows experience index its the main bottleneck on my system by almost like 2 points with a 5.9 score), the difference in gameplay is huge and u can clearly see it on arma 2 mark. the textures in all the test runs are actually on the screen for one thing (ha).

SSD and velociraptor people, i dont know if the effect is so stunning. but qwertz said he had an SSD setup and still the effect for him was huge, so who knows. of course he loaded everything onto ramdisk not just some textures like i have to.

My C and D are raptors and still the difference is night and day. RamDrive has nothing that can touch it for silky smooth gameplay.

On the subject or ArmA2Mark, I really noticed a difference in the final test. It was much improved plus my scores are now higher than before.

There is also a part in the third test, I think. when the camera looks toward an M1A1 tank as the camera passes overhead. You know you are doing well when it looks to the left at the tank and not to the right. This is a sign that it has been smooth up to that point.

[tcp 10]

Depending on your HDD speed, its going to take a while to load everything back into RAM after a reboot. Although I don't know why Windows can't just continue loading without waiting for the RAMdisk. It must be default behavior in case you have startup programs on it.

Regarding the bottleneck, PBO files aren't compressed. Even if they were, entire PBO files don't need to loaded into memory, just the files that are needed. Textures will also be reduced in size depending on the quality settings.

All games have this problem. We've all played the games with long loading screens. They get can get smooth graphics by taking the time to load everything in memory ahead of time and actually being capable of fitting everything they need for that level into memory together.

If ArmA attempts precaching on-the-fly, it does a horrible job at it. The game should know what you need depending what area of the map you are on. However, I think ArmA uses too many different textures for plants and buildings. It should reuse them to save much-needed space. Another thing that is interesting is that flying supposedly uses less IO because it only loads the far LOD. However, on the ground, all LODs should be loaded together, but they don't seem to be because there's usually a pause between LOD switching. What's weirder is that you still have that pause is still there when you zoom back on the same object a minute later. I don't think its the video card processing lag (I have a 4790). I think that there's so many textures that they are constantly being pushed out of memory. There's also supposedly a lot of page file usage. However, the ArmA2 process never uses it full address space in memory (2047MB). So it must be the reserved RAM for system cache that is paging out all the time. However, isn't that just a secondary buffer for the video card. How can 1GB of VRAM back by .8GB of system cache not be enough?

When you buy the flash drive make sure its a highspeed USB2.0. Saying it supports Windows ReadyBoost means its somewhat fast. However, you should probably look online to find out actual performance, because speeds can vary greatly on flash despite their claims. A slow write speed is OK, but you want an ultra-fast read speed. Also, it should only be used in conjunction with your HDD. HDD will always be faster, but its going to get maxed out by ArmA so having some addons on a flash drive might boost the speed a little. Also flash has the advantage of nearly no seek time. Each USB bus (you probably have two or three on a desktop) can only support 40MB/s, while a typical hard drive is over 100MB/s.

Some reading on Windows Memory Limitations: http://www.geoffchappell.com/viewer.htm?doc=notes/windows/license/memory.htm

Edited October 21, 2009 by tcp

[TimRiceSE 10]

hmmm so someone FINALLY at least TRIED the usb thing, and it does work. well thats a simple $40 (for 16GB) solution that just requires plugging a stick into your computer externally.

I'm not sure how i can "benchmark" the performance with regards to stuttering though. I tried making a video, but eh, fraps basically takes a crap on performance anyway.

[qwertz 10]

hmmm so someone FINALLY at least TRIED the usb thing, and it does work. well thats a simple $40 (for 16GB) solution that just requires plugging a stick into your computer externally.

It might be worth a try, but in theory there is a problem:

USB sticks tend to suck in IOPS once you are reaching a certain transfer size level. A typical trasfer size pattern in ArmA2 is 80% between 4k and 64k, 17% between 64k and 1M, and 3% over 1M.

So whilst you will see a 2x -3x increase in 80% of the times (over a normal HDD), in 20% of the occasions you might be down to less than HDD performance, in extreme case down by 4/5.

But again, it is worth a try - it might well be that you can get rid of a substantial level of stutter at the expense of having a small number of extreme stutters and/or increased level loading times.

If anyone tests this, could you post the results using the tool in my sig so we better understand whether this helps ?

Edited October 21, 2009 by qwertz

[BCA Cat Toaster 10]

I used the I/O-Analysis-Tool today and here is the result. What I did is spawning on the Airfield west of Chernogorsk, entered a Helicopter and went to Chernogorsk and back to the countryside on very low altitude with a lot of stuttering, Popus and LOD-Switching.

Hardware:

E8600@4GHz, ATI 5780, Intel DP45SG Extreme, 4x 2GB DDR3-10600 Kingston, Samsung PBJ-22 256GB SSD

Ingame-Settings:

Very high for texture detail, video memory, AF, terrain detail

Medium object detail

Low AA (2x)

Shadows High and PP off.

1920x1200@7-64b

Full Screen

1600m VD

Now I created a 5GB-RAM-Disk and added about 4.8GB of the most I/O-relevant pbo´s of the analysis to it and repeated the test: I experienced exactly the same behaviour with Popups, Stuttering and LOD-Swichting coming to Chernogorsk as running from disk. I can´t see any differences between RAMDISK and SSD in this setup.

I really MOVED the pbo´s from there original location to the RAMDISK and I also forgot to add the -mods-line in 1st place and wasn´t able so start the game at all because of the missing files, so the RAM-Disk was definetely used as I added the propper command. At least I noticed that the Test-Mission loads a lot faster than from disk.

I can´t explain it, but it seems in some situations even a RAM-Disk does not help at all.

[Leon86 13]

Well actually, now you mention it.

I did a few runs last night with pagefile on the RamDisk only and the game ran miles smoother and I had no memory creeping up problems.

I have to try on my non beta version in MP tonight just to completely sure of it though.

I am also upping my RamDisk to 5gb out of my 8 as I have barely went over 80% Ram usage while playing, so I have some room to spare I think.

Means pagefile increase and more game files in there too.

Update: when I use a smaller Ramdrive (was 3.5 now 2.0 GB and I have 6GB ram) the pagefile isn't used for arma2 at all. Too bad arma2 only wants to use about 1GB of memory. Probably good for stability but bad for performance. The strangest thing was that about 800MB of memory still wasn't being used when windows decided it would start paging arma2 stuff.

Edited October 21, 2009 by Leon86

[qwertz 10]

I used the I/O-Analysis-Tool today and here is the result. What I did is spawning on the Airfield west of Chernogorsk, entered a Helicopter and went to Chernogorsk and back to the countryside on very low altitude with a lot of stuttering, Popus and LOD-Switching.

Hardware:

E8600@4GHz, ATI 5780, Intel DP45SG Extreme, 4x 2GB DDR3-10600 Kingston, Samsung PBJ-22 256GB SSD

Ingame-Settings:

Very high for texture detail, video memory, AF, terrain detail

Medium object detail

Low AA (2x)

Shadows High and PP off.

1920x1200@7-64b

Full Screen

1600m VD

Now I created a 5GB-RAM-Disk and added about 4.8GB of the most I/O-relevant pbo´s of the analysis to it and repeated the test: I experienced exactly the same behaviour with Popups, Stuttering and LOD-Swichting coming to Chernogorsk as running from disk. I can´t see any differences between RAMDISK and SSD in this setup.

I really MOVED the pbo´s from there original location to the RAMDISK and I also forgot to add the -mods-line in 1st place and wasn´t able so start the game at all because of the missing files, so the RAM-Disk was definetely used as I added the propper command. At least I noticed that the Test-Mission loads a lot faster than from disk.

I can´t explain it, but it seems in some situations even a RAM-Disk does not help at all.

Hi Toaster,

could you upload the full tool file so I can have a look at the raw data and try to find the reason for this result?

It seems like putting only a part of the files on a ramdisk is a hit and miss - maybe we can find the reason for this.

[yanquis 10]

what you refer to as LOD-switching, i have always interpreted as 'renderpop' or the GPU processing the texture. it would seem to me that textures can load instantly but the GPU still has to process how they should look to the viewer once that happens so in a chopper at fast speeds you really notice this happening as you flyover. obv if thats what it is there is nothing any HDD can do its all up to the gpu.

btw, did anyone else see in the arrowhead interview video how these guys create their tree models -- they actually have a computer simulate the growth of a x-year old tree, i guess they do that for each one or several of them at least? crazy.

[tcp 10]

What kind of GPU would you need to not have renderpop? I have an ATI 4890.

I am talking about moving on foot and zooming in on a tank.

I doubt its due to GPU processing lag.

[BCA Cat Toaster 10]

Hi Qwertz,

here is the full report. Perhaps I missed a single allover relevant file. Perhaps on my resolution the bottleneck is somewhere else?

[yanquis 10]

What kind of GPU would you need to not have renderpop? I have an ATI 4890.

I am talking about moving on foot and zooming in on a tank.

I doubt its due to GPU processing lag.

moving on foot i dont think i notice any. i have the same GPU. zooming in definitely. i agree it is not lag, id think its the result more of at what range the GPU decides to process textures than how fast it is. if so that is probably tweakable somehow, somewhere, maybe those 'maxframes' settings in the config. i have yet to really experiment with them. of course im sure the farther the range the lower the fps. (im also sure i have no idea whatsoever what im talking about but this is all what common sense tells me based on what i see happening on the screen)

[Leon86 13]

I've been playing the game with a 2 .pbo's on my Ipod and 1 on a usb stick +2GB on a ramdisk. Haven't noticed anything out of the ordinary, I chose some of the files that have the smallest average kB. I made a log with the performance monitor but the files are used so rarely that I dont think you'll notice any difference if they were on the harddisk.

logfile:

http://www.mediafire.com/?sharekey=798fb57f39f7f5edaf924764f9977b1de04e75f6e8ebb871

I couldn't test my Ipod with HD Tune btw, it says read error, test aborted. but with file transfers it's a lot faster than my el-cheapo usb stick.

What I also did was measure my fps with Fraps.

In Chernogorsk there are so many buildings walls etc that the bottleneck fps-wise is probably my cpu (an e6750 slightly overclocked to 3464Mhz), I could turn on anti-aliasing in most places without a decrease in fps.

Outside the city the trees are a major performance killer. I could go from 60 fps (who knows how much without vsync) to a mere 22 fps by zooming in on a group of trees. this time anti-aliasing decreased the fps even more, so here a faster gpu (now a 8800gtx) might help. Those trees look nice up close but I'd rather throw out the highest LOD (is that what the proper mod does?) and have more fps. Changing object or terrain detail seems to have NO effect on trees whatsoever... I changed most things to low and now the game looks terrible when I'm in the middle of a field just so I can have acceptable performance when near trees.

[oktane 0]

using 1.5gb RAMDisk for structures, plants2_tree and buildings2 pbo's, and the rest of the pbos on the USB flash drive. Very very smooth gameplay...

Cool

When I tried this a month or so ago, it didn't like my USB flash(even though its a fast one).. the game would crash to desktop and in the RPT file there were errors like "device not ready" as if the USB traffic was so great that it was causing the drive to poop out. Maybe my USB drive is junk though. I had moved the addons folder to it and ran it like arma2.exe -mod=f:\@ramdisk and put all the files in f:\@ramdisk\addons\

---------- Post added at 04:01 AM ---------- Previous post was at 04:00 AM ----------

In Chernogorsk there are so many buildings walls etc that the bottleneck fps-wise is probably my cpu (an e6750 slightly overclocked to 3464Mhz),

You know, with the ramdisk, Cherno acually is playable for me.. but the water around cherno makes my fps dive! If im looking at the water towards the town, it tanks.. but if i move into town then its ok. (using camera.sqs)

---------- Post added at 04:07 AM ---------- Previous post was at 04:01 AM ----------

Hi oktane,

that's great insight! You might be on the right track here. I guess that the amount and size of data blocks have just gotten too large to be efficiently handled by the Windoze system cache. On top of that, with arma2.exe still being a 32bit app, it can only address 2GB of data including program code and therefore has not much address space left for any user level caching (which would cause the same probs as using the Windoze cache in the first place, thats why I doubt they did this), so it needs to constantly access the data storage volume to stream the content.

A way to fix this (besides putting the data on a RAMdisk so that effectively cache=disk) would be a 64bit ArmA on a PC with 6GB+ RAM that comes with its own user-mode cache.

Wait a minute - why wait for BIS - Are you a programmer? How about hacking a system-level driver that allocates a lot of system RAM for its own proprietary cache in 64bit, prefetches the stuff and then captures ArmA's unbuffered I/O's and redirects it to that buffer? Sorry if that is BS, I am not a coder :D

.

Hi

No, not a programmer for real, just hackin around with logic. Definitely not a device driver coder, lol! Thats why I was looking in the disassembly for a way to change that flag, that's about the extent of my skills. And I'm not even sure that would help. I think there is a way to hack the PE header so that arma can use more ram.. editbin /LARGEADDRESSAWARE should do it. but I have no idea if the game will actually use the extra gig ram. Here is a page about it.. note the comment at the end.. http://www.jaloonz.com/2008/04/enabling-large-address-aware-for-games.html

Edited October 22, 2009 by oktane

[EDcase 87]

You know, with the ramdisk, Cherno acually is playable for me.. but the water around cherno makes my fps dive! If im looking at the water towards the town, it tanks.. but if i move into town then its ok. (using camera.sqs

Thats weird. Just using standard hdd install my fps shoot up to about 50 when I look out to sea. Normal play is around 20-30

[EricM 0]

The sea-fps issue is a video driver/shader thing. I had that before in 1.00 , updated my drivers and the stutter was gone. I heard it was back for some...

[ModeZt 0]

I've fixed the sea-fps issue by disabling V-Sync in drivers.

[.kju 3240]

You can find a topic here about:

Preloading content - information sharing.

[Leon86 13]

You know, with the ramdisk, Cherno acually is playable for me.. but the water around cherno makes my fps dive! If im looking at the water towards the town, it tanks.. but if i move into town then its ok. (using camera.sqs)

I get about 36 fps in cherno, it's playable but lower settings give no higher fps, my cpu is bottlenecking there. I know some of you have 4ghz core i5's or i7's but a 3.5ghz core2duo is well within optimal requirements (says here on the back of the box: intel core 2.8ghz). On 2.8ghz I wouldn't even get 30 fps in cherno. There are plenty of places where I get 60+ fps and there I can aim so much better/faster.