CPU VS RAM Performance & CPU Threading Benchmarked

[Leon86 13]

this is about ddr3, not the harddisk.

and it's not like 1333 cl7 outperforms the 1600 cl9

Edited April 23, 2013 by Leon86

[zebra 10]

Well, I made extra an account for this thread, as I normally only read forums. I neither have the time nor I am willing to discuss with the one or other guy which could be my son, without any knowledge about designing a game and without being in close interest to the game-industry since the last 30 years.

Iam in common with some guys like froggyluv, BangTail, Bigpickle and Fabio_Chavez, which gives me some hope that not everyone in todays time has such a ridiculous view.

1st, it’s a really rare chance to have such a close contact to the developer as its possible here with BIS

2nd no one of us has any idea of how hard it is to program such a complex engine and to put it into a game which is unreached in history.

Some of you guys may never played games in the 80s and is able to appreciate this kind of virtual open world with its almost realistically visual rendering.

So please just stop being that much childish to force again and again a statement from BIS that they have some troubles in optimizing this or that. We all know that they are very well have knowledge about this issue and everyone should be clear that they for sure would change some lines in their code to improve it. But maybe I surprise you with the fact that its an hell of work to change an engine, which has been programmed and further extended during a couple of years.

Iam also a fanboy for this series but anyway its just a game where you spend 35 bucks for getting hundreds of hours pure fun and the possibility to create worlds/missions and mods, instead listen for some hours to music in a club with some beers and awaking afterwards with a bad taste in your mouth. You can chose yourself.

Hopefully BIS can further improve the performance of their outstanding engine or/and put in additional highlight, like a new fog improvement, which looks just awesome.

[dnk 13]

@Insanatrix, etc

Disk streaming only seems to affect pop-in, not actual performance. Seems like the engine loads the lowest LOD/mipmap first, displays it fairly quickly, and then waits for the larger file to get streamed into RAM and then to the GPU for display. Certainly, being able to address more than 2-3GB of RAM would help reduce pop-in a lot since we could keep LOD/textures on hand, but it wouldn't improve FPS much if at all, as the SDD/Ramdisk users have discovered.

I think the GPU underutilization issue is clearly (ignoring bugs/poor "optimization" for certain setups) an issue of limited bandwidth and/or latency issues. You agree, yes.

Ultimately, I think it comes down to:

The Arma series has a similar amount (maybe more?) of textures it needs to load into the GPU constantly compared with other games (assumed)

The Arma series does less image processing compared with other games

That's how it makes sense to me that the quality of graphics rendering in this game is clearly much less advanced than in other current games (even years back), yet we have low performance and GPU core underutilization occurring. Other games are also affected by bandwidth/latency issues, yet this isn't as prominent for overall performance because all the eye candy keeps the core running at full-steam, and itself becomes a limiting factor often.

Edited April 23, 2013 by DNK

[jbetbrice 10]

So , i just did a test with Fraps to see wich game is the more '' Demanding'' , so i downloaded the AiA Mod for ArmA III , and compared with ArmA II on the same places

Just did a few screenshot to illustrate that :

First ArmA II Screens and FPS

[img ]http://cloud.steampowered.com/ugc/576735140866189277/1DB067BE9DB75962EE4C7A17A2025730F1BAC0C5/[/img]>100kb

51 FPS on it , settings in the following screen

[img ]http://cloud-2.steampowered.com/ugc/576735140866197805/BEE5749DBCD529BD2B3B4DBA70D9E49806E2C5B5/[/img]>100kb

Texture quality : Very High

Memory : Default

Aniso Filtering : Very High

AA : Disabled

ATOC : Disables

Terrain quality : Normal

Object Quality : Low

Shadow Quality : High

HDR : Normal

+ FXAA Sharp Filter on High

Post Treatment : Normal ( Please , don't kill me )

VSYNC : Disabled

Now ArmA III !

[img ]http://cloud-2.steampowered.com/ugc/576735140866264477/DA02AA14CCAC5DE3D9F60D0EE6B9F8574B1FE9B5/[/img]>100kb

Now here , fps looks nice , 50 fps too here , same settings as in ArmA II ( Some options Exclusive to ArmA III like PIC disabled)

ArmA II , Tchernarus Island

[img ]http://cloud-2.steampowered.com/ugc/576735140866219081/CD1CC3CAD4758E87F20F6D4055DC1E403D072A87/[/img]>100kb

Ok , 30 Fps here , in front os Chernogosk on ArmA III , but still looks playable

[img ]http://cloud-2.steampowered.com/ugc/576735140866295927/F2122C716B4CA927B0357EAE3F95F7E87E16FA42/[/img]>100kb

On ArmA III , 25 Fps , a little loss , but nothing dramatic here

Also , i noticed lower FPS in air on Arma III !

[img ]http://cloud-2.steampowered.com/ugc/576735140866310004/3BBA362088F3E3ABA25FA93AB1B432FBC0134019/[/img]>100kb

15 Fps Here !!!! , on Arma II , it's the opposite , more FPS on Air , lower on the ground !

That conclude my post for now , will try to do the same with the FRAPS fps counter on screen !

Configuration

WINDOWS 8

HD7770 Radeon 1 GB

8GB Ram

AMD FX4100

Edited April 23, 2013 by [FRL]Myke

[Leon86 13]

I'm not sure settings translate 1:1 from a2 to a3, a3 also has an object viewdistance, which a2 doesnt have.

[Insanatrix 0]

@Insanatrix, etc

Disk streaming only seems to affect pop-in, not actual performance. Seems like the engine loads the lowest LOD/mipmap first, displays it fairly quickly, and then waits for the larger file to get streamed into RAM and then to the GPU for display. Certainly, being able to address more than 2-3GB of RAM would help reduce pop-in a lot since we could keep LOD/textures on hand, but it wouldn't improve FPS much if at all, as the SDD/Ramdisk users have discovered.

I think the GPU underutilization issue is clearly (ignoring bugs/poor "optimization" for certain setups) an issue of limited bandwidth and/or latency issues. You agree, yes.

Ultimately, I think it comes down to:

The Arma series has a similar amount (maybe more?) of textures it needs to load into the GPU constantly compared with other games (assumed)

The Arma series does less image processing compared with other games

That's how it makes sense to me that the quality of graphics rendering in this game is clearly much less advanced than in other current games (even years back), yet we have low performance and GPU core underutilization occurring. Other games are also affected by bandwidth/latency issues, yet this isn't as prominent for overall performance because all the eye candy keeps the core running at full-steam, and itself becomes a limiting factor often.

If you look at it from the perspective of graphics only, then yes. My question and overall issue is, how does the streaming affect data being queued for the CPU?

Say an instruction references an FSM in Virtual Memory that's been paged. It then has to read from the pagefile. This can happen millions of times a second. The fastest SSD's have a seek time of .1ms . Seems really fast right? Well .1ms is 100,000 ns. Just for comparison your DDR memory typically has a latency of 8-12ns. Also there may not be room to cache it in RAM or it may need to be immediately unloaded from RAM and paged back to disk to make room for something else. So when it's called again it may have to go through the same process again. When you begin to picture this with multiple instances of data that would need to be fetched from paged memory, you can see where even though it's very minuscule data, it can have a very heavy impact on thread stalling and CPU Execution wait times due to the increased latency. Obviously there are ways to reduce the occurrence of this with pre-fetching and intelligent caching and such, but there's no way to completely eliminate it.

Even if you sat with an API monitor and watched hard drive usage, you wouldn't see a lot of usage because you're measuring bandwidth not transfer latency.

RAMDrives wouldn't matter because data is still paged because it cannot all fit into physical memory once the process is loaded and starts assigning Memory Address Points because the process is limited to 32-bit registers and pointers while your OS is not. You can't get a page file on to a RAMDrive, go read Microsoft Knowledge Base Articles.

Does it mean that it's the sole cause of the problem? No. I would venture to say that it would be one of the issue's as to why this game has bad multi-core usage and multi-threading capability though. How can you feed 4-8 core's with data when you can barely feed 1-2 in a real-time processing environment with such a high latency I/O caching and delivery system? Also do you see any other developers that use page file streaming as a solution to NOT go 64 bit or as a means to facilitate better multi-core usage and multi-threaded processing?

Edited April 23, 2013 by Insanatrix

[dasa 20]

while a ramdrive may help in some situations i wouldnt be surprised if it also lowered fps a bit by hogging some of the memory bandwidth... could be fun to test :)

[Leon86 13]

If you use a ramdrive you'll have very fast loadtimes, but there's also a higher chance paged data that's still in memory gets overwritten because there's not enough memory available to keep everything in. If you actually need the harddrive for critical data you'll get freezes of a second or half a second, very annoying.

for average fps it wont matter much.

[daze23 1]

really nothing new, it was done before by many, let's name e.g. Mount & Blade (available since Alpha and cost increasing each milestone) or well known Minecraft and dozens others ...

it's nothing new, but obviously some people have problem with 'get game early for way lower price' ... :rolleyes:

I said it was something new, not that BI was the first to try it. I stand by that

ROFLMFAO.

You didn't pay for early access - you paid for a game that isn't released yet.

As a perk of the pre-order, you got access to the Alpha and the Beta.

this has been gone over. the game is categorized in Steam's "Early Access" section. someone was also denied a refund by Steam, and told in so many words that this does not qualify as a "pre-order" (as you should have the option to cancel any "pre-order")

Edited April 23, 2013 by daze23

[MavericK96 0]

If you use a ramdrive you'll have very fast loadtimes, but there's also a higher chance paged data that's still in memory gets overwritten because there's not enough memory available to keep everything in. If you actually need the harddrive for critical data you'll get freezes of a second or half a second, very annoying.

for average fps it wont matter much.

In my experience, if you are already using an SSD to run ArmA, there is no point in using a RAM Disk.

[white 1]

I'm not sure settings translate 1:1 from a2 to a3, a3 also has an object viewdistance, which a2 doesnt have.

i think right now object is stuck along with regular view distance, i might be wrong but i think Dwarden wrote that somewhere. and also, shouldnt it help to gain fps instead of losing it? afterall lots of people keep saying how great arma 3 performance is over arma 2, and when someone actually tests it under the same scenario, its worse.

Edited April 23, 2013 by white

[jumpinghubert 48]

i think right now object is stuck along with regular view distance, and also, shouldnt it help to gain fps instead of losing it? afterall lots of people keep saying how great arma 3 performance is over arma 2, and when someone actually tests it, its worse.

for example in arma2 the object distance set with 4000m viewdistance is less than 2500m. Shadowdistance isn´t really known in arma2 (i think its roughly 50m) and the grass radius is higher when you set terraindetail on the same level as in arma3. Not easy to compare. For me (rig/settings/chernarus) the performance is a little bit better in arma3. But you can better compare it with ToH-rearmed @ chernarus because there you can set objectdistance and shadowdistance. Compared with toh-Chernarus arma3 has significantly better performance, at ground and in the air.

[white 1]

for example in arma2 the object distance set with 4000m viewdistance is less than 2500m.

wouldnt that make arma 2 lose more performance compared to arma 3? and not the other way around?

[maddogx 13]

wouldnt that make arma 2 lose more performance compared to arma 3? and not the other way around?

Lower object draw distance = better performance, so no.

[white 1]

Lower object draw distance = better performance, so no.

yeah ive read it wrong, you are right.

but anyway, right now theyre locked togheter in arma 3 which means that if you do a test with 2000 view distance you will get 2000 object distance, so is kind of easy to use both on the same parameters to try it out.

[R4IDER 10]

I started playing ARMA 2 at the end of 2010 and I have been aware of the poor CPU utilization since then, with the release of ARMA 3 I was expecting many of the issues that plagued ARMA 2 to be fixed but they haven't and it doesn't matter how many times people reply with "dude its an ALPHA" I am willing to bet that the same issues will remain when the game switches to BETA and then even the full release. If BI were really aiming to fix the utilization issue then it would be on the known issues page but its not there for the problem will be ignored like it was in ARMA 2, what we need in more threads like this so people can talk about the issues with the game and make sure BI know its a primary problem.

I fear that to many people just think that the issue will be resolved when the game goes into beta and final release but I get the impression that the problem is being swept under a rug, there must be some real limitations with the ARMA engine preventing BI from fixing the issue there for I expect 4-6 years down the line someone will be reading this thread because the utilization issue is still valid.

[el_muerko 0]

I'd love to see some sort of official Benchmark tool that's relatively easy to use and creates a report for BIS, then they might get a decent volume of benchmarks that'd help them expose hardware/software performance bottlenecks.

[jumpinghubert 48]

yeah ive read it wrong, you are right.

but anyway, right now theyre locked togheter in arma 3 which means that if you do a test with 2000 view distance you will get 2000 object distance, so is kind of easy to use both on the same parameters to try it out.

if you want a 2000m object visibility in arma2 you have to put the viewdistance to 4000m.

[ric 1]

I've been playing these games since OFP released so I am well aware of the teething problems that every iteration has.

I suspect you are not.

A game this complex that offers a unique sandbox has to overcome technical limitations that your average corridor shooter does not and when you have developed an engine that can do everything Real Virtuality 4 can while pinning 120FPS, feel free to let us know.

No dev has managed a game on this scale since OFP was released almost 13 years ago.

Yes, there are problems for the umpteenth time but some people are more interested in 'I wa nt, I want' than actually bothering to try and understand how difficult this series of games is to make or why these technical limitations exist.

for starters i am not new to gaming....I use to play chess on a TRS80 that was loaded off a cassette drive and nor am i new to computers as i have always built my own systems. i am also very aware of the complexities involved in making computer games.

Sandbox ..did you say sandbox? have you ever heard of Cliffs of Dover? it is a WW2 air combat simulator and just like A3 it has a ME and is also script friendly along with very high detail both in model and engine management complexity. check out some screen shots http://forum.1cpublishing.eu/showthread.php?t=33161

CLoD was such a mismanaged nightmare that it was released in 2011 with no multicore support,no SLI support,no documentation,no dedicated server software and they tried to make for lost time by finishing the game in C#...you could have 80fps and the game would stutter like mad from inter op problems. to make a long story short the review were so bad along with the performance that 18 months later they pulled the plug and the time of death was pronounced....now the community is salvaging what it can.

why did i tell you about CLoD...because there are some scary similarities here, you cant release a game in 2013 that is hobbled by it own unevolved nature...a 6 year old video card as the minimum?!?!? we have 4/6/8 cores CPU's that can hit 4+ghz and SSD's and video cards with duel GPU's...etc how can this be the next gen mil sim if it is locked in the past? When the original IL2 came out even the stoutest rigs had a hard time playing it at max settings because the game was capable of more than the hardware could deal with but right now the opposite seems Tobe true for A3.

This is not a "teething" issue as you put it...just like CLoD this is a fundamental issue at the core of the engine and again as you put it this not a new issue for the umpteenth time..and as for your perception that we are just here to scream "i want I want" well ...your wrong I for one want Tobe TOTALLY wrong and if i am i will belly up for a healthy dose of crow. I want A3 to succeed becasue there are really no other choices...the BF and COD series are nothing more than nade spamming bunny hoping statserbating score whoring twitch shooters...OH LOOK I GOT ANOTHER @#$@$ING FICTITIOUS RIBBON!

This really is pointless to discuss or argue...we have no control and the die is cast ...as i said we will see what is what in Q3.

---------- Post added at 22:23 ---------- Previous post was at 22:19 ----------

if you want a 2000m object visibility in arma2 you have to put the viewdistance to 4000m.

i see your running at 4.8ghz...how does A3 run for you? and have you compared 4.8

vs stock?

[white 1]

if you want a 2000m object visibility in arma2 you have to put the viewdistance to 4000m.

so if you compare arma 2 to arma 3 and both using 2000 viewdistance, in arma 2 the object distance will be 1000?

well then that would mean that even if the fps were the same, the performance in arma 2 would acctually be inferior since the object distance was in reality half.

and btw thats fucked up, why does it work that way?

[dasa 20]

If you use a ramdrive you'll have very fast loadtimes, but there's also a higher chance paged data that's still in memory gets overwritten because there's not enough memory available to keep everything in. If you actually need the harddrive for critical data you'll get freezes of a second or half a second, very annoying.

for average fps it wont matter much.

your right it made no difference to average fps

same benchmark but used fraps to record from start to finish

i added another 2x4g kit for 16g total made a 8g ramdrive and put the entire arma install on it

this is at 4.7GHz DDR1600

blue line is hdd+ssd cache red lines is the ramdisk

[zebra 10]

... you cant release a game in 2013 that is hobbled by it own unevolved nature...a 6 year old video card as the minimum?!?!? we have 4/6/8 cores CPU's that can hit 4+ghz and SSD's and video cards with duel GPU's...etc how can this be the next gen mil sim if it is locked in the past? When the original IL2 came out even the stoutest rigs had a hard time playing it at max settings because the game was capable of more than the hardware could deal with but right now the opposite seems Tobe true for A3.

So for you it would make sense to have which minimum requirements? Actual gaming-machine for 1200€ ? Did you take only one minute to think about BIS is also about to make some economical gain? They are aready in a quite small niche of interested parties. Now they shall additionally aim to enthusiast gamers, too?

Who has 6 or even 8 cores running towards 4GHZ? (No enthusiast gamer will buy a Bulldozer these days). Which percentage has cards with dual GPUs ?

So how they would aim to 0,1% of the millions gamers ? This is just nonsense.

I have a 5 year old system and can play A3 just fine, for sure with limited grahical pleasure, but happy about it. I guess Iam not alone...

[dnk 13]

It then has to read from the pagefile.

Even if you sat with an API monitor and watched hard drive usage, you wouldn't see a lot of usage because you're measuring bandwidth not transfer latency.

It's possible that highly-used FSMs are being kept on the pagefile, but I would highly doubt such a mistake would not have been corrected long ago by the devs. Those are fairly small files, I believe, and could easily fit on even L1 cache, easily on L3, certainly on main RAM. Now the latency between RAM and the CPU may very well cause performance issues when they're referenced 80 times per simulation step if they aren't being stored on-die.

We've talked about this before. If you can isolate the FSMs from their PBOs and use them as an addon, it's not too hard to figure out if they're really being read often or not from disk (but not RAM - not sure how to check if they're on cache or not).

Otherwise, I agree with the rest (if true). Perhaps it's not the FSMs but some other, larger data that needs referencing through the FSMs that's causing the hangup. There's a lot of data that needs to be saved for each unit (time duration values, states, etc) that might not be able to be saved on-die. If this is the case, there's little BIS can do about hardware limitations (assuming, as I would, they've already optimized cache usage). Many other games wouldn't have this issue due to much simpler AI requiring far fewer reads from off-die. I'm not qualified to say, though, this is something we need a dev to answer.

...a 6 year old video card as the minimum?!?!?

That's not so uncommon. The last STALKER title (which was a bit similar with lots of semi-competent open-world AI and a decent draw distance with complex terrain) had an old FX-series nV card for its minimum. I think those came out in like the 90s or something. By comparison, A3's min specs are much higher, and tbh the game doesn't look half as nice as the STALKER series.

[liquidpinky 11]

Try testing it with your pagefile on the RAMdisk, I akways found that more beneficial than putting the whole game on it.

[Insanatrix 0]

It's possible that highly-used FSMs are being kept on the pagefile, but I would highly doubt such a mistake would not have been corrected long ago by the devs. Those are fairly small files, I believe, and could easily fit on even L1 cache, easily on L3, certainly on main RAM. Now the latency between RAM and the CPU may very well cause performance issues when they're referenced 80 times per simulation step if they aren't being stored on-die.

We've talked about this before. If you can isolate the FSMs from their PBOs and use them as an addon, it's not too hard to figure out if they're really being read often or not from disk (but not RAM - not sure how to check if they're on cache or not).

Otherwise, I agree with the rest (if true). Perhaps it's not the FSMs but some other, larger data that needs referencing through the FSMs that's causing the hangup. There's a lot of data that needs to be saved for each unit (time duration values, states, etc) that might not be able to be saved on-die. If this is the case, there's little BIS can do about hardware limitations (assuming, as I would, they've already optimized cache usage). Many other games wouldn't have this issue due to much simpler AI requiring far fewer reads from off-die. I'm not qualified to say, though, this is something we need a dev to answer.

That's not so uncommon. The last STALKER title (which was a bit similar with lots of semi-competent open-world AI and a decent draw distance with complex terrain) had an old FX-series nV card for its minimum. I think those came out in like the 90s or something. By comparison, A3's min specs are much higher, and tbh the game doesn't look half as nice as the STALKER series.

It depends on how intelligent the caching system is and if there are "types" of data with priority flags or not. Also data doesn't tend to sit in on-die cache for long. It largely depends on the size of the program you're running and the size of the instructions being performed as well as the amount of instructions there could be.

http://en.wikipedia.org/wiki/CPU_cache

http://en.wikipedia.org/wiki/Translation_lookaside_buffer

https://en.wikipedia.org/wiki/Instruction_cycle

Those wiki pages kind of help you understand how CPU cache works. Cache doesn't store the data. Cache store's the instruction's to be done by the CPU both before and after execution. An FSM doesn't reside on CPU cache, it will reside in RAM. Think of it like one of those, "there are 2 trains heading in opposite directions..." type of math questions. The FSM simply holds a problem. The CPU is like your brain, it takes a formula ( Instruction ) and uses that to calculate the answer to the problem. The cache would be like a scratch pad, it holds the formula so you can work through it step by step, and then holds the answer after calculation and allows you to remember it or keep it known so that you can then transfer it or return the answer.

It's purely hypothetical, but it could explain what is happening. This is how I think the engine handles it.

The FSM's being an addon would make no difference, they already are addons to the engine. Every PBO in your "Addons" folder is an addon, they all get loaded the same way. It all gets loaded into virtual memory. It would be insane to try and decompress every time you wanted to load a new piece of data from a .pbo, so you just basically copy the .pbo's to memory with a very fast way to access and decompress specific parts. Think of it like opening up a .rar in Winrar, selecting a file and then decompressing that to disk outside of the .rar. That's basically what the engine does, it loads the .pbo's whole into virtual memory, and pulls out what it needs when it needs it. So you would need a buffer of space in RAM just like you would need a buffer of space on your hard drive to "extract" the file to. You couldn't extract a file to a full hard drive could you?

So if you broke it down, you might have half a pbo sitting in RAM and half sitting paged memory on disk. With Virtual Memory it's all one contiguous block of "Space", with your pagefile acting as an extension of your RAM. You couldn't easily prioritize what's paged and what's not paged, unless you decompressed everything and had it all sitting as single cataloged data files and that would be an ungodly task and your load times would probably be as long as it would take to depbo your entire addons folder, or every .pbo in the ArmA directory. Also larger texture files and models and the graphical stuff that generally takes the most space, would preferably be stored in RAM or mostly all in RAM because the disk access would be ungodly if it wasn't.

So you have a 3gb addressing window for physical RAM, you have to have a "transfer" buffer window in that Physical RAM to be able to store new data, you have to store as much vital data as you can in RAM for fast access, and you have to try and prioritize all of this around a disk caching system so that you minimize the need to stream data. Now granted I'm sure that they try to keep vital data in RAM, but to put it bluntly, almost everything is "Vital". So it's basically like a very very fast game of musical chairs with 3,000 chairs and 6-7,000 people.

What I'm trying to get at here is that when you look at the RV Engine's disk streaming, it's like trying to stick an Elephant in a Prius. You may eventually be able to get it to work, but it's not going to be the most optimal choice, there's going to be little room for error or mistakes and it's certainly not going to have the best performance. There's a reason why RAM exists, to store and feed data on a extremely low latency medium with very high bandwidth. If Hard Disks or even SSD's could replace RAM, why would we even have RAM in the first place. You stop and think about it, RAM is nothing but a small hard drive. The only difference is the extreme low latency, the high bandwidth and the direct nature of communication it has with the CPU. That's the key point.

Edited April 24, 2013 by Insanatrix