Improve framerate limitations with AI

[Dead3yez 0]

Since some recent on-going discussion within the troubleshooting section here, I decide to create a suggestion topic for this.

More info:

Whether in MP or SP, when there is a large amount of AI ~60+ FPS becomes limited to ~30. Changing the graphic settings does not improve performance when this happens, settings can be very low to very high and the persistent ~25-30 FPS will be showing. This can be somewhat annoying to not be able to play the game smoothly with systems which can perform better. In MP domination is the perfect example where the same issue would be expected.

It would be nice to see some improved optimisations made in future patches regarding this issue.

Here are some screenshots demonstrating. lowest video settings vs high video settings / amount of AI:

http://www.armaholic.com/datas/users/3321-squadailow.jpg

http://www.armaholic.com/datas/users/3321-squadaihigh.jpg

http://www.armaholic.com/datas/users/3321-someailowgfx.jpg

http://www.armaholic.com/datas/users/3321-someaihighgfx.jpg

http://www.armaholic.com/datas/users/3321-moreailowgfx.jpg

http://www.armaholic.com/datas/users/3321-moreaihighgfx.jpg

http://www.armaholic.com/datas/users/3321-loadsailowgfx.jpg

http://www.armaholic.com/datas/users/3321-loadsaihighgfx.jpg

My PC specifications:

AMD Phenom II X4 965 BE @ 3.55Ghz

Asus M4a785td-v evo

4Gb DDR3 Ram

260GTX 896mb

x-fi xtreme gamer

XP32

All BIOS and drivers updated to latest version. Everything running in tip-top condition.

If anyone else has these issues then please specify your CPU model in particular, etc. :)

I have also created an issue at dev-heaven, please head over there, sign in and vote/comment on it if this effects you as well:

http://dev-heaven.net/issues/show/6963

Edited December 15, 2009 by Dead3yez

[jasonnoguchi 11]

I seem experience the exact same limitations and our system specs are somewhat similiar... does people with very different specs have the same experience? How about the i7 guys?

[johncage 30]

yes. the more ai, the more lag. it's been like this with all the games. even ofp.

this game needs optimization now more than ever because the engine is built upon such old technology.

[Dead3yez 0]

yes. the more ai, the more lag. it's been like this with all the games. even ofp.

this game needs optimization now more than ever because the engine is built upon such old technology.

Yes, but we are discussing how the FPS is not affected by the video settings.

This is a case of ArmAII bottlenecking performance.

[edit]

NO, this particular issue does not happen in ArmA 1.

Edited December 15, 2009 by Dead3yez

[.kju 3245]

See my comments in your CIT ticket.

I think you are asking the wrong questions or looking for the wrong solutions.

If you want to have mass AI, you need to outsource it to a dedicated server with massive power.

The reason is people asked for more capable AI.

So BI made micro AI to put more calculations into AI behavior (= more CPU use).

As compromise they decoupled all the other processing to different CPUs (multicore use).

Of course you can say, the AI should getting more dump again, the more AI you put,

or the less FPS you have or even deactivate micro AI calc altogether.

Probably to some degree that is already the case. For the rest the AI was not designed to work

that way I guess.

Even with multicore the different CPU caches needed to be synced with each other,

and synced with the GPU.

In other words it is unrealistic to place massive amounts of AI and expect no impact on FPS.

Does the system has problems/bugs, could it be improved, optimized, designed different

- I think only BI can tell.

To help them in this task, you can submit information in a way useful for them. See the CIT.

PS: If you are only talking about MP dedicated server use, and seeing this as a client,

you should make this very clear and provide the reproduction information for it.

Again it is described in the CIT wiki.

Edited December 15, 2009 by kju

[Dead3yez 0]

Thanks for the info Kju, I will try to post more information later in the day.

May I ask if you have done the same tests with AI, then found that not altering your video settings increased or decreased the performance of frame rates?

I really have no idea what could be bottlenecking this CPU usage is never 100%. Unless I put 1000s of AI down etc.

PS: If you are only talking about MP dedicated server use, and seeing this as a client,

you should make this very clear and provide the reproduction information for it.

Again it is described in the CIT wiki.

Domination is a common mission that re-creates this problem. Whether I run the mission locally on my machine myself, or join a dedicated server to play. It also happens on many other missions online, specifically the ones with more AI etc.

[.kju 3245]

If you read Suma blog post, he explains that 100% CPU is neither easy to achieve nor good.

If you see the positive side, you can bump up most of the time your GPU settings without

(much) FPS impact (object detail and terrain detail (grass) do matter a lot still).

At some point you run into lower FPS with higher AI numbers.

From my impression a1 had less of an impact than a2 in terms of AI numbers.

I personally would prefer more AI to more complex micro AI.

So one theoretical approach is to reduce or disable the micro AI calculations.

At least for units not close to player units. Yet probably already done to some degree

and pretty complex to do.

My recommendation is to test (first) with a dedicated server.

Or use a local dedicated server to test and use task manager to assign 2 cores to

the client and 2 cores to the dedicated server.

[RavoC 10]

Processor:

Intel® Core2 Duo CPU E8500 @ 3.16GHz (2 CPUs), ~3.2GHz

Memory:

4096MB RAM

Hard Drive:

500 GB

Video Card:

EVGA Nvidia GeForce GTX 260 896MB

Operating System:

Windows Vistaâ„¢ Ultimate + Service Pack 2

Motherboard:

Asus P5K, P35, Socket-775, DDR2, ATX, GbLan, PCI-ex16 demo

I got almost the same specs as you and I get the same problem. I can run everything on very high and sometimes run 40 frames on Chernarus without any problem but when more player or AI are in it calculates the whole map which means I run around 18 fps and sometimes 8 in the capital.

With all on low even render resolution on 50% I get 30 in campaign but there is a limiter on there but anyway I expect a empty map to run 80 at least on lowest graphics, and I mean lowest and not 45 or something, it really depends on what you're looking at, big city or a empty field.

Just hope new patch with the texture render improves it, beta patch made me stutter much less when coming close to the capital. Now I just stutter because it's loading and I can feel that in the air!:eek:

[Dead3yez 0]

I have obtained some results from a test to reproduce the issue. I will add them to the report over at dev-heaven when I have completed more tests and obtained more results.

Here is the mission:

http://www.filefront.com/15144915/Fpstest1.7z

It will tell you when to start fraps and when to stop it. All this does it create AI, wait 10 seconds at the follow points 25,50,75,100,150,200,500. Then it creates more AI to the next point and waits ten seconds again to record FPS. You will be staring at some sandy terrain and user input will be disabled whilst the test is running.

Here are my results.

Edited December 16, 2009 by Dead3yez

[-=seany=- 5]

I think you are asking the wrong questions or looking for the wrong solutions.

If you want to have mass AI, you need to outsource it to a dedicated server with massive power.

PS: If you are only talking about MP dedicated server use, and seeing this as a client,

you should make this very clear and provide the reproduction information for it.

Again it is described in the CIT wiki.

Yeah, this is my biggest gripe with the issue that Dead3yez is talking about. It seems that dedicated servers with massive power dont do squat as far as AI calculations. Or maybe, no one is running a coop server on a powerful enough server yet.

I dont think that they could "fix" the CPU bottle neck with lots of AI that lowering GFX settings has no affect on. Many games are like this when you heavily tax either the CPU(changing GFX settings have no affect on FPS) or the GPU (lower the number of CPU calculated objects will also have no affect of GFX)

[.kju 3245]

You can capture ingame via scripting frames too:

http://community.bistudio.com/wiki/diag_tickTime

http://community.bistudio.com/wiki/diag_frameno

http://community.bistudio.com/wiki/time

save to rpt with

http://community.bistudio.com/wiki/diag_log

Overall the CPU vs GPU use might be interesting.

Maybe RAM, VRAM and other details too.

[snkman 351]

Well thats the way it is.

A2 really need a lot of resources thats true and so far there is no other game out there which brings my system to it's limits like A2 do.

But hey be happy this means we need much better machines like we have now.

For example: Back in the O.F.P. day's i was never able to enjoy O.F.P. with settings on full, wide view distance and DXDLL toghetter.

Today i can play O.F.P. with every setting i ever want and i still have 80 FPS and more.

I personally would say that it's good to have a game which you can not enjoy to it's full beauty directly after buying it.

This gives me some kind of "damn one day i will be able to enjoy this game in it's full beauty" feeling.

Of course B.I.S. really need to improve some more things of A2 but so far i'm very happy with A2.

[Dead3yez 0]

The next piece of the puzzle:

The more AI, the less CPU usage?

:391:

From beginning to end, these graphs are the same length and in sync with the graph abov, from the mission I posted earlier. Here is the CPU usage against the FPS limitations.

And the RAM usage (only the last quarter of the graph counts):

http://www.armaholic.com/datas/users/3321-ramusageai.jpg

Stay tuned until next time, whilst I find out how to record the GPU usage.

[edit]

whoopz, it seems i did the tests with cpucount=3, howeve,r i repeated them with cpucount4 and in principle there is no difference in the observations.

It would be nice to see some results from other people using this mission:

http://www.filefront.com/15144915/Fpstest1.7z

Edited December 16, 2009 by Dead3yez

[.kju 3245]

Could be effects of synchronization becoming harder and harder for the CPU/cache.

[76 0]

Or maybe, no one is running a coop server on a powerful enough server yet.

For all the Aussie players, The OGN ArmA members put in and came up with $4300 for a new server... so when OGN comes back online (thx to TSN) we will see how well our new server stacks up...

- ASUS P6T DELUXE V2 MB

- Intel Core i7 Extreme 975 CPU, 3.33 GHz

- 2x (raid) Intel X25E SSD's 32GB

- Corsair DOMINATOR-GT 6GB Kit DDR3

[ActionMan 10]

Here is the CPU usage against the FPS limitations.

CPU usage (as measured by the task manager) is actually a pretty useless metric, it's borderline meaningless to a programmer.

If a core is stalling (doing nothing, but waiting for another core), that shows up as used CPU time on these graphs. So if you've got an inefficient program where two threads are constantly waiting for each other (but not getting any work done), it will show up as 100% CPU usage.

You can have an application that uses 100% CPU and one that uses 10% CPU, but both get the same amount of work done in the same time.

For all we know, these graphs could be demonstrating that the Arma2 threading system becomes *more efficient* as you add more AI! (i.e. there is less stalling/overheads with more AI characters).

For all the Aussie players, The OGN ArmA members put in and came up with $4300 for a new server... so when OGN comes back online (thx to TSN) we will see how well our new server stacks up...

The OGN server is coming back!? Oh how I've missed it :D

Guess I should actually join/read the OGN forums... :o

[76 0]

The OGN server is coming back!? Oh how I've missed it :D

Guess I should actually join/read the OGN forums... :o

Yes you should! :) but all of OGN is down atm, we are moving in with TSN as our previous provider was just a temporary solution...

The fastest ArmA2 server in Oz coupled with our new very generous ISP.... OGN will own :)

sorry off topic... back to the FPS and AI :)

[Dead3yez 0]

With this new test:

http://www.filefront.com/15145697/Fpstest2.7z

Just to show what I mean some more. (If I had done these tests over a larger timescale like in my previous graph, this would probably be much more clear.)

I'm just showing, the difference noticable with the amount of almost idle AI between 0-80 is Nothing.

But just adding 20 more AI can slice the frame rates in half.

And video settings will have no play as shown in increasing or decreasing FPS (if can perform better).

Edited December 16, 2009 by Dead3yez

[11aTony 0]

I have played Warfare BE last time and everything was good with 4000m VD untill I discovered enemy base. Everything started lagging a bit and I lowered VD to 1600 and it was better... So wtf? I could be wrong but lowering VD seemed to do some good becouse I was watching their base from a hill and there was bunch of enemy T72s rolling to me. So I lowered VD, I couldnt see them but they were still on radar and it was a bit better. I will trie to measure FPS and such next time I play it and get some figures together.

Core 2 Quad Q9450

GTX 260

4GB RAM

P5N-T Deluxe (nForce 780i)

[Dead3yez 0]

More graphs. :D

CPU and dedicated server to compare with previous CPU behaviour.

CPU split (2CPUs each).

ArmA2server.exe looks like it is increasing CPU whilst Arma2.exe is decreasing for some reason.

Test1: 25 - 500 AI

Test2: 50 - 100 AI

[HeinBloed 0]

Playing on a dedicated server with many AI will reduce the FPS for the clients almost as playing the mission in single player with all the AI local. On such missions the graphic settings does not make any significant change, and the distance between the AI and the player is not a factor for the FPS.

I made a test mission with 300 AI.

- All AIs are more than 2 kilometers away from the player.

- The viewdistance is 100 meters.

- All graphic settings are reduzed to the lowest, or deaktivated if possible.

- Resulution 640x400.

- After starting the mission i look straight up to the sky to get the maximum FPS.

Test mission: fps_test_300_ai.utes.pbo

Same mission with 0 AI to find the FPS limit: fps_test_0_ai.utes.pbo

Editor view of Test mission:

Test 1:

First i have to find my FPS limit without any AI.

Client hosting the test mission with 0 AI. Framerate = 880 FPS.

CPU Usage:

Test 2:

Client hosting the test mission with 300 AI. Framerate = 33 FPS.

CPU Usage:

Test 3:

Client playing the test mission with 300 AI on a dedicated server. Framerate = 36 FPS.

CPU Usage:

CPU Usage of the dedicated server. The server has 25 FPS.

Test System:

Client: Q9550, ATI4890, 2GB, Windows XP Professional.

Dedicated Server: E8400, 2GB, Windows 2008 Server Standard.

[-=seany=- 5]

Excellent representation of the problem. Now if only we could get a Dev to tell us why it's like this. Is it supposed to be like this, or is it some thing that can be fixed?

To an non-programmer those graphs look very odd, why does the usage drop so low when the cpu is being used more? And why can't clients get a significant boost when playing on a dedicated server?

[.kju 3245]

Thank you HeinBloed. :)

[dayglow 2]

I think it is very simple to explain. As reported beofre by devs that program threads are tied together. As you add more AI the AI thread takes longer and more resources to caculate. Since other threads such as the render thread are tied to info from the AI thread they slow down as they wait for input.

Very simply we cannot have our cake and eat it too. One of the features of A2 is the complex, free form, non scripted AI. That takes a lot of processing power. As CPU power goes up so will fps.

[McArcher 0]

the game is not optimized for multicore CPUs properly. i think that's the reason. too many single threaded processes. old OFP-engine with some tweaks =( IMHO.

---------- Post added at 05:25 PM ---------- Previous post was at 05:23 PM ----------

our super-pooper modern CPUs can be thrown away with such game engine =)

---------- Post added at 05:29 PM ---------- Previous post was at 05:25 PM ----------

I think, BIS must improve the situation immediately, if they want their game to live a long life... btw, same falls in GPU, as in CPU load when increasing AI number =( BIS are disappointing =(

i hope they are reading this and will say something inspiring about new or upgraded game engine in patches... no new Addon-Campaign-or-else will help here... we want a good game engine! :)