Improve framerate limitations with AI

[Dead3yez 0]

Alright. I did a little more testing.

I placed down loads of groups so that my FPS was constrained again. Then, after a massacre of these AI units, there was very little improvement. I went from about 29 FPS to 33 FPS (even at a far distance from them) (Opposed to around 50 when there is zero AI). So even when the AI is dead I'm still seem to be getting this problem.

Can anyone re-create and confirm this?

[.kju 3241]

Rommel made a CIT ticket about that phenomenon.

[Herbal Influence 10]

the game is not optimized for multicore CPUs properly. i think that's the reason. too many single threaded processes. old OFP-engine with some tweaks =( IMHO.

---------- Post added at 05:25 PM ---------- Previous post was at 05:23 PM ----------

our super-pooper modern CPUs can be thrown away with such game engine =)

---------- Post added at 05:29 PM ---------- Previous post was at 05:25 PM ----------

I think, BIS must improve the situation immediately, if they want their game to live a long life... btw, same falls in GPU, as in CPU load when increasing AI number =( BIS are disappointing =(

i hope they are reading this and will say something inspiring about new or upgraded game engine in patches... no new Addon-Campaign-or-else will help here... we want a good game engine! :)

That's what I call bashing without knowing ANY facts. Simply maintaining game isn't optimized and ignoring the fact that it is simply immensely complex. Especially: More complex than any game you compare it with.

I had some loudspeakers (or bashers) calmed down here.

The thread was started because some people maintained that a proof of BISs "poor-optimizing" would be using too few threads. No need to explain how those ended.

Must be hard for BIS to be bashed by cogamers who simply ignore the immense complexity of the game and in the next second ask for more fraps. :(

As of lately I read some cpu-tests who showed that MINIMAL fraps drop when using MORE cores.

EDIT: Perhaps it's simply not easy to understand what is meant by "simply immensely" complex. ;-)

[McArcher 0]

Though I am not a professional programmer, I used to study assembler and C-language in my previous university, and I know something about CPU architechture, so i am not speaking as an insane fan :D

Of course, i dont have sources of Arma2, but I see the compiled results of them :D

There are no complicated tasks in the world, that cannot be divided into sub-tasks if you say that this game is "More complex than any game you compare it with" :)

---------- Post added at 22:24 ---------- Previous post was at 22:20 ----------

I think they just don't want to rewrite it from scratch and optimize, because low level takes too much time.

[galzohar 31]

Obviously if AI only has 1 thread, and that thread must complete a cycle every frame, and since 1 thread can't be simultaneously assigned to multiple cores (it *can* bounce between cores to reduce heating though which is why you don't see a single core at 100%), the more AI you have the longer it takes that 1 thread to complete 1 cycle and thus the more time the other threads are spending idling, which sums up to lower total CPU usage. You'd still get better performance with a faster CPU, though, as that means the AI thread would cycle faster and thus the rest of the game is allowed to continue. I think us quad-core/i7 users would get much better performance in heavy-AI situations if AI actually used more than 1 thread.

What I don't understand (and I actually have done some experimenting that contradicts it (see ACE_Co_05 patrol city, except place 75 AI instead of 35) is why you're getting FPS drops for high AI counts even when the dedicated server is handling them. Like I said on the other thread, when testing this specific mission (except with 75 AI) I was getting smooth FPS when the dedicated server was handling it, but low FPS when hosting it myself. 75 AI is as much as I could put on the server without its FPS dropping below 20, ~35 is as much as I could put when running it on the client without dropping under 20. So I really wonder what aspects of AI cause a strain on both the dedicated server and the client, because in this specific mission the server is taking on the load just fine and the client seems to run as smooth as if there weren't any AI in there.

[McArcher 0]

i think there three moments:

1) the game engine is too old and was written for other purposes (where even no JIP was, I doubt that it was MP-friendly, because server cannot even change clients' positions...that's very strange to me..... game engine is wrong even in basic things, too separated server/client and so on...)

2) BIS must study to programm parallel calculation more intensively :D and especially in low level code (assembler and so on), though it is not so easy.

and

3) all scripts and missions must be written in game's code with built-in-editor compiler to make them less laggy.

(of course it will limit number of addon-makers and reveal some part of engine's code, but it is better for quality of game/missions. but they will never do this, because they have VBS projects and so on.... i think you get the idea why)

Edited January 9, 2010 by McArcher

[Blaaarg 10]

Same here. Single or multiplayer, the game hit a fps cap if these too many AI. I tried to change video settings, tried every single tweaks of this forum, ... not any change. (maybe 1-2 fps maxi). I usually get 25-27 fps on big CTI missions with 32 players. This is not a graphical limitation, without AI i get good fps even with a lot of (empty) vehicule in the field of view.

Q6600 @3.5ghz watercooled

4go ddr2

gtx260

x25-M postville 80go (1 week installation, os + game on it)

Corsair vx550w

Win7 x64

[GLeek 10]

Playing on a dedicated server with many AI will reduce the FPS for the clients almost as playing the mission in single player with all the AI local. On such missions the graphic settings does not make any significant change, and the distance between the AI and the player is not a factor for the FPS.

I made a test mission with 300 AI.

- All AIs are more than 2 kilometers away from the player.

- The viewdistance is 100 meters.

- All graphic settings are reduzed to the lowest, or deaktivated if possible.

- Resulution 640x400.

- After starting the mission i look straight up to the sky to get the maximum FPS.

Test mission: fps_test_300_ai.utes.pbo

Same mission with 0 AI to find the FPS limit: fps_test_0_ai.utes.pbo

Editor view of Test mission:

Test 1:

First i have to find my FPS limit without any AI.

Client hosting the test mission with 0 AI. Framerate = 880 FPS.

Test 2:

Client hosting the test mission with 300 AI. Framerate = 33 FPS.

Test 3:

Client playing the test mission with 300 AI on a dedicated server. Framerate = 36 FPS.

CPU Usage of the dedicated server. The server has 25 FPS.

Test System:

Client: Q9550, ATI4890, 2GB, Windows XP Professional.

Dedicated Server: E8400, 2GB, Windows 2008 Server Standard.

it's so sad ... :( now we know: dedicated doesn't help too much for cti game

[Perfect Dark 0]

so the fps issue has a name now!

The things we need now is an official statement and hopefully some kind of fix or tweaking! :eek:

[Kralizec 10]

So this brings on to our next question:

What part of our rig do we have to upgrade/overclock MOAR to get better FPS? Number of cores/speed of cores/cache size/memory bandwidth? Where exactly is the bottleneck?

(this is more like a rhetorical question, I'm not looking for an answer right away, but it is a very important question)

[das attorney 858]

Better FPS in this instance is determined by number of cores and frequency. The CPU has more AI to process, therefore GPU and model/texture (HDD) retreival speeds are not the restriction. It seems A2 is mostly bottlenecked by processes like scripts and AI calculations, not rendering (at the high end). More horsepower and to a lesser extent, multiple (more than 2) cores are the deciding factor.

[Dead3yez 0]

Better FPS in this instance is determined by number of cores and frequency. The CPU has more AI to process, therefore GPU and model/texture (HDD) retreival speeds are not the restriction. It seems A2 is mostly bottlenecked by processes like scripts and AI calculations, not rendering (at the high end). More horsepower and to a lesser extent, multiple (more than 2) cores are the deciding factor.

Don't worry though. A quad core @ 3.6Ghz still isn't enough. :rolleyes:

[Hbomber110 0]

Same here. Single or multiplayer, the game hit a fps cap if these too many AI. I tried to change video settings, tried every single tweaks of this forum, ... not any change. (maybe 1-2 fps maxi).

Same here:

3.4ghZ Phenom II X4 965 Black Edition

4go DDR3 @1333

HD4890 1go

Win7 64bit

[Yapab 10]

If the AI thread is the bottleneck here, a faster Mhz CPU will do better than a CPU with more cores.

What is the point of more cores when all other 3 cores are waiting for core 0.

What needs to happen is to have the AI thread split into different sections, a complex task and probably wont be done until ARMA3... maybe.

In a game like ARMA2 this should be a priority however, as playing @ 20fps really is not fun, and Quad systems over 3.5ghz do not help, the resources arent being used to deliver maximum performance.

Yapa

[galzohar 31]

If AI are slowing you down, the more MHz the better. Number of cores (above 2) is only relevant if you still feel you need extra performance even without a lot of AI. That is, if you use lots of AI, cores other than the AI core don't really do much. If you only have dual core and run lots of AI, the single core that is left for the rest of the game may or may not be enough, depending on its speed, your graphic settings, and whether the AI core is slowing it down or not.

Bottom line is with the way it is now quad core helps, but speed helps a lot more.

[McArcher 0]

I have seen in many BIS's pseudo-code scripts a lot of "sleep 0.01;" things. I think they are one of the reasons of such phenomena in game, because the more MHz CPU has, the less percentage will all scripts take , but sleep command uses realtime pause in SECONDS/milliseconds, which will be the same on different CPUs, and when you have tonns of AI scripts which use tonns of such "sleep 0.01;"s, it will result in what we see... they will make bigger and bigger pause in milliseconds/seconds...and cpu load will DECREASE....... it is very logical ! now I know one of the reasons :D

---------- Post added at 17:23 ---------- Previous post was at 17:21 ----------

to avoid this, BIS should rewrite all their scripts without sleep usage! but i doubt they will do this...

---------- Post added at 17:27 ---------- Previous post was at 17:23 ----------

but, much better way is to forget about scripts and write in game's code (C++ or what is it), like in some games that I'd told much about. (NWN2 is a good example of a long-life game, because of its easy modification, 3D editor, built-in compiler in comfortable C++ style code).

[dmarkwick 261]

I have seen in many BIS's pseudo-code scripts a lot of "sleep 0.01;" things. I think they are one of the reasons of such phenomena in game, because the more MHz CPU has, the less percentage will all scripts take , but sleep command uses realtime pause in SECONDS/milliseconds, which will be the same on different CPUs, and when you have tonns of AI scripts which use tonns of such "sleep 0.01;"s, it will result in what we see... they will make bigger and bigger pause in milliseconds/seconds...and cpu load will DECREASE....... it is very logical ! now I know one of the reasons :D

---------- Post added at 17:23 ---------- Previous post was at 17:21 ----------

to avoid this, BIS should rewrite all their scripts without sleep usage! but i doubt they will do this...

---------- Post added at 17:27 ---------- Previous post was at 17:23 ----------

but, much better way is to forget about scripts and write in game's code (C++ or what is it), like in some games that I'd told much about. (NWN2 is a good example of a long-life game, because of its easy modification, 3D editor, built-in compiler in comfortable C++ style code).

The sleep command does not literally put ArmA2 in pause mode, only that one single script. It allows the engine to use more CPU power for all the other stuff that is going on. The sleep command is as essential a part of ArmA script as any other.

[McArcher 0]

The sleep command does not literally put ArmA2 in pause mode, only that one single script. It allows the engine to use more CPU power for all the other stuff that is going on. The sleep command is as essential a part of ArmA script as any other.

then maybe the lag is caused by this "next task selection" after sleep command has arrived and counter is active?...

---------- Post added at 17:36 ---------- Previous post was at 17:33 ----------

or maybe AI tries to process too much data syncroniously, which makes it wait for some cores of CPU to wait... maybe they could calculate something else, not just wait... for example some sub tasks of the tasks being processed now...

---------- Post added at 17:37 ---------- Previous post was at 17:36 ----------

it is like a big factory with universal assembly lines.... GPU is btw good example of it.....

if your management is poor, your assembly lines will wait and only "eat" energy and time while being not loaded...

[dmarkwick 261]

then maybe the lag is caused by this "next task selection" after sleep command has arrived and counter is active?...

No, because that next-task selection would happen regardless when the end of the script is reached.

[galzohar 31]

Sleep .01 (unless you have more than 100 FPS) means that the engine waits until the next frame to resume the script, as you cannot interrupt a script for a duration shorter than 1 frame, but sleep (as well as waitUntil) will always halt the script at least until the next frame.

[Flash Thunder 10]

That would explain why everyone gets horrible FPS in the second benchmark theres tons of AI in that! :eek:

I get 5-10 FPS no matter what settings, really pathetic.

but the first one I get 40FPS on HIGH

[ziiip 1]

Deadeye, you must have a pretty impressive machine if you have 40 FPS with 100 AI on high settigns. :D

Anyway, most of the MP games that use AI have horrible performance. CTI, Domination, Evo...freaking 25 FPS. Thats why I kinda semi-abandoned this game(it blows my mind when it runs smooth).

[afp 1]

I have studied this issue too, here are my conclusion:

Causes:

- number of graphic objects when many AI in view. That's it, the more AI in view, the more polygons, the more processing graphic power needed.

- AI processing when hosted server. This involve pathfinding, distances calculations, decisions, generally the FSM stuff. If your computer has to do all these together with graphics processing, you'll get more lag.

Solutions:

1. Use dedicated server on a separate computer. If you do it on the same computer you're playing the game, you will get even more lag because some objects have duplicate processing, in server space and also in client space. If you have a dedicated server, you will get rid of AI processing on your computer and share processing power on multiple processors. This involves a decent bandwith and some tunnings but results are generally better.

2. Do not put all AIs on the mission map, spawn AI when needed. Millions of AI in the same time is not always a good idea. This depends on the mission design, the best thing you can do is to use respawn camps, like the ones in DAC. Instead of having 300 AI simultaneous on map, you can have only 30 and respawn groups as they die, having much better speed and the same feeling. Plus you can have countless AIs. DAC also have others optimisations like zones removal or group reducing, I think zones removal may worth take a look too.

3. Optimise scripts. Some patrols scripts add a lot of lag while big number of human players, consider replacing some patrols with camps or static observers. Try to understand how thread switching works and adapt paralel computing to game engine correctly. Count the number of loops you use and check if they are really needed. Too many features added to a single mission also may impact over performance, if you want artillery, revive, air stuff, etc etc each one of them adds something to server processing.

4. Use AI game engine if possible. If you use some fancy script for each AI on the map, this script will execute and overlap with AI engine processing, as they are not independent each other. This will add a lot of overhead and may lag in the end. Guard waypoints have very good results in terms of AI behaviour, as far as I could notice. Use AI scripts only where needed.

Well, the performances resides in the end in mission design and designer skills. Also, some custom AI systems added to the mission in the right way may have good results in terms of AI behaviour and performance balance.

Edited February 6, 2010 by afp

[galzohar 31]

I think most people here are talking about just placing a bunch of static AI on the map.

Even with what you say it still does not explain why high AI count on a dedicated server clearly drops the FPS on the clients as well.

[afp 1]

I think most people here are talking about just placing a bunch of static AI on the map.

Even with what you say it still does not explain why high AI count on a dedicated server clearly drops the FPS on the clients as well.

Well I guess its because of syncronisations, the position and data of each AI may be send to each client, no matter if it's in view or not. I'm not for static AI around the map but for dynamically generated AI.

Edited February 6, 2010 by afp