Advanced performance investigation (data needed)

[Der_Richter 1]

Greetings.

First of all let me say congratulations to BIS on a successful (judged by the large interest online in the product) release of the Alpha!

Then let may say that if this is in the wrong category (was deciding between here and Alpha subforum), please help me move it.

I really like the Alpha so far and i can clearly see the efforts made to streamline the experience for new and old users alike. It's looking

good.

However, being a former simulation developer (defence contractor) myself i as usual have a large interest in the technical

aspects of the simulation. I'm an avid simulator user (be it flight, milsim, racing etc.) and that is about the only type of software i

run for entertainment.

I have actually helped in some other alpha/beta programs, in my spare time, to look for optimizations and bugs etc. due to the fact that i have

access to some specialized hardware to bechmark and analyze bus bandwidth etc. (eg. specialized monitoring boards).

Since i have enjoyed the ArmA-verse since OfP days, i feel that now that the chance is given it is given that i should try to help in ArmA3

Alpha development and bug testing as good as i can.

I found some free time yesterday evening (well not really "free" but it was prioritized :) ) to start testing on a prepared benching system

i connected to my reference computer (my own home computer).

I am still crunching the numbers, but it seems that issues exist surrounding AI. If i look at the initial graphs for CPU LOT (Load over time)

i see that when alone on the map, the player can set 10k of View Distance and all details max on GPU

without issues (using at least 30fps+ as reference).

However if adding AI (tested by playing scenarios included) the VD can be as low as 3800 with 3900 object distance and run smooth, UNTIL you

look in a direction where AI (any AI) would be present. Then it's instant drop to 25-ish FPS (in Helicopter scenario as low as 19ish).

Looking at the Tiger VMk2 Virtual test board graph output i see congestion on CPU and loads of it. The GPU is keeping up even despite huge

amounts of high detailed objects so at least this is a good sign. But the CPU congestion and core utilization is screaming red figures

at me.

I will continue to help in any way i can and i just wanted to share these first findings.

However, i would like som more reference systems to compare the test rig with. Perhaps this could be a general thread for these

comparisons as well. I will keep it as my vent for tests i do, to try to help.

If you would like to compare this is the spec. system:

I5 3570K @ 4 GHz. on all cores (no throttling)

GeForce GTX680 4Gb. VRAM with latest WHQL drivers

MSI Z77A-S01 motherboard with attached Tiger VMk2 and SEcCSys CU LOAD analyzer boards (not when playing for fun obviously :) )

Crucial 1600 Mhz. DDR3 RAM

Samsung 568Mb/s. SATA-3 SSD as ArmA 3 install drive

and these are the settings:

No launch parameters added

All graphical settings on max/ultra with 1920x1080 resolution and 8x AA combined with FXAA Ultra

View Distance varying but for test spec. 3800 with object distance 3900 and shadow distance 100

Can you also try this on your computer giving my average FPS on the scenarios and your specs. and i should be able to cross compare with my

data, to try and find bottlenecks to help BIS if i can. The main issue however seems to be core utilization for the AI at the moment...

The average for spec. system is 35-50 FPS. with exception of helicopter scenario (16-25 fps.)

Investigation continues! :)

Kind Regards

Richard

Edited March 6, 2013 by Der_Richter

[Der_Richter 1]

Further findings:

PiP in a vehicle increases CPU usage by nearly 45% on spec. machine. Something seems odd, because regardless of quality in PiP it kills the CPU. Disabling it takes the load off.

I can see that it hooks a pipeline in the GPU driver that looks remarkably similiar to the pipeline i think is for the main viewport. Could it be that it is actually just rendering a second viewport (with the same Viewdistance as main)?

[tansvanio 1]

I have a worst system than yours, but I experienced almost exactly what you describe. I will do some testing again and post the results in here, along with my specs.

[cruzifer 0]

same issue here on my rig (i7 920 @3.4GH, sabertooth 57, gtx 670, 24 gig ram). massive framerate drop with ai. also the pip ist causing alot of slowdown.

cheers cruzifer

[maxx2504 10]

Hi !

Same problems here. Singleplayer works lot better than multiplayer. I have always high Pings on the Servers...and when im host...the Frames gets down to nearly 5fps.

My System is not the best, but i can play other games (incl. Arma 2, Mechwarrior Online, Crysis3) without any problems:

Win7 64bit

AMD Phenom II X4 955 @3,6GHz

AMD 7870 (not OC)

8GB Ram 1333MHz

My graphicsettings are all standard to high. Textures very high, AA is off, PPAA to high, Anisotrop to high, Objectrange only 500m, viewdistance around 2000m.

I tried hosting without AI, set priority of arma 3 in taskmanager to high, hosting only with 2 players, massive reduce my graphic settings...nothing helps in MP.

So the only chance is to wait for Fixes :)

maxx

PS: I mentioned that this forum is also slow since ArmA3 Alpha release...or did i miss something ?

[Der_Richter 1]

same issue here on my rig (i7 920 @3.4GH, sabertooth 57, gtx 670, 24 gig ram). massive framerate drop with ai. also the pip ist causing alot of slowdown.

cheers cruzifer

Interesting. Your high amount of RAM would really rule out a memory size issue (that could theoretically be the case on my 8Gb. test machine). Indeed it looks CPU bound.

For reference i actually managed to capture it happening with the CU monitoring equipment, acanning directly on the FSB and back-bus for instructions and data coming to CPU and possible congestion (CPU out of step with data, buffer backed out):

If interested parties reading, this is capture from CPU-bus and congestion data as viewpoint is turned from no AI in view to small group of AI in view:

CUL001 on CPUID,65 -Report:

0x08000 Entering run...

0x?? State (100%)

0x?? Run...

Run...

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

/cap bus01

BUS01: 32% (unloaded) <- No AI

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

0x?? State(100%)

/cap bus01

BUS01: 32% (unloaded)

0x?? State(100%)

0x?? State(100%)

0z LOCK

LOCK

0x?? State(32%)

/cap bus01

BUS01: 100%(congestion - full load***WARNING99: freezeCong) <- Here i look at backout from CPU when AI comes into view

0x?? State(32%) <- 32% utilization = buffer is overworked

0x?? State(100%)

0x?? State(100%) <- Return to normal, buffer keeps up

[Bee8190 10]

Find this thread interesting but you state that date are needed, how exactly could I contribute to the testing aside of mentioning my rig specs / resolution and so on?

[4 IN 1 0]

Further more did anyone open bug tracking tickets for this?

[Der_Richter 1]

Find this thread interesting but you state that date are needed, how exactly could I contribute to the testing aside of mentioning my rig specs / resolution and so on?

At the moment it would be really helpful to get avg. Fps for the different scenarios in the alpha, using the full settings in original post along with specs of running machine. I need some more baselines to see if there is a large spread compared to my only reference at the moment...

I also noticed just now that there is a possible memory leak when spawning ai, causing memory to keep being locked by unusable resources. At least it looks that way. I could be wrong on this though, but if anyone wants to help out further, please use the editor to spawn yourself in an empty map... Notice fps over a period of say 5 minutes. Then add a trigger and have its activation set condition of presence for some ai... Activate it.... And keep following framerate, 5 mins before and 5 mins after activating the trigger/ai. Is there a change in behaviour and fluctuations in framerate?

[0xDEAD 10]

Is there a way to view fps in game or do I need an application like fraps to get that information?

[Der_Richter 1]

Is there a way to view fps in game or do I need an application like fraps to get that information?

I think tou could script a frame counter, but i have always used external software. For this i would prefer fraps...

[AussieStig 10]

My Specs are as follows:

-Windows 8 64-bit

-AMD FX-6100 @3.6ghz

-GTX 560ti

-8gb 1600mhz ram

I'm currently running the game in mostly standard settings, with 1500m view distance.

My findings have been pretty much the same as you OP. I get roughly 40-60fps while in the editor without any AI, depending on where I am. As soon as I start adding AI, it starts to drop down a lot, and at about 50 AI I go below 30 fps, down from 60 on the airfield. In the 2 online game modes that were released with the Alpha, I found the same thing happening, I only get 20-30fps, depending on the server, how long it's been up, and where I was, but never above 30fps.

Naturally, I thought this has to be an AI issue. But I decided I'd go on some wasteland, and TDM servers. Some with max players, some with very few players. Again I found that I was only getting 20-30fps, even on servers that have literally no AI.

I don't have any idea what the issue could be here, but I feel that AI optimization is only one part of the issue, and there is more to it.

As reference, I regularly played Arma 2 with medium-high settings, 5000 view distance, on servers with 100's of AI at around 50-60fps constant.

[djdafi 1]

You can use MSI Afterburner. Well, I generally use it to monitor my GPU temp. but the FPS counter is also optional.

I have noticed the the game bogs down slightly when in the presence of AI. I'll have to test how well Ultra High works without any AI at all.

[R4IDER 10]

I have a similar system to the OP's I will run some tests when I get home and post the results.

[Der_Richter 1]

I have a similar system to the OP's I will run some tests when I get home and post the results.

Interesting. How did it go?

[EmirSc 10]

Is there a way to view fps in game or do I need an application like fraps to get that information?

go for MSI afterburner. can check fps, GPU temp. clocks, usage and more.

[sbobovyc 1]

Put this in a playable unit's init:

thread = [] spawn { while {true} do {hint str (diag_fps); sleep .2; };};

[paradox87 0]

Since the helicopter showcase seems to be the most demanding on my system I decided I'd run a test using the settings you posted.

My system specs are:

i7 2600k @ 4.0ghz

2x GTX 570 in SLI

8gb G.Skill 1600hz memory

FPS for the mission (Ultra everything + 8x AA and FXAA Ultra):

Low - 15 FPS

Avg - 24 FPS

High - 33 FPS

The average GPU usage was right around 30-35%.

Edited March 11, 2013 by paradox87

[Zordon 1]

insn't arma 3 a 32 bits application? if so having 24 Gb of ram won't really help the performance that much.

Naturally, I thought this has to be an AI issue. But I decided I'd go on some wasteland, and TDM servers. Some with max players, some with very few players. Again I found that I was only getting 20-30fps, even on servers that have literally no AI.

I don't have any idea what the issue could be here, but I feel that AI optimization is only one part of the issue, and there is more to it.

As reference, I regularly played Arma 2 with medium-high settings, 5000 view distance, on servers with 100's of AI at around 50-60fps constant.

It could also mean that the server was running for a few hours, or the host's computer can't handle the server. It's really hard to test with other peoples servers unless you know for fact they have a dedicated host or a good computer.

Edited March 11, 2013 by Zordon

[tremanarch 6]

Interesting. Your high amount of RAM would really rule out a memory size issue (that could theoretically be the case on my 8Gb. test machine). Indeed it looks CPU bound.

did you compare SSD vs HDD?

I noticed a huge fps increase with a SSD in ArmA 2. ArmA doesnt work like other games where first all stuff gets loaded. it streams constantly data, maybe if the stream is too slow the cpu has to wait too often?

---------- Post added at 16:46 ---------- Previous post was at 16:44 ----------

At the moment it would be really helpful to get avg. Fps for the different scenarios in the alpha, using the full settings in original post along with specs of running machine. I need some more baselines to see if there is a large spread compared to my only reference at the moment...

the give the exact settings and lets say we record the fps of the first 3 minutes? of the air mission and let calculate the avg fps as its fps / time.

this isnt possible:

View Distance varying but for test spec. 3800 with object distance 3900 and shadow distance 100

____

I benchmarked:

My System

2500K @ 4.5 ghz

8 GB RAM

560TI (@905 MHZ gpu and 2120 MHZ mem and 1810 Shader clk)

using SSD (win and game and steam on same SSD)

what I did:

start heli showcase; run to heli enter it; watch in the cockpit left <-> right then center throughout the game; flew to first WP straight. 60 seconds over when radio message appears..

I recorded fps for 60 Seconds.

In Game settings: ALL MAX (like in first post) but Vieweing distance and Obj. Dist. 3900 / shadow 100

this is the result:

2013-03-11 16:52:40 - arma3

Frames: 1055 - Time: 60000ms - Avg: 17.583 - Min: 13 - Max: 30

Edited March 11, 2013 by tremanarch

[Der_Richter 1]

did you compare SSD vs HDD?

I noticed a huge fps increase with a SSD in ArmA 2. ArmA doesnt work like other games where first all stuff gets loaded. it streams constantly data, maybe if the stream is too slow the cpu has to wait too often?

---------- Post added at 16:46 ---------- Previous post was at 16:44 ----------

the give the exact settings and lets say we record the fps of the first 3 minutes? of the air mission and let calculate the avg fps as its fps / time.

Test system uses SSD SATA3 568 MB/s r/w. I wrote settings in first post i believe, but i like your idea of measuring an estimate... Helicopter mission is indeed a good basis as it it easily the most intense.

Settings to use:

Res - 1920x1080

AA - 8x

Post Processing - FXAA Ultra

Quality Settings - All max, Ultra where applicable

View Distance Overall - 4000m

View Distance Objects - 3200m

Running this on the test system yields following over 3 mins of flying towards first objective (across the airfield and town):

MIN: 19

AVG: 24

MAX: 38

Measured using FRAPS.

Interesting side note is that the CPU core utilization according to the monitor board is only ~45% total, however load on bus (queue for CPU) is 100%

[tremanarch 6]

okay 3minutes with new viewing distance is

2013-03-11 17:08:33 - arma3

Frames: 3769 - Time: 240000ms - Avg: 15.704 - Min: 0 - Max: 39

but I think 60 seconds should be enough.

maybe we could record just 60 seconds doing nothing standing at place.

as soon as the 3d world appears after sattelite we hit the button and wait. then it should be easiest to reproduce. looking at the sky etc give an avg fps boost.

----->

2500K @ 4.5 ghz

8 GB RAM

560TI (@905 MHZ gpu and 2120 MHZ mem and 1810 Shader clk)

using SSD (win and game and steam on same SSD)

2013-03-11 17:19:30 - arma3

Frames: 1090 - Time: 60000ms - Avg: 18.167 - Min: 16 - Max: 20

Heli mission record at start do nothing.

Edited March 11, 2013 by tremanarch

[spamurai 3]

Maybe someone can physically create a simple, but standardized benchmarking scenario mission that everyone can run.

That way, we have a standardized reference point. Seems a little redundant to solicit performance stats for comparison if there is no control over the environment the stats where gathered from.

Also, V-Sync being on or off will manipulate the MIN/AVG/MAX data collected as it tries to mechanically throttle frame rate in multiples of 15.

[tremanarch 6]

in my eyes heli showcase 60 seconds doing nothing VSYNC off and the rest Ultra and these view settings should be enough.

if you think heli show case 60 seconds is often different we could record it 5-6 times and see how it differs...

[Der_Richter 1]

Maybe someone can physically create a simple, but standardized benchmarking scenario mission that everyone can run.

That way, we have a standardized reference point. Seems a little redundant to solicit performance stats for comparison if there is no control over the environment the stats where gathered from.

Also, V-Sync being on or off will manipulate the MIN/AVG/MAX data collected as it tries to mechanically throttle frame rate in multiples of 15.

I like this approach. Perhaps we can make a mission file where the player will stand still and record fps as soon as 3d world loads, while a set number of AI units in different distances from the player move towards waypoints and engage in firefights. The average of three such runs (to account for dynamic AI) should give a close approximation?

It's also interesting if CPU and GPU loads could be measured as averages as well... I can do it by hardware boards, very precise, but i guess there is software others may use to compare? Is this an option in Afterburner (have not used this software)?

---------- Post added at 00:12 ---------- Previous post was at 00:09 ----------

This as well... Is an approach, as that mission is already there. It is very interesting to me if someone can record the load seen on CPU at the heli mission, for all cores (if multi-core) and total. I see, as stated, values as low as 45% total. Does seem like there is an issue somewhare. The amount of indivdual threads is 37 however.