ArmA2 FPS analyser - benchmarking tool

[sbsmac 0]

Great. thanks langgis. I will update the tool shortly.

[old_painless 182]

Hello, this seems like a very promising tool! I just tried it, though, and ran into problems. I downloaded the archive and ran setup, I then got that the Analyzer app was installed.

I copied the 3 BM-missions to the Missions folder and started up Arma2 with no shortcut, just doubleclicked the exe. I selected the city run mission. From there on I just got "Receiving ..." on a black screen for like 20 minutes. After pressing escape I CTD'ed.

Any advice, please?

Cheers,

OP

EDIT: Got it working - restarted the PC, that helped.

Edited May 16, 2010 by Old_Painless

[sbsmac 0]

I've updated the tool to hopefully work correctly in regions (such as parts of Europe) where floating point numbers don't normally have the form "1.23". I think this should resolve the problems seen by thebarricade, langgis08 and alex72.

The tool should automatically update next time you start it.

OldPainless - do you get this problem all the time? I've occasionally had missions that acted like that but it doesn't seem to be reproducible. If you are still seeing it, I would be interested to send you some test missions to try to narrow it down.

[old_painless 182]

Thanks, I've got it working now. Just had to reboot my system.

On another note, I wanted to delete the rpt file before trying it out as you recommend. Where is that usually located, I searched quite a bit for it?

-OP

[sbsmac 0]

It depends on what your OS is.

For Vista it is under C:\Users\<your login name>\appdata\Local\ArmA 2\arma2.rpt

Sorry - I can't remember where it is for XP or Win7.

[old_painless 182]

Aha, brilliant. Found it :)

Will try it out now with a clean rpt file. As for usage, you say that we should keep Arma2 running, Tab out of it and run the tool. Will the tool not work if we close down Arma2 and then run the tool?

[sbsmac 0]

Yes - it will work if you shut down ArmA2 first. In fact that's a better way to do it :-)

[old_painless 182]

OK, Cheers.

[suma 8]

I understand the affect of 'acctime' of course, and also notice that if the game is actually halted (eg, if it is dragged whilst windowed) 'time' is halted. Other than that is it an inherently unreliable measure ?

Yes, it is inherently unreliable. There are other edge causes besides of time acceleration. One case which is of particular importance here (and which was the reason why we added the new functions while developing internal automatic fps testing) is time is unreliable once game fps is low. The problem is once simulation is using all available CPU power and running on the largest possible time delta, there is no way game time could keep up with the real world time and game time must be slowed down. In practice this means that once the game is running under ~10 fps, the game time will be running slower, therefore if you use it for bechmarking, the fps will be seemingly higher than it really is.

[sbsmac 0]

Thanks Suma, I really appreciate the information. Benchmarks have now been modified to used diag_ticktime so should be more accurate.

[sickboy 13]

What I would be interested in would be measuring the current load of scripting engine, to measure the performance of the running scripts.

I'd want to use it to make simple comparisons between vanilla, cba, cba+ace, aswell as compare different versions / different systems on / off, in different situations, to measure the effect these scripts have on the performance and especially delays.

Since the spawned scripting is now scheduled, FPS is no measurement for it, maybe it is something your tool could support, or if anyone has some ideas on how to accomplish this?

[sbsmac 0]

That's an interesting idea Sickboy. Unfortunately it's usually very difficult to measure overall scheduler load from within a scheduled thread ! What you really need to do is read the start and end time of each scheduler slice but it's impractical to do this from within the scripting environment - there isn't any way to force the execution of code at the beginning and end of a slice. What might work would be if a new diag function was added...

diag_TimesliceInfo - the return value would be an array containing the start and end timestamps of the previous timeslice, eg [5.678,5.683]. Alternatively, the return value could just be the duration but I prefer to see start and end.

If that was available, it would be very easy to plot script usage in a number of ways.

As a poor substitute, it is already possible to measure individual script performance by adding diagnostics at the start and end of the script and I could easily modify the tool to allow this to be output and plotted. The drawback is that any system of scripts complex enough to be interesting is also complex enough that instrumenting it completely is impractical.

Let me have a think on it... :)

*Edit* What would simplify the above problem is something like diag_scriptTimes where the return value is an array of script handles and duration used during the previous timeslice.

Even this isn't perfect - there is a chance that the monitor script will not get the opportunity to run during any given timeslice but this condition could be detected by including a timeslice-id in the results array. I'll put some more thought into this and raise a CIT ticket.

Edited May 17, 2010 by sbsmac

[sbsmac 0]

Here's a mock-up of what it could look like (with randomly-generated data in this case).

[Alex72 1]

Thank you sbsmac for the fix. All good now.

Could you expand the highest FPS value in the charts? 100FPS is max and it seems i have dots in those highest values wich could/should mean i had higher FPS than that. Could be nice to see maximum. Maybe 200FPS could be maximum? Thinking of own benchies people will make where you do flying higher up. That gives me sometimes ~200FPS.

EDIT: When i did another run with different settings i noticed that the first run has the same time stamp as the last one so its hard to know wich is wich. I know from the numbers #1 and #2 of course but with a bunch it will get harder. So every bench has the same time stamp wich is from the last run. Could be nice if that is fixed. I also noticed that the second run where i upped the settings a bit took the highest FPS value in the windows down to 80 - even when i select the first run only wich was above 80FPS in the peaks.

Will be on MSN later if you want better explanations. Thanks again for creating this. Much appreciated tool.

Edited May 17, 2010 by Alex72

[sbsmac 0]

Alex, you can use the View->Zoom->Custom to set the maximum fps on the graphs. It never occurred to me that people were likely to get over 100fps ! ;-)

The timestamps are generated from the 'session', ie the time at which you started ArmA2. Unfortunately there is no way from within a mission to directly get the system time, hence the #1,#2 etc (Actually, thinking about it I might be able to do something to calculate the time of the benchnmark using diag_ticktime as an offset from the session time.) Anyway... if you go to the benchmarks->details page, you will see a text-edit box called 'label' and you can use this to assign more meaningful labels to particular benchmark runs.

>I also noticed that the second run where i upped the settings a bit took the highest FPS value in the windows down to 80 - even when i select the first run only wich was above 80FPS in the peaks.

That seems a bit strange - I will take a look to see if there is a bug there.

[thebarricade 7]

Work perfectly now!

Thank you for a great program,

Anders

[langgis08 10]

[sbs]mac, great work

thx a lot for fixing regions-floating-point-problem in today's update,

it runs fine, now also for the ","-countries ;)

[langgis08 10]

the tool ran several times very fine, but now I get this one with startup (OS = xp 32 sp3):

System.NullReferenceException: Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.

bei WindowsFormsApplication1.Form1.Form1_Shown(Object sender, EventArgs e)

bei System.Windows.Forms.Form.OnShown(EventArgs e)

bei System.Windows.Forms.Form.CallShownEvent()

bei System.Windows.Forms.Control.InvokeMarshaledCallbackDo(ThreadMethodEntry tme)

bei System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)

bei System.Threading.ExecutionContext.runTryCode(Object userData)

bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)

bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

bei System.Windows.Forms.Control.InvokeMarshaledCallback(ThreadMethodEntry tme)

bei System.Windows.Forms.Control.InvokeMarshaledCallbacks()

************** Geladene Assemblys **************

mscorlib

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3603 (GDR.050727-3600).

CodeBase: file:///c:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll.

----------------------------------------

ArmA2FPSAnalyser

Assembly-Version: 1.0.0.0.

Win32-Version: 1.0.0.0.

CodeBase: file:///C:/Dokumente%20und%20Einstellungen/Tom/Lokale%20Einstellungen/Apps/2.0/BJQG7KNC.7MK/0P2HDRO5.M5L/arma..tion_3f34483d80f35235_0001.0000_360d166b604c7a09/ArmA2FPSAnalyser.exe.

----------------------------------------

System.Windows.Forms

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3053 (netfxsp.050727-3000).

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll.

----------------------------------------

System

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3053 (netfxsp.050727-3000).

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll.

----------------------------------------

System.Drawing

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3053 (netfxsp.050727-3000).

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll.

----------------------------------------

System.Windows.Forms.DataVisualization

Assembly-Version: 3.5.0.0.

Win32-Version: 3.5.30729.116.

CodeBase: file:///C:/Dokumente%20und%20Einstellungen/Tom/Lokale%20Einstellungen/Apps/2.0/BJQG7KNC.7MK/0P2HDRO5.M5L/arma..tion_3f34483d80f35235_0001.0000_360d166b604c7a09/System.Windows.Forms.DataVisualization.DLL.

----------------------------------------

Accessibility

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3053 (netfxsp.050727-3000).

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/Accessibility/2.0.0.0__b03f5f7f11d50a3a/Accessibility.dll.

----------------------------------------

System.Core

Assembly-Version: 3.5.0.0.

Win32-Version: 3.5.30729.1 built by: SP.

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Core/3.5.0.0__b77a5c561934e089/System.Core.dll.

----------------------------------------

mscorlib.resources

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3603 (GDR.050727-3600).

CodeBase: file:///c:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll.

----------------------------------------

System.Windows.Forms.resources

Assembly-Version: 2.0.0.0.

Win32-Version: 2.0.50727.3053 (netfxsp.050727-3000).

CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms.resources/2.0.0.0_de_b77a5c561934e089/System.Windows.Forms.resources.dll.

----------------------------------------

************** JIT-Debuggen **************

Um das JIT-Debuggen (Just-In-Time) zu aktivieren, muss in der

Konfigurationsdatei der Anwendung oder des Computers

(machine.config) der jitDebugging-Wert im Abschnitt system.windows.forms festgelegt werden.

Die Anwendung muss mit aktiviertem Debuggen kompiliert werden.

Zum Beispiel:

<configuration>

<system.windows.forms jitDebugging="true" />

</configuration>

Wenn das JIT-Debuggen aktiviert ist, werden alle nicht behandelten

Ausnahmen an den JIT-Debugger gesendet, der auf dem

Computer registriert ist, und nicht in diesem Dialogfeld behandelt.

uninstalled, reinstalled, renamed/removed/deleted .rpt, nothing helps atm

Edited May 17, 2010 by langgis08
unhandled exception

[sbsmac 0]

Hmm- you may see a file in the same directory as arma2.rpt called 'FPSlabels.txt'. This file contains the labels you apply to traces in the 'details' dialog. Can you try renaming this file to something else and see if you still get the error? If the error goes away, can you send me the contents of the labels file - it's possible it has become corrupted.

[langgis08 10]

FPSlabels.txt seems to be corrupt, because without or renamed analyser works fine again.

Maybe I did wrong labelling ?

btw. I sent you the corrupted file

[sbsmac 0]

The contents look fine - my best guess atm is that for some reason the tool couldn't open the file for read access. Can you try creating a new fpslabels.txt and pasting the contents of the old one into it ?

[langgis08 10]

did it with notepad, pasted content of 'corrupted' in it and same error :-)

so the content itself must be the problem ...

I just played around in a known good FPSlabels.txt with notepad and filling text randomly with many other different keys - like a monkey playing piano ;-)

nothing, the good .txt remains good, analyser runs like it should ...

the trouble's just with the file I sent you :rolleyes:

to be more precise: the trouble's just with the CONTENT of the file I sent you

could it be that: I just opened the corrupted file in notepad again and pressed right-arrow-key and watched the cursor ... after the last text-line ('inland flight 70313 exthreads3') it stops after letting one line blank. in a good file there's no blank line after the last text-line. you can see this in notepad also by marking all text. very funny, indeed.

like that:

bad:

...

inland flight 70313 exthreads3

<cursor stops here>

good:

...

inland flight 70313 exthreads3

<cursor stops here>

Edited May 17, 2010 by langgis08
symptom

[sbsmac 0]

Good catch - the problem is with the extra blank line at the end of the file. I'm not sure how it got there but delete that and things should be fine. I'll take a look tomorrow at how the problem might be occurring.

[langgis08 10]

thx for your job, sbsmac

in the meantime I found out 2 ways to get the problem solved:

1) deleting/renaming FPSlabels.txt (data will still be visible with "default"-labels made by analyser itself)

2) deleting the blank line in an editor, like you suggested

the problem is not really serious or urgent, because it can be fixed - as I know now - very quick and it's no showstopper at all. it might just be 'annoying' for someone who don't know what it is (like me, when I recognized it).

Unfortunately, I can't reproduce the error and so I cannot say how the blank line smuggled itself in the end of file (the text-line in the "Details" where you can type the label(s) seems to be stable, for example typing "return" doesnt create that blank line)

I'm off for work now, sya :)

[sbsmac 0]

This post shows some pretty startling differences between performance with hyperthreading enabled and disabled on an i920. Quad-core owners should definitely disable hyperthreading or use "-cpuCount=4"