ArmA2 FPS analyser - benchmarking tool

[Batstat 10]

Great app sbsmac. Tips, the correct rpt file would not be found if the user start arma with -profiles=drive:\dir\dir... in his shourcut.

Serveral ways to fix this,

a) put the info in the readme (ask user not to use -profiles), or tell him to manually copy the rpt file after the benchmark is run to ...AppData\local\arma 2\

b) Give the user the option where he can drag his shortcut to a dockable surface in your application. Then read the -profiles variable and maybe more info about how the game was started. Not sure if this is doable if the user often switch between different shortcuts (how to correlate, the shortcuts with the correct results - dunno)

c) Give the user a way to tell where the rpt file should be read from.

regards

[sbsmac 0]

Thanks Batstat. Actually I have a few improvements in mind which will solve this problem as well. Watch this space... :-)

[Batstat 10]

Mission creep or new request?

Yesterday I encountered that my RPT file probably are under a round robin regime. All my Arma2fps numbers was gone.

Could you save the fps data in another file aka fslabels? Then I can compare my new settings with old one, without doing the measurement twice?

Regards

Batstat

[sbsmac 0]

Just working on it now ;-)

[sbsmac 0]

There is a new version now available. This adds the ability to batch-run large numbers of benchmarks with different parameters.

Database->Run Benchmarks...

Select the parameters you are interested in varying (for example, you might want to see how view-distance affects performance or whether the latest beta peforms better than an earlier release) the choose 'Run' to spawn arma2 with all the combinations of selected parameters. This can take some time so you might want to do this overnight ! (Make sure you shutdown any background programs and ensure your virus scanner will not start its scheduled scan half way through the run !)

After you have run benchmarks, you can 'explore' the database of results using the 'Database->explorer' menu option. By default, all results in the database are shown but you can use the sliders to scroll through different values to get a feel for how they affect performance.

For example, you can see here the affect of changing cpuCount from 4 to 8 (conclusion: Hyperthreading is very bad for performance)

Here is the affect of chaning the -exThreads option:

You can quickly take screenshots using the Screenshot menu item or create an animated gif (like those above) from a sequence of screenshots. Screenshots and gifs are stored in your 'My Pictures' folder.

---------- Post added at 08:34 PM ---------- Previous post was at 08:30 PM ----------

Note, the new database is separate from any previous benchmarks you may have run and I'll soon remove the previous ability to run benchmarks outside of the tool.

Edited June 11, 2010 by sbsmac

[Dwarden 1124]

sbsmac ... very nice ...

yet , could you please add some more options? :)

for ARMA2.cfg:

HDRprecision=8;

(default 8, applicable are 16 and 32, usually higher value improves slightly the visual (HDR) while lowers performance)

GPU_MaxFramesAhead=1000;

(default is 1000; driver default is usually 3, driver or manual change in ARMA2.cfg there to 2,1,0 may improve performance/smoothness)

[sbsmac 0]

HDRprecision=8;

(default 8, applicable are 16 and 32)

Will do. At the moment, the value from Arma2/arma2.cfg is used.

GPU_MaxFramesAhead=1000;

(default is 1000; driver default is usually 3, driver or manual change in ARMA2.cfg there to 2,1,0 may improve performance/smoothness)

What is the interaction with "GPU_DetectedFramesAhead" ? Does Arma use MIN(GPU_MaxFramesAhead,GPU_DetectedFramesAhead) ?

And one quick question which would help a bit with skipping combinations that have already been tested in previous sessions.... Is there a quick way to extract the build-id from an arma2.exe binary without having to launch it and grep the value from the rpt file ?

---------- Post added at 09:31 PM ---------- Previous post was at 09:23 PM ----------

*Edit* Grepping for BuildMapID and VersionMapID seem to be the way to go. :)

Edited June 11, 2010 by sbsmac

[Dwarden 1124]

GPUdetectedFrames is value detected from drivers, it most of time is correct but it might be wrong ...

it's write only (engine writes it to arma2.cfg as detected)

also GPU_MaxFramesAhead range is 0 to 1000, yet 0 can cause unforseen problems like crashes, so better put some warning on that value

[sbsmac 0]

also GPU_MaxFramesAhead range is 0 to 1000, yet 0 can cause unforseen problems like crashes

It sounds the useful set of values is really just [1,2,3,1000] with 1000 simply acting as a way to say 'as many as the GPU will support'?

The reason I'm asking is that I'd prefer to use a checkedListBox as with the other options so don't particularly want to offer the freedom for the user to enter any value up to 1000 unless it truly makes a difference to performance.

When I previously had problems with mouse-lag I modified the render-ahead setting in the driver without touching GPU_MaxFramesAhead and it solved the problem implying that the driver setting takes priority over the cfg-setting. I'm therefore a bit suspicious that there is a variable here which I can't actually control or read and this might invalidate some of the results unless people take care.

[Dwarden 1124]

offer 0,1,2,3,4,5,1000 and i think that's enough for everyone :)

the cfg there takes over the driver settings in terms if you define in drivers 2 and here 1 the 1 will be used ...

ofcourse this may vary depending on what drivers allow etc.

p.s. would be also worthwhile report somewhere for the graph what's detectedGPUmaxframesahead ;)

just as info

Edited June 11, 2010 by Dwarden

[sbsmac 0]

Excellent - will work on this over the weekend.

[amateur 0]

"Run Benchmarks" seems not working for me. When I click "Run" - nothing happens. Could it be because of non-default location of my arma2.exe?

[sbsmac 0]

Actually it's a bug which I'm working on now. Should have a fix shortly.

---------- Post added at 11:46 AM ---------- Previous post was at 11:38 AM ----------

*Edit* Should now be fixed. Restart the application to update.

[amateur 0]

[/color]*Edit* Should now be fixed. Restart the application to update.

Nothing changed :( Pressing "Run" causes nothing, exept of progress bar...

[sbsmac 0]

Ooops - that was a bit embarrassing, I forgot to remove some debug code which would have prevented anything being run. Anyway, new version uploaded.

Changes:-

* HDR and MaxFramesAhead supported as requested by Dwarden

* "Run Benchmarks" now avoid re-running benchmarks that are already in the database.

* You can hover the mouse over a line on the graph to get a tooltip showing the settings for that run (settings common to all runs on a graph are not shown).

Edited June 12, 2010 by sbsmac

[MJK-Ranger 0]

Hi and it required Microsoft .NET Framework 4

Download here: http://www.microsoft.com/downloads/details.aspx?FamilyID=9cfb2d51-5ff4-4491-b0e5-b386f32c0992&displaylang=en

Cheers

[sbsmac 0]

Affect of changing HDR... The main hit appears to be in going from 16 to 32 rather than 8 to 16..

GPU_MaxFramesAhead - not much difference between settings although GPU_MaxFramesAhead=1 seems very slightly better (though might just be normal variation in measurement). If anything I would have thought larger values of MaxFramesAhead should give better framerates ?

Edited June 12, 2010 by sbsmac

[MJK-Ranger 0]

hmm i can't run the "run benchmark" got some error:

See the end of this message for details on invoking

just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************

System.IO.FileNotFoundException: Could not find file 'C:\.ArmA2Profile'.

File name: 'C:\.ArmA2Profile'

at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath)

at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)

at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)

at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize)

at System.IO.StreamReader..ctor(String path, Boolean detectEncodingFromByteOrderMarks)

at Mac.ArmA.Profile.Read(String path)

at Mac.ArmA.ArmA2..ctor()

at ArmA2FPSAnalyser.Form5..ctor(String home)

at ArmA2FPSAnalyser.Form1.runBenchmarksToolStripMenuItem_Click(Object sender, EventArgs e)

at System.Windows.Forms.ToolStripMenuItem.OnClick(EventArgs e)

at System.Windows.Forms.ToolStripItem.HandleClick(EventArgs e)

at System.Windows.Forms.ToolStripItem.HandleMouseUp(MouseEventArgs e)

at System.Windows.Forms.ToolStrip.OnMouseUp(MouseEventArgs mea)

at System.Windows.Forms.ToolStripDropDown.OnMouseUp(MouseEventArgs mea)

at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)

at System.Windows.Forms.Control.WndProc(Message& m)

at System.Windows.Forms.ToolStrip.WndProc(Message& m)

at System.Windows.Forms.ToolStripDropDown.WndProc(Message& m)

at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

************** Loaded Assemblies **************

mscorlib

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 (RTMRel.030319-0100)

CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll

----------------------------------------

System

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll

----------------------------------------

ArmA2FPSAnalyser

Assembly Version: 1.0.0.0

Win32 Version: 1.0.0.0

CodeBase: file:///C:/Users/bjoern/AppData/Local/Apps/2.0/AD5N117L.V77/Q0A5XHQ8.GRD/arma..tion_3f34483d80f35235_0001.0000_31449b47eaffb989/ArmA2FPSAnalyser.exe

----------------------------------------

System.Windows.Forms

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll

----------------------------------------

System.Drawing

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll

----------------------------------------

Accessibility

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Accessibility/v4.0_4.0.0.0__b03f5f7f11d50a3a/Accessibility.dll

----------------------------------------

System.Windows.Forms.DataVisualization

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms.DataVisualization/v4.0_4.0.0.0__31bf3856ad364e35/System.Windows.Forms.DataVisualization.dll

----------------------------------------

System.Core

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll

----------------------------------------

System.Data

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 (RTMRel.030319-0100)

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_64/System.Data/v4.0_4.0.0.0__b77a5c561934e089/System.Data.dll

----------------------------------------

System.Xml

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll

----------------------------------------

mac.Arma

Assembly Version: 1.0.0.0

Win32 Version: 1.0.0.0

CodeBase: file:///C:/Users/bjoern/AppData/Local/Apps/2.0/AD5N117L.V77/Q0A5XHQ8.GRD/arma..tion_3f34483d80f35235_0001.0000_31449b47eaffb989/mac.Arma.DLL

----------------------------------------

mac.XML

Assembly Version: 1.0.0.0

Win32 Version: 1.0.0.0

CodeBase: file:///C:/Users/bjoern/AppData/Local/Apps/2.0/AD5N117L.V77/Q0A5XHQ8.GRD/arma..tion_3f34483d80f35235_0001.0000_31449b47eaffb989/mac.XML.DLL

----------------------------------------

System.Web

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 built by: RTMRel

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_64/System.Web/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Web.dll

----------------------------------------

System.Configuration

Assembly Version: 4.0.0.0

Win32 Version: 4.0.30319.1 (RTMRel.030319-0100)

CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll

----------------------------------------

Gif.Components

Assembly Version: 0.0.0.1

Win32 Version: 0.0.0.1

CodeBase: file:///C:/Users/bjoern/AppData/Local/Apps/2.0/AD5N117L.V77/Q0A5XHQ8.GRD/arma..tion_3f34483d80f35235_0001.0000_31449b47eaffb989/Gif.Components.DLL

----------------------------------------

************** JIT Debugging **************

To enable just-in-time (JIT) debugging, the .config file for this

application or computer (machine.config) must have the

jitDebugging value set in the system.windows.forms section.

The application must also be compiled with debugging

enabled.

For example:

<configuration>

<system.windows.forms jitDebugging="true" />

</configuration>

When JIT debugging is enabled, any unhandled exception

will be sent to the JIT debugger registered on the computer

rather than be handled by this dialog box.

PS! I'm running Windows 7 Professional x64 and .Net Framework 4.0.30319.1

[sbsmac 0]

it's having difficulty finding your current ArmA2 profile file. A couple of questions..

1) What OS are you using ?

2) Does your user-name have lots of strange non-alphabetic characters in it ? If so, you could try creating a user-profile with a very simple name like "test" and see if that helps.

[MJK-Ranger 0]

Hi.

1: I'm running Windows 7 Professional x64 and .Net Framework 4.0.30319.1

2: My ingame username is +Bjørn+

I'm gonna install your app again and make a new simple username and test it then.

Thanks :)

---------- Post added at 18:47 ---------- Previous post was at 18:40 ----------

Hi again. :)

Yes the problem was my username in arma2. I made a new one called fpstest and no problem now :D

Thanks :)

[sbsmac 0]

Thanks - that's useful information. The problem occurs because ArmA stores your user name in the registry as a UTF-8 string but creates a directory loosely based on the URL encoding of the name. So if you look in your "ArmA2 Other Profiles" directory you will sometimes see folders that look like '!%22%c2%a3$%20%20player%20@{}%60' Unfortunately the encoding is not exactly the same as URL encoding and I haven't yet figured out quite what the algorithm is which is why sometimes the program can't find a match between the username in the registry and the folder in the profiles directory.

The workaround is to create a very basic default profile but I'd be interested to hear form other users whose profile names aren't correctly identified.

*Edit* Updated the program to report this error more clearly.

Edited June 12, 2010 by sbsmac

[amateur 0]

"Run Benchmarks" works, but now when I try "Explore..." - app crashes with http://paste.org.ru/?zmbfar (sorr, log is partially in russian).

[sbsmac 0]

Possibly an error in one of the rpt files. What would be useful is if you could zip up the contents of your Documents/Arma2FPSAnalyser folder and post a link to the archive. Alternatively, there may be an issue with the globalisation - could you let me know what language you are using on your machine (I'm assuming russian!).

Finally, any chance you could translate the russian for the first 4 or 5 lines of the log you posted ?

Thanks.

[amateur 0]

Was some bug with rpts, I've cleaned "results" folder and started benchmarks again - now it works.

[sbsmac 0]

Any chance you still have the 'broken' rpt available ? I'd like to improve the error-handling if possible.