PvPscene Benchmark Suite

= PvPscene Benchmark Suite

The goal is to create a better system to benchmark the ArmA engine to

understand the effects of different game video settings, different hardware

components and system configuration.

The approach is to improve the suite step by step by adding

new benchmark scenarios, improving the benchmark system

and automating the process more and more.

The system is largely based on the work and insights from Mac

and his ArmA2 FPS analyzer - benchmarking tool.

Please show him your gratitude and motivate him to update the tool

to OA and extend the functionality. :)

The suite can be run fully automated and logs the result into the rpt file.

Sample output:

"_Preloading - PvPscene_BM_City (Chernarus)"
"avg fps	avg min	highest	lowest"
"20	14	26	5"

The suite contains 12 missions so far and takes 25 minutes to process.

You can also run a subset with configuration or copy the benchmark missions

to arma2\missions and run them one by one as singleplayer mission.

== Current scenarios

The current scenarios focus on the benchmarking the rendering only.

In other words no AI and no interaction - a simple camera scene moving along

some waypoints twice:

1) First a preloading run with faster speed.

2) Followed by the same run in normal speed.

All given scenarios enforce very high terrain detail (most dense grass/clutter

with highest radius) and 3000 viewdistance (10k for LongVD) to put enough

pressure on GPU and CPU.

The types so far are:

• City
  
• Coastline
  
• OpenFields
  
• Woods
  
• LongViewDistance
  

on Chernarus, Takistan, Zargabad and ProvingGrounds.

== Possible new scenarios

• Camera to zoom in and out to simulate RMB zoom.
  
• Camera that simulates the optic view.
  
• Non combat AI scenarios with different numbers of AI (10, 100, 250, 500, 1000).
  

== Install guide

1) Download the suite.

(file section at: http://dev-heaven.net/projects/clanbase/files)

2) Extract to the arma2 folder, like

c:\arma2\autotest\PvPscene_BM

3) Configure your arma2 path in _configureSettings.bat, like

(right click the _configuresettings file and choose edit)

set PATHTOARMA=c:\arma2

You can also select the set of missions to be run in there.

As a start it is best to run the complete suite.

You can also set custom startup parameters, if you want to:

set CUSTOMPARAMS="-profiles=c:\arma2" -window "-name=test"

4) Start the suite with the run.bat or runBeta.bat in .\arma2\autotest\PvPscene_BM.

5) Wait for the benchmarks to finish and check the rpt for the result.

6) You can run the suite again with different settings and compare the results yourself

or submit your results to the online database (no registration required).

== Notes

If you run arma in window mode, you have a look at the GPU use

while the benchmarks are run with a tool like MSI Afterburner.

You can follow the CPU use, if you have the task manager running with the

performance tab open.

=== Online Database

Please provide your values the following graphic settings along with your test results

to make the comparison between different systems and graphic settings useful.

For some graphic settings, you need to look in the arma2.cfg.

To find out your hardware, you can use the tool PC-Wizard for example.

To make the copy-paste action less effort, remove the duplicate lines in the rpt first.

You can do this with EditPadPro (trial/lite) for example nicely.

Just open the rpt / drag it into the text editor window, select in the menu 'extra',

and in the middle 'remove duplicate lines'.

The abbreviations for the graphic settings values are:

• D - Disabled/Default
  
• VL - Very Low
  
• L - Low
  
• N - Normal
  
• H - High
  
• VH - Very High
  

Edited December 5, 2011 by .kju [PvPscene]

Really cool mate! Congrats on release!

ok :o I realize a lot of effort has gone into this, so I will try to participate.

Edited February 17, 2011 by [DirTyDeeDs]-Ziggy-
:o

Lots of time? :) Download, unpack, configure - once. Can be done in 2 minutes.

The benchmarks are fully automated - you can go watch TV, eat or whatever meanwhile.

If you want a meaningful benchmark, you found one here.

Edited February 18, 2011 by .kju [PvPscene]

:j:

there are 420 fields to fill out? is that a random number or by design? ;)

my benchmark results reflect gfx setting i use in normal gameplay.

Edited February 17, 2011 by [DirTyDeeDs]-Ziggy-

Hey guys might be a stupid question but i can't set my shortcut up, can anyone give me a step by step on how to?

Edited February 17, 2011 by AC337

Start the suite with the run.bat or runBeta.bat.

from within the ArmA 2/autotest/PvPscene_BM folder

also

right click the _configuresettings file and choose edit

looks like this

Edited February 18, 2011 by [DirTyDeeDs]-Ziggy-

-Ziggy-;1858307']from within the ArmA 2/autotest/PvPscene_BM folder

also

right click the _configuresettings file and choose edit

looks like this

Thanks alot man!

Edited February 18, 2011 by Dwarden

Nice work on this benchmark suite!

Since it forces high terrein details, and also VD, maybe make it enforce all

in game video options, and create .bats for a few different resolutions?

Edited February 18, 2011 by Birdseye

= PvPscene Benchmark Suite

The goal is to create a better system to benchmark the ArmA engine to

understand the effects of different game video settings, different hardware

components and system configuration.

in line with the goal of this 'addon', please submit your results to the online database so they can be compared with other results.

Most of the fields can be copy pasted. ;)

Adding your information to the database will also express your appreciation to the author for the hard work that has gone into this effort. :clap:

Is there an explanation of each field for the Database not sure what order my settings coniced with on I-L, O-AB. But I got my Bench's up there from Takistan run.

Also sorry Helo thought someone was typing hello!

LOL I found it, but I have no idea what settings I ran under I think just a stock Arma 2 run up used test as the profile and seeing as I dont have a test profile I would assume A2OA just setup what it felt was best for the game.

Think I will redo them and under a created Benchmark Profile.

And thanks for the sweet tool PvPScene!! :ok:

Edited February 18, 2011 by Rabble-Rouser

Nice Work.

Alrighty got a set of Bench's on there as Normal settings across the board next batch should be of my standard gameplay settings.

Lovin this tool set man very nice job!

Thanks [DirTyDeeDs]-Ziggy-!

Updated the first post with several clarifications.

Birdseye this is possible - however to test different settings in one go is one major

feature of Mac's FPS analyzer. Lets motivate him to update it to make it a lot easier. :)

Rabble-Rouser you can find the explanations in the first post now.

Nifty. Will check it out over the weekend. :)

.

=== Online Database

The following graphic settings you are asked to submit in addition to your test results

to make the comparison between different systems and graphic settings useful.

i understand having at least one group of settings the same across the different benches to make comparisons meaningful, but why Windowed mode and 3D resolution 1280x960?

dont you think full screen and 3d resolution matching native resolution more appropriate?

I never play in windowed mode, and that 3d res is way off matching anything in my drop down.

I'd be happy to fulfill your request, but I hesitate to run a bench when I feel little can be gained from the test.

Please explain to this ignorant soul what the importance of those two particular settings is.

another question. online database. gfx settings. terrain detail field.

Since you are forcing Very High Terrain Detail in the benchmark, should all benchmarks report VH for that setting?

thanks

Edited February 19, 2011 by [DirTyDeeDs]-Ziggy-

Well the wording is indeed confusing.

The screenshot was merely meant to explain the abbreviations like TD = texture detail.

It is NOT about recommended settings. Rewording it now.

The goal is to let people submit their current settings or those they prefer.

A direct comparison is something mid term and will be easier to achieve with Mac's tool.

PS: TD = texture detail. :) Terrain detail is not part of the screenshot.

PSS: Tomorrow a new layout with all result in one sheet will be available.

It will contain all results already submitted.

Edited February 19, 2011 by .kju [PvPscene]

I dig the fact that you can compare what others run there settings at to help give a comparison on settings I have used or may want to use and see if it offers benefit or grief. Having one test to run everones settings the same would be handy just to set a standard for machine comparison. But every machine runs different wether you built it yourself and know it in and out or wether you bought it off a shelf and wonder why, with all the bloat ware, it is so slow.

I've been doing some scripting yesterday so I dint get around to doing a second test at what I actually run the game at for gaming. But adding it as a second comparison may shed light to others with a similar system as to what may be a better setting for them.

The new version is up. Hopefully it is somewhat easier to use and provides a better overview.

Edited February 20, 2011 by .kju [PvPscene]

Where do I find my results?

---------- Post added at 11:05 AM ---------- Previous post was at 10:30 AM ----------

NVM. I figured it out. But still dont know how to post them.

Nice update in the database. It's easier now to post results, so I addded a few benchies.

Due to PvPscene being banned and no longer able to communicate on BIF, I have been asked to place a link here to a thread continuing this discussion elsewhere.

http://213.245.0.237/pvpscene/forums/index.php/topic,269.0.html

Edited February 21, 2011 by [DirTyDeeDs]-Ziggy-

He isn't banned, he's post restricted until tomorrow.

I've added three more runs of mine with different MFtRA (Max Frames to Render Ahead).

It basically says that for my system in terms of FPS there is no difference.

Right now I am using 1 as it makes mouse lag a non issue.

http://www.editgrid.com/user/kju/PvPscene_Benchmark_Suite

Added my results with the latest version (1.59.79122).

It is actually considerably less FPS (10%) - even though I had PP disabled.

PvPscene_Benchmark_Suite

Edit: Yet it might be due to higher LODs shown overall and less LOD switching. FPS alone does not tell the whole picture.

Edited March 22, 2011 by .kju [PvPscene]