[MP CO25] SPB Server Performance Benchmark

[whiztler 135]

CHANGELOG

Current version

VERSION 1.10

Added: Headless Client (autodetect)

Updated: Slots, 49 Client slots + 1 HC slot

Previous versions

VERSION 1.00

Updated: Slots, now 50 slots (as requested)

Updated: RPT reporting, more detailed

Updated: Script optimizations

Added: Mission init RPT logging (version, mission params, connected clients, etc)

VERSION 0.60 (internal release)

Updated: Script optimizations

VERSION 0.40 (internal release)

Fixed: Combat reporting / logging

VERSION 0.15

Added: More frequent combat RPT logging (every 5 seconds for first 60 seconds).

Updated: Autopilot reporting functions. Less network traffic

Fixed: double logging in RPT of spawn/combat reports

VERSION 0.10

Added: Autopilot - automated test scripts

Added: Combat reporting (autopilot only)

Added: Baseline FPS

Added: Mission Parameter 'Day or Night'

Added: Mission Parameter 'Expert Mode'

Added: Mission Parameter 'Autopilot Config'

Updated: Slots > now 25 slots (1 tester, 24 clients)

Updated: Reporting functions

Updated: simplestats (manual check) > serverstats (specific for SPB)

VERSION 0.05

Beta release nopryl.no

DOWNLOAD

Mega.co.nz: http://goo.gl/PvZCkf [version: 1.10]

Armaholic: http://www.armaholic.com/page.php?id=25649 [version: 1.10]

ABOUT SPB

The Server Performance Benchmark (SPB) mission was put together to benchmark dedicated server performance (Server FPS).

SPB offers a fully an autopilot function, fully automated test scripts that can be configured in the mission parameters (at mission start/slot selection).

If the tester does not wish to use the autopilot function, one can test manually. SPB can spawn up to 500 AI Bluefor units and up to 10 Opfor squads to simulate combat situations. After each Bluefor group spawn a report is generated:

* Displayed on screen (hint)

* Copied to the clipboard (comma delimited)

* Inserted into the Server RPT file

Screenshots

http://i62.tinypic.com/245iowh.jpg (113 kB)

http://i59.tinypic.com/5lwe3m.jpg (108 kB)

Sample RPT log

20:57:30 [==============] SPB Mission Start [=============]
20:57:30 
20:57:30 c:\w\c_branch\poseidon\futura_stable\lib\network\networkserver.cpp NetworkServer::OnClientStateChanged:NOT IMPLEMENTED - briefing!
20:57:30 "JayArmA2Lib: ACTIVE"
20:57:30 "0 - MCC Headless Client available: false"
20:57:30 "0 - MCC Local Headless Client: false"
20:57:30 String STR_A3_dlc_loading_screen_text not found
20:57:30 [22899,117.193,0,"XEH: PostInit Started"]
20:57:30 [22899,117.211,0,"CBA_VERSIONING: cba=1.0.5.208, "]
20:57:30 [22899,117.357,0,"XEH: PostInit Finished. State: _isClient=false, _isJip=false, _isDedClient=false, _isServer=true, _isDedServer=true, _playerCheckDone=true, _sp=false, _startInitDone=true, _postInitDone=true, _mpRespawn=true, _machineType=0, _sessionId=1, _level=0, _timeOut=false, _game=3, BIS_functions=L Modules:1, group=L Modules"]
20:57:30 "Added 'mcc_setup' EventHandler for Server"
20:57:30 [22899,117.205,0,"x\cba\addons\network\XEH_postServerInit.sqf:3","Initializing: cba_network version: 1.0.5.208"]
20:58:17 
20:58:17 
20:58:17 [================] SPB Baseline [================]
20:58:17 
20:58:17 "Server Avg FPS: 50"
20:58:17 "Server Min FPS: 48"
20:58:17 "Server Frame: 25134"
20:58:17 
20:58:17 
21:54:13 
21:54:13 
21:54:13 [=======] SPB Report | Start Autopilot [========]
21:54:13 
21:55:13 
21:55:13 
21:55:13 "[==========] SPB Report | first Group [==========]"
21:55:13 
21:55:13 "Total BlueFor units spawned/alive: 50 (1 groups)"
21:55:13 "Total OpFor units spawned/alive: 0 (0 groups)"
21:55:13 "Server Avg FPS: 50"
21:55:13 "Server Min FPS: 45"
21:55:13 "Server Frame: 195894"
21:55:13 "Total objects: 69 (67 entities)"
21:55:13 
21:55:13 
21:57:43 
21:57:43 
21:57:43 "[==========] SPB Report | second Group [==========]"
21:57:43 
21:57:43 "Total BlueFor units spawned/alive: 100 (2 groups)"
21:57:43 "Total OpFor units spawned/alive: 0 (0 groups)"
21:57:43 "Server Avg FPS: 50"
21:57:43 "Server Min FPS: 43"
21:57:43 "Server Frame: 203384"
21:57:43 "Total objects: 119 (117 entities)"
21:57:43 
21:57:43 
22:00:14 
22:00:14 
22:00:14 "[==========] SPB Report | third Group [==========]"
22:00:14 
22:00:14 "Total BlueFor units spawned/alive: 150 (3 groups)"
22:00:14 "Total OpFor units spawned/alive: 0 (0 groups)"
22:00:14 "Server Avg FPS: 50"
22:00:14 "Server Min FPS: 45"
22:00:14 "Server Frame: 210867"
22:00:14 "Total objects: 169 (167 entities)"
22:00:14 
22:00:14 
....
22:01:53 [=======] SPB Report | Start Combat Report [========]
22:01:53 
22:01:53 "Elapsed mission time: 3830 seconds"
22:01:53 "Total BlueFor units spawned/alive: 150 (3 groups)"
22:01:53 "Total OpFor units spawned/alive: 80 (10 groups)"
22:01:53 "Server Avg FPS: 32"
22:01:53 "Server Min FPS: 18"
22:01:53 
22:01:53 ----------------------------------------------------
22:01:53 
22:01:54 "Game duration: 3831 seconds"
22:01:54 "Total BlueFor units spawned/alive: 150 (3 groups)"
22:01:54 "Total OpFor units spawned/alive: 80 (10 groups)"
22:01:54 "Server Avg FPS: 20"
22:01:54 "Server Min FPS: 7"
22:01:54 
22:01:54 ----------------------------------------------------
22:01:54 
22:01:57 "Game duration: 3834 seconds"
22:01:57 "Total BlueFor units spawned/alive: 148 (3 groups)"
22:01:57 "Total OpFor units spawned/alive: 76 (10 groups)"
22:01:57 "Server Avg FPS: 34"
22:01:57 "Server Min FPS: 15"
22:01:57 

Author

Concept & development by whiztler (www.nopryl.no)

VANILLA & CBA/A.C.R.E.

There are two versions of SPB available:

ARMA III Vanilla

The SPB Vanilla version requires no addon's and runs with ARMA III Vanilla. Make sure that the server admin configures the server as Vanilla and instruct the clients to join the SPB mission with a Vanilla profile (no addon's).

Filename: SPB_Vanilla_v100.Altis.pbo

CBA / A.C.R.E.

The SPB CBA/A.C.R.E. version is dependent on CBA A3/A.C.R.E. (and Jay's ArmaLib A3). Other than the required addon's, the SPB CBA/A.C.R.E. is the same as SPB Vanilla.

Filename: SPB_CBA-ACRE_v100.Altis.pbo

SPB MISSION PARAMETERS

You can configure several mission parameters:

* Day or Night. When selecting 'Day', time is set to 12:00 hrs (noon). When selecting night, time is set to 00:00 hrs (midnight).

* Expert Mode. Disables the instructions screens at the beginning of the mission. You'll jump right into the benchmarking feature.

* SPB Autopilot config (see 'Automated Benchmarking using the Autopilot test scripts' for more information)

BENCHMARKING

PREPARATIONS

For reliable testing please use the same 'mod-line' each time you run SPB (both server and clients). SPB was designed to test using up to 50 connected clients of which one is the 'tester' client. Only the tester client can access the SPB action menu and spawn up to 10 AI Bluefor groups of 50 units each group.

Read the 'Server bandwidth & Optimisation' tutorial by Terox for information about getting the most out of your ARMA III server.

Server

It is recommended that you start with the default ARMA III server profile to create a baseline. Start testing with the SPB Vanilla version and benchmark various server profiles until you are satisfied with the performance.

Once you have a satisfactory SPB Vanilla benchmark you can proceed with the SPB CBA/A.C.R.E version or add mods to your server modline to test performance impact of certain addon's.

Make sure that before you start, you have several server profiles ready that you want to test. With 'Tophe's Arma Dedicated Server Tool' (TADST - http://www.armaholic.com/page.php?id=11655) you can create and save various profiles that you want to test at a later stage. Simply load the profile, start the server and Bob is your uncle.

For real-time server performance monitoring I suggest you install the 'Arma Server Monitor' (ASM: http://www.armaholic.com/page.php?id=21436 -or- https://github.com/fred41). ASM is an external application that runs on your server. It can be used alongside SPB for additional reporting.

Benchmarking can be a lengthy process. But when done right, you'll end up with a server profile that is able to run a smooth ARMA III server. People that play on your server(s) will thank you for it!

Client

Participating clients (incl. the tester) should run a similar modline as the server. Please make sure that clients do not have enabled any client-side addon's such as ST-Hud, Blastcore, SpeedofSound, JSRS2, etc., etc.

The tester should brief people that will join as a client re which profile they should use before joining the server. For SPB Vanilla, clients can start ARMA III without any mods activated.

When testing

Benchmark reports are generated after each Bluefor group spawn (manual and Autopilot) and after spawning Opfor squads (Autopilot only).

AUTOMATED BENCHMARKING USING THE AUTOPILOT TEST SCRIPTS

SPB offers fully automated benchmarking using the SPB Autopilot feature. The SPB Autopilot needs to be configured at mission start-up. Using mission parameters you can setup SPB Autopilot to spawn 50, 100, 150 ... 500 units. The script will take care of the reporting in between the group spawns and during spawning of Opfor squads. The SPB Autopilot cycle is concluded with spawning 10 Opfor squads.

You can initialize the SPB Autopilot, after the SPB Baseline has initialized, using the SPB Action Menu.

Warning

Do not configure the SPB Autopilot to spawn 10 groups of Bluefor AI's if you do not know if your server can handle it. Start with 3 groups and increase from there on. I cannot be held accountable for any damage done to your server!

MANUAL BENCHMARKING

If you prefer to have more control over the benchmarking process, you can manually spawn Bluefor groups using the SPB Action Menu. You can decide if you wish to spawn 1, 5 or 10 Opfor squads after or during the Bluefor group spawning. A Bluefor group spawn report is created 60 seconds after each groups has spawned. If you wish to monitor server performance during combat you can start the 'server stats' feature with Radio Juliet [0-0-0] and disable it again with Radio Alpha [0-0-1]. You can display the last test result with Radio Bravo [0-0-2].

Recommended manual benchmarking method:

1. Spawn the first 3 Bluefor groups. Wait for the report to appear before spawning the next group (60 secs).

2. Spawn 5 Opfor Squads. Wait until all Opfor units have been killed.

3. Check Server Stats with [0-0-0] during combat and disable with [0-0-1]

4. Spawn the fourth and the fifth Bluefor groups.

5. Spawn 5 Opfor Squads.

6. Check Server Stats with [0-0-0] during combat and disable with [0-0-1]

7. Spawn the sixth and seventh Bluefor groups.

8. Spawn 5 Opfor Squads.

9. Check Server Stats with [0-0-0] during combat and disable with [0-0-1]

10. Spawn the remaining three0 Bluefor groups.

11. Spawn 10 Opfor Squads.

12. Check Server Stats with [0-0-0] during combat and disable with [0-0-1]

If you decide only to spawn a few Bluefor groups, then conclude your cycle with step 11 and 12.

TESTER & CLIENTS

The tester slot controls the testing process. All clients (including Tester) cannot be harmed or killed. For optimal testing results it is important that clients remain still and move as little as possible.

Edited June 23, 2014 by whiztler
Updated to version 1.10

[kremator 1064]

Excellent ... will set this running on my dedi to see what happens. Thanks for doing this. More information on the causes for server FPS drop are very welcome.

[yxman 90]

Tested it alone with def. settings. Mods: CBA_A3 Beta5

"[==========] SPB Report | third Group [==========]"

"Total BlueFor units spawned/alive: 150 (3 groups)"

"Total OpFor units spawned/alive: 0 (0 groups)"

"Server Avg FPS: 48"

"Server Min FPS: 40"

"Server Frame: 39015"

"Total objects: 157 (156 entities)"

[=========] SPB Report | Combat Report [=========]

"Total BlueFor units spawned/alive: 114 (3 groups)"

"Total OpFor units spawned/alive: 0 (10 groups)"

"Server Avg FPS: 48"

"Server Min FPS: 40"

[whiztler 135]

SPB updated to 0.15

Changelog:

VERSION 0.15

Added: More frequent combat RPT logging (every 5 seconds for first 60 seconds).

Updated: Autopilot reporting functions. Less network traffic

Fixed: double logging in RPT of spawn/combat reports

[whiztler 135]

SPB updated to 1.00

See OP for change log and download links.

@Moderator: please change the topic title into SPB Server Performance Benchmark

Edited May 18, 2014 by whiztler

[ZertyKchan 14]

Thank you for this mission, I was looking for something like this. Great job.

Would it be possible to add the stuff for a headless client to test out performance gain?

[whiztler 135]

Thank you for this mission, I was looking for something like this. Great job.

Would it be possible to add the stuff for a headless client to test out performance gain?

Thanks buddy. HC support is in the pipeline.

[whiztler 135]

SPB updated to 1.10

See OP for change log and download links.