Jump to content
emrak

Methods of measuring performance?

Recommended Posts

I'm aware of BIS_fnc_codePerformance, but that only measures relative performance for a specific script. How would I measure relative performance (overall) for an entire mission?

 

Here is an example. Let's pretend that I open the Editor and create two separate multi-player scenarios. They are identical, but the first has 40 quad bikes in it and the second has 40 offroads in it. Is there any way of telling which mission generates more overhead and uses more resources?

Share this post


Link to post
Share on other sites

I'm aware of BIS_fnc_codePerformance, but that only measures relative performance for a specific script. How would I measure relative performance (overall) for an entire mission?

 

Here is an example. Let's pretend that I open the Editor and create two separate multi-player scenarios. They are identical, but the first has 40 quad bikes in it and the second has 40 offroads in it. Is there any way of telling which mission generates more overhead and uses more resources?

 

1. Listen to complaints from the players. Do they whine about FPS/lag/stuttering? Do the performance complaints occur all the time or only after 5-6 hours of server uptime? Take note of when you hear the complaints. Put some diag_log use into your threads to keep track of what is happening and when, to help figure out what their issues are. I used this method to determine what things to prioritize.

 

If you don't have players, stick diag_fps command into a loop and monitor it. Generally in the editor and single player it will stay at 49-51 unless you spam a bunch of AI or have some nasty script errors.

 

2. Is it significant? The difference between 40 quads and 40 offroads is negligible (is even measurable?). Don't worry about micro-optimization, you don't have the time in your human lifespan to worry about measuring 0.00001ths of a second for something as trivial as a game.

Share this post


Link to post
Share on other sites

Use performance counters externally or even integrate them with an extension

Share this post


Link to post
Share on other sites

1. Listen to complaints from the players. Do they whine about FPS/lag/stuttering? Do the performance complaints occur all the time or only after 5-6 hours of server uptime? Take note of when you hear the complaints. Put some diag_log use into your threads to keep track of what is happening and when, to help figure out what their issues are. I used this method to determine what things to prioritize.

 

If you don't have players, stick diag_fps command into a loop and monitor it. Generally in the editor and single player it will stay at 49-51 unless you spam a bunch of AI or have some nasty script errors.

 

2. Is it significant? The difference between 40 quads and 40 offroads is negligible (is even measurable?). Don't worry about micro-optimization, you don't have the time in your human lifespan to worry about measuring 0.00001ths of a second for something as trivial as a game.

 

I'm really not concerned about micro-optimization, I just come from a standard programmatic background wherein testing occurs early and often. In a normal programmatic job, it's so easy to test and generate metrics for a given application. I feel like I'm blind without knowing how long it will take a mission to load and/or what it's overhead will be. I hope I'm making sense. :) Simply posting a mission without having any performance metrics and waiting for feedback from players seems like a 1990s/turn-of-the-century development approach (I know, I was coding then lol).

 

EDIT: And I will definitely check out diag_fps. Thanks for bringing that up!

Share this post


Link to post
Share on other sites

Use performance counters externally or even integrate them with an extension

 

This sounds like an avenue to pursue! Do you have any brief examples you could describe? Thanks

Share this post


Link to post
Share on other sites

This sounds like an avenue to pursue! Do you have any brief examples you could describe? Thanks

I will give you a C# project that I used to play around with performance counters and wmi. It is not a best practise, but you might be able to learn something from it.

The output is as follows:

Time Elapsed 00:00:56.9663781

Time(ms) CPU(%) RAM(MB) Hard Drive(%)

261 0 0 0

621 1.08 0 3

980 5.44 3 0

1343 5.4 9 6

1702 14.11 10 8

2062 18.44 13 31

2424 4.34 14 13

2784 9.73 15 3

3143 24.99 25 0

3503 22.78 47 12

3864 8.67 49 15

4225 1.08 50 16

4584 9.76 52 7

4942 28.28 53 0

5302 35.96 54 11

5664 10.85 57 14

6023 11.9 62 17

6384 11.9 65 5

6744 23.88 78 9

7104 10.86 86 18

7463 4.34 87 23

7823 2.18 87 16

8183 3.25 89 8

8542 11.98 95 1

8902 20.6 105 0

9263 24.95 116 16

9623 3.25 116 13

9983 11.92 116 5

10345 11.9 119 15

10704 1.08 126 0

11064 4.36 131 0

11424 10.86 133 11

11787 24.95 154 13

12148 47.34 160 6

12508 22.72 172 4

12868 23.89 183 0

13229 22.75 188 0

13590 21.67 194 12

13949 4.33 200 0

14310 20.61 201 8

14672 23.84 204 0

15032 24.81 204 0

15392 24.95 204 0

15752 24.95 204 0

16112 24.95 204 1

16473 23.87 210 0

16833 24.95 210 3

17193 26.04 210 0

17553 28.2 211 0

17913 28.2 211 0

18273 22.72 213 17

18634 9.77 219 6

18995 7.58 222 0

19356 0 222 7

19716 10.83 228 0

20077 24.95 244 6

20438 20.55 275 8

20798 21.66 292 17

21159 21.62 297 17

21519 17.42 301 16

21879 19.51 305 16

22240 27.12 312 18

22601 20.57 317 14

22960 22.72 320 0

23320 25.02 321 0

23682 23.86 322 19

24042 24.82 330 19

24402 27.04 335 19

24762 32.63 364 19

25122 27.12 386 23

25482 24.95 392 18

25842 29.29 394 19

26202 24.95 397 0

26562 24.95 398 0

26923 24.95 400 0

27282 24.95 401 0

27642 25.02 413 1

28001 24.96 416 9

28364 11.91 417 16

28724 10.8 420 14

29083 9.79 424 13

29443 20.61 439 12

29803 23.87 446 0

30163 44.48 446 0

30523 45.69 446 0

30883 29.32 452 10

31245 31.38 460 12

31606 18.32 467 10

31968 36.85 470 4

32331 32.39 479 0

32696 27.9 479 0

33058 40.67 480 1

33420 38.97 480 1

33782 33.35 482 0

34144 32.41 482 0

34506 32.45 482 0

34867 33.44 482 0

35229 35.7 482 0

35589 33.55 483 0

35952 21.55 488 5

36314 33.44 486 0

36675 32.47 486 0

37036 33.44 486 0

37399 26.97 486 3

37760 35.73 486 0

38121 29.13 486 0

38483 31.37 486 0

38843 31.37 486 0

39203 28.21 486 0

39567 27.04 486 0

39929 32.24 488 0

40292 23.75 488 0

40655 31.19 488 0

41017 33.37 488 0

41379 28.05 488 3

41744 33.47 488 0

42106 29.96 488 0

42468 30.19 488 3

42828 24.89 464 7

43188 24.93 548 8

43549 27.09 572 3

43909 24.88 572 0

44270 26.11 572 5

44632 24.88 565 5

44992 24.76 609 0

45352 24.95 682 1

45713 24.96 723 6

46075 21.57 728 18

46437 25.95 731 15

46797 23.8 721 17

47158 24.9 726 14

47518 24.95 730 15

47878 22.81 734 16

48238 21.69 738 16

48597 23.89 745 15

48958 22.78 750 14

49318 17.4 754 10

49678 25.97 759 14

50038 25.99 764 5

50399 22.78 796 2

50761 23.76 801 0

51121 25.89 801 3

51481 24.95 801 0

51842 25.02 801 0

52202 23.8 817 8

52561 17.41 828 10

52922 22.73 834 0

53281 26.09 834 17

53642 30.34 845 18

54003 32.54 854 13

54364 41.18 859 6

54725 41 860 18

55085 42.37 863 7

55448 42.07 864 1

55808 34.62 864 0

56168 3.26 864 0

56529 4.32 864 0

56889 4.34 864 0

The result comes from launching arma and then loading altis.

You can paste the results in excel and graph them if you wish.

 

The time intervals aren't constant either, but this will give you the information you need.

 

Just take into mind that the first time you load a mission will take longer than the second time you load it because the resources are still stored in your memory.

 

https://dl(dot)dropboxusercontent(dot)com/u/56725922/Resource%20Monitor%20-%20Copy(dot)rar

 

I stripped some content from the project because it was actually used to monitor server performance during running queries on MSSQL and MySQL servers. So, there might be strange references. (Project from an old school assignment)

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now

×