Custom Memory Allocator for engine since b85869

[On_Sabbatical 11]

sceneComplexity=1000000;

It's too much !

[.kju 3245]

setViewDistance 3000;

setTerrainGrid 3.125;

Both are set. So your personal settings should have no effect.

sceneComplexity is the same as object details in video options.

(you might apply lower or higher values though than those settings available via UI)

[scruffy 22]

Both are set. So your personal settings should have no effect.

sceneComplexity is the same as object details in video options.

(you might apply lower or higher values though than those settings available via UI)

Ah, so the first are set anyway, didn't care to check first.

So what we can take from this is just: Don't bench in your gpu limit.

New results with Version 1.60.87136

malloc=tbb3malloc_bi
["AVERAGE FPS: 71"]

malloc=tbb4malloc_bi
["AVERAGE FPS: 80"]

malloc=jemalloc_bi
["AVERAGE FPS: 78"]

malloc=nedmalloc_bi
["AVERAGE FPS: 73"]

malloc=tcmalloc_bi
["AVERAGE FPS: 80"]

[.kju 3245]

Version 1.60.87636

//malloc=tbb3malloc_bi
["AVERAGE FPS: 30"]

//malloc=tbb4malloc_bi
["AVERAGE FPS: 32"]

//malloc=jemalloc_bi
["AVERAGE FPS: 33"]

//malloc=nedmalloc_bi
["AVERAGE FPS: 32"]

//malloc=tcmalloc_bi
["AVERAGE FPS: 32"]

//malloc=system
["AVERAGE FPS: 29"]

[Dwarden 1125]

i think best would be report always

minfps, avgfps, maxpfs

cause spikes in minfps is what i consider most important, then i look at avg fps

[.kju 3245]

yep but thats the stats from the basic OA benchmark mission.

the PvPscene benchmark suite has a lot more detail

[Dwarden 1125]

yep but thats the stats from the basic OA benchmark mission.

the PvPscene benchmark suite has a lot more detail

not realized that :) anyway the aim of my comment stays ...

avg FPS don't tell the whole picture

[BL1P 35]

How do i update the runMemAllocBeta to use my paths ?

[DualJoe 10]

avg FPS don't tell the whole picture

Indeed, even more so when testing the different malloc options I've noticed. I've forced VSync on, so I got an average below 60 fps for the different malloc-settings, I don't care about the range above 60hz anyway. Eventhough the average framerates are very close to each other with the pvp-benchmark, what you see on screen is a whole different story.

Tbb3 and 4 are much too jittery for me even at 60 fps, I also get a number of graphicall issues. I get the smoothest motion with tcmalloc, too bad it crashes after a while. Which leaves me with nedmalloc and jemalloc, I've run with both for long periods of time with no crashes so far. Jemalloc gives me a higher average framerate, but subjectively nedmalloc feels smoother to me.

It could be because of the major improvements in the latest betas and 1.60, but I'm starting to suspect the AI is more heavily influenced by these memory allocators than framerates are. For example pathfinding. In benchmark #8 in the 3rd scene there's a panning shot of a grassy field and a usually empty road, suddenly with jemalloc and nedmalloc I more often than not get to see 2 tanks driving by in the middle of the shot. There are more instances I noticed a difference, but this one is the most striking and reproducible example I could think of.

[Dwarden 1125]

afaik tcmalloc and maybe also jemalloc and nedmalloc would/shall benefit from recompile using newer version of codebase which may prevent some of the crashes ...

feel free to compile with newer code and tell us results

http://code.google.com/p/google-perftools/downloads/list

http://www.canonware.com/download/jemalloc/jemalloc-latest/

https://sourceforge.net/projects/nedmalloc/files/nedmalloc/

as the changelogs often mentions crash and performance improvements ;)

[DBGB 10]

Is it just me or is the allocator not reported in the .rpt file anymore ?

Only seems that the argument's passed when launching the game is written in the latest official patch 87580.

[Neopas 10]

I don`t see it either when using default.If I define a different one as a parameter it shows up though.

[DualJoe 10]

@Dwarden: Thanks for those links, recompiling tcmalloc has fixed the crashing issue for me, but I also made it a little less smooth. Despite that it still is noticeable faster for me on all fronts, be it framerates, load-times or AI. With tcmalloc the tanks are either long gone or I just get to see the last tank in benchmark #08 scene 3. Maybe some sort of AI-benchmark could be useful as part of the malloc tests.

Haven't got a clue how I should compile jemalloc or nedmalloc though. I've become so comfortable with linux, that I don't even know where to begin anymore if I have to compile something for windows. Would have liked to try some hardware specific compiler flags to see how much of a difference that would make.

[Dwarden 1125]

can You provide Your recompiled libraries as download? (including sources ofc to keep with the licenses)

also please try it with beta build 1.60.87662

(note since recent betas IntelTBBv4 is default allocator)

[DualJoe 10]

EDIT3

Lol, I messed up, as SUMA pointed out. Everywhere I use tcmalloc down below, read windows 7 system default.

Wel it explains why it felt a lot less smooth, but I do find it odd that the system default is also the quirkiest and most inconsistent performer of the lot.

Sure I don't mind, I'll have to search for a place to upload them first.

Do I have to post the sources eventhough I didn't do anything apart from downloading the latest version and running the visualbasic file that is included?

Darn, hang on a moment, I just installed the latest beta without thinking what that would do to my replacement dll.

I'll post the link here later this evening and do a couple of testruns with the latest beta and the different malloc-options.

EDIT

I removed the tcmalloc_bi.dll and replaced it with this dll (<= removed the link, because it was rubbish) which I compiled from version 1.9.1 of google-perftools available at the link Dwarden posted earlier.

EDIT2

First result of testing with beta 1.60.87662 and the above tcmalloc dll.

I used the runMemAllocBeta.bat from the pvp-bechmark and ran that a couple of times and then ran 5 test runs.

testrun:	1	2	3	4	5		average
tbb3		46	49	54	55	49		50.6
tbb4		50	51	49	52	52		50.8
jemalloc	51	52	50	53	49		51
nedmalloc	47	48	45	48	47		47
tcmalloc	57	53	50	51	50		52.2

I'm running windows 7, on a Core2Duo e8500 clocked at 4Ghz, 8 Gig ram and a nvidia GTX560Ti.

Eventhough nedmalloc has the lowest framerates it looks the smoothest. Tcmalloc does have a rather dramatic higher framerate on my system compared to the others. Especially when there's a lot going on like explosions and smoke.

Some things that I noticed while running this benchmark numerous times. Tbb4 consistently fell behind with pathfinding, I only saw the 2 tanks driving by in scene 3 once and then only at the end when the screen fades to black. With tcmalloc the pathfinding was less consistent than the remaining 3, sometimes the tanks were too fast, but it also happens that only the first tank is visible at the far left of the sweeping cameraview. Tbb3 always had the camera pointing between the 2 tanks in each and every run. Jemalloc, nedmalloc the tanks are consistently faster with the camera pointing behind the second tank or catching up to them at the end of the pan.

Edited December 31, 2011 by DualJoe

[suma 8]

I removed the tcmalloc_bi.dll and replaced it with this dll which I compiled from version 1.9.1 of google-perftools available at the link Dwarden posted earlier.

Replacing allocator is not this easy. The allocator needs to be adapted to provide an interface required by the game, as shown by our sources. If the interface is not there, the Windows allocator will be used.

[DBGB 10]

I also tried to compile the latest tcmalloc from google perftools - the differences between the sourcecode provided by BI and v. 1.9.1 can be used to infer what needs to be updated in 1.9.1.

I worked a bit on adapting 1.9.1 -> creating the interface, but it's a wip because there are some substantial differences that gives me some issues.

You have to specify some compile options that will enable C++ exceptions.

Especially in the sourcefile tcmalloc.h + tcmalloc.cc changes have to be done that I haven't figured out yet.

But maybe its easier to adapt TCMalloc_bi to comform to the functions and number of arguments in tcmalloc.h (+ tcmalloc.cc).

Further inferences can be done by comparing between the base for BI's implementation of the different mallocs and the sourcecode they are based on.

FX. take TBB4 from intel and compare that to BI's implementation which includes the necessary interface.

[DualJoe 10]

Lol, well sure shows what I know. I'll edit my previous post then, to avoid confusion.

[XATRIXKIEV 10]

Hi guys...

I'm using Linux version of a server 1.60.87589. And i couldn't find any information about how to tell the server to use custom memory allocator.

I'm running Lockless Memrory Allocator (http://locklessinc.com) on my Gentoo system.

Currently i have such a global variable in my system LD_PRELOAD="/usr/lib/libllalloc.so"

and my server gives me an error during the startup process... but still be able to complete startup process:

WATCHDOG (5459): [Mon Jan 2 16:36:21 EET 2012] Starting server (port 2310)...

ERROR: ld.so: object '/usr/lib/libllalloc.so' from LD_PRELOAD cannot be preloaded: ignored.

Item str_disp_server_control listed twice

16:36:45 Dedicated host created.

16:36:46 Host identity created.

Anyway i can make a "dll" directory, down to the Arma2OA root dir. And place my "libllalloc.so"

Just tell me the right server parameter to preload a custom .dll ?:

xatrix@XATRIX-SERV ~/games $ ls dll/

libllalloc.so

xatrix@XATRIX-SERV ~/games $ cat arma2oaserver

....

OTHERPARAMS="-world=none -cpucount=2 -exthreads=3 -maxmem=1600 -malloc=libllalloc.so"

....

[Dwarden 1125]

Linux server uses OS (nix kernel) allocator, no way to add it yourself and no point to let it use any custom ...

the line explaining it is also on BIKI: http://community.bistudio.com/wiki/ArmA_2:_Custom_Memory_Allocator#Dedicated_server

[LkFp 10]

Are you sure you are not somehow mixing 32/64bit archs and/or allocator lib exists? LD_PRELOAD should AFAIK work independently on target application. No support needed from Arma side.

[.kju 3245]

Version 1.60.87662

"AVG FPS	AVG MIN		HIGHEST		LOWEST"
//tbb3malloc_bi
"31		25		53		8"
//tbb4malloc_bi
"32		25		44		6"
//jemalloc_bi
"32		26		48		6"
//nedmalloc_bi
"31		25		57		7"
//tcmalloc_bi
"33		26		60		5"
//system
"30		23		51		5"

Version 1.60.87756

"AVG FPS	AVG MIN		HIGHEST		LOWEST"
//tbb3malloc_bi3
"33		26		57		6"
//tbb4malloc_bi
"32		26		50		2"
//jemalloc_bi223
"32		25		53		7"
//nedmalloc_bi
"29		24		40		2"
//tcmalloc_bi
"33		26		57		6"
//system
"30		23		48		6"

---------- Post added at 17:36 ---------- Previous post was at 17:34 ----------

Anyone up to build a simple app to build a graph from FPS data?

Or some app already out there we can use?

[Fireball 16]

Anyone up to build a simple app to build a graph from FPS data?

Or some app already out there we can use?

ArmA2FPSAnalyzer

[.kju 3245]

It doesn't work with OA..

[-=seany=- 5]

Is tbb3malloc_bi still the default Memalloc since the 1.6 patch and the following betas?

And tbb3malloc_bi has always been the default also, is that right ?

Just wanted to check as I feel I'm getting more instability since the 1.6 patch.