xtbbmalloc - A custom memory allocator for A3

[St. Jimmy 272]

Nice. I benchmarked multiple times AVX with LargePages and it's improvement almost like the fred's malloc. This is bit behind it (not really much but what I benched it's just a bit lacking behind) but better than the vanilla.

 

My results in YAAB:

 

Vanilla was around 36,7fps

xtbbmalloc 39,4 all the 4 times that I tested

fred's tbbmalloc was around 39,9fps, never fell behind the xtbbmalloc.

 

// I also tried SSE2 but the results waved from 38,0-41,4, so dam hard to say about that one.

[_blub 28]

fred´s used intels tbb 4.2 Update 2. I am using intels tbb 4.4 Update 5.

So those performance differences can come from changes between both versions.

 

There is nothing I can do about that. I will update this allocator if there is a new release of tbb :)

[lordprimate 159]

any word on getting this whitelisted with battleEye?

[_blub 28]

A word? How about "pending"? :)

I sent an email to BE with the file and the source. Lets see what they will do.

[torndeco 128]

Already told you absolutely no point sending source code.
BattlEye doesn't even look at the email attachments :)

edit: Infact when i included the source code, they just randomly whitelisted an older version of an extension. And told me it was done ;)
Been there, done that.

Just get it the dll blocked by battleye, then send them an email with the filename for the dll. 
Thats all you need todo
Try put in the title callextension / malloc developer, might get them to look at email faster *shrugs*

[_blub 28]

I am not at home so I could not produce any logs :)

Anyway, please download the following file (xtbbmalloc.dll). It is supported by battleye now.

 

https://dl.dropboxusercontent.com/u/103425066/xtbbmalloc.7z

[DieselJC 196]

So I downloaded the .dll that is accepted by battleye but where is this "ini" file?

[_blub 28]

You can use the old one (the one you find in the other download archives).

But you are right, I will include it in the new archive too.

ETA 5 mikes ;)

 

Edit:

Fixed :)

Configuration file is now included.

[screek 10]

/del fixed by myself

[DieselJC 196]

Ok so I downloaded the new updated file..have the dll in the dll folder..so do I put the "Configurations" folder in the main root or just the cfg file..sorry for the dumb questions.

[_blub 28]

This is not a dumb question :)

Just choose the right (the one you want to use) configuration file (xtbbmalloc.ini) from the "Configuration" folder. Move only this file (xtbbmalloc.ini) into your Arma3 root directory.

[lordprimate 159]

oops dont mind me thanks for the update!!

[Jastreb 69]

I just tried this and getting about 30 fps in EndGame EU#03 official server, however I also have this in the log:

15:40:47:	Init
15:40:47:	Config:
	User is admin: No
	UseLargePages: Yes
	SeLockMemoryPrivilege: Yes
	SeIncreaseWorkingSetPrivilege: Yes
	ForceMaxWorkingSet: Yes
	HoldMemory: No
	PreAllocBytes: 0.000
	LockPages: No
15:40:47:	Ext0SetPrivileges: 46, Error: 1300
15:40:47:	MinimumWorkingSetSize: 4096.000 mb
15:40:47:	Ext0SetWorkingSet: 0, Error: 0
15:40:47:	Using normal pages
15:40:47:	Init done

16:26:12:	Exit...
16:26:12:	...ok.

This is my ini

[Default]
# DebugBreak:
# Only for debugging purpose.
DebugBreak = 0

# UseLargePages:
# If you set this to '1', xtbbmalloc will try to allocate large pages (2MiB)
# instead of normal pages (4KiB)
UseLargePages = 1

# ForceMaxWorkingSet:
# Set this to '1' to force a working set size of 4GiB
# This is experimental
ForceMaxWorkingSet = 1

# SeLockMemoryPrivilege:
# Set this to '1' if you set UseLargePages to '1'
SeLockMemoryPrivilege = 1

# SeIncreaseWorkingSetPrivilege:
# Set this to '1' if you set ForceMaxWorkingSet to '1'
SeIncreaseWorkingSetPrivilege = 1

# HoldMemory:
# Dont release free memory
# Can improve performance in some cases
HoldMemory = 0

# PreAllocBytes:
# Pre allocate X bytes on startup. This can take a while...
# This is experimental. You should keep it '0'
PreAllocBytes = 0

# LockPages:
# Locks the allocated pages into physical memory.
# Cannot be used with UseLargePages
# >>Could<< improve performance
LockPages = 0

I am an admin on my PC but I dont run Steam/Arma3 as admin, I tend to avoid doing that as Ive heard it does not need elevated permissions. If I am wrong please do tell..

 

 

Running Phenom II X4 955BE, 8GB RAM, HD6950 all drivers latest available. Note that this CPU does not support AVX or any pre Buldozer CPU. I dont know if this allocator utilizes it but it sure is not required since I loaded it.

 

I believe that there is no change in performance at all. Will try again and update.

 

 

Edit: 

 

Running Steam and Arma3 as admin now gives this

16:38:02:	Init
16:38:02:	Config:
	User is admin: Yes
	UseLargePages: Yes
	SeLockMemoryPrivilege: Yes
	SeIncreaseWorkingSetPrivilege: Yes
	ForceMaxWorkingSet: Yes
	HoldMemory: No
	PreAllocBytes: 0.000
	LockPages: No
16:38:02:	Ext0SetPrivileges: 46, Error: 1300
16:38:02:	MinimumWorkingSetSize: 4096.000 mb
16:38:02:	Ext0SetWorkingSet: 0, Error: 0
16:38:02:	Using normal pages
16:38:02:	Init done

What is the error about, I cant seem to get it working? Its using normal pages. Also note that I dont know anything about any custom exe, I run stock stable 1.62.

[_blub 28]

Did you give your account the lock pages in memory privilege?

If not, check this guide if you need help https://msdn.microsoft.com/en-us/library/ms190730.aspx

 

I think this might be the problem :)

 

Edit:

A bit more time now...

Give your local user account the "lock pages in memory" privilege. You can to this with secpol.msc oder gpedit.msc (check the link above)

Then run Arma as admin.

[Horus 83]

This is my results with xtbbmalloc vs. BIS default malloc.

Phenom II X4 955BE@4.0GHz

4x4GB DDR3 RAM@ 1886MHz

ATI R9 270@1070/1500

w/o page file

[alinz 1]

 

 

Just a small gain shown in the benchmark, although it feels more like 5 fps higher in actual multiplayer games.

 

This is using the LargePages config.

[_blub 28]

See initial post for update.

New version: https://dl.dropboxusercontent.com/u/103425066/xtbbmalloc_2016_10_09.7z

 

I am short of time at the moment so I could not test this properly. If you encounter some bugs, tell me.

Because of this I did not send this version to battleye yet. If you guys tell me there are no bugs or problems I will do that later.

[Guest]

New version frontpaged on the Armaholic homepage.

xtbbmalloc v1.0.3

[DieselJC 196]

Going to try this new one out today...been using the old one and was getting better performance..maybe not so much frames but smoother with minimal stuttering for sure.

 

A question though..what are you guys/gals all running in your launcher..I found that keeping the memory limit boxes in the launcher both un-checked seems to be better for me...still really not clear on how those work if they limit the memory or allow a certain amount to be used.

 

Thanks for the newest version.

 

Diesel

[_blub 28]

Hi,

 

This is my config, I will go only over the performance related stuff:

- CPU Count: unchecked

- Extra threads: unchecked (there is no need to force anything I guess)

- Memory allocator: as desired

- Enable Hyperthreading: set (only set if your CPU supports hyperthreading)

- System memory limit: This one can be interesting. If you believe https://community.bistudio.com/wiki/Arma_3_Startup_Parameters then the game should auto select someting between 512-1536MiB if you leave this box unchecked. But I could imagine that this page is not up to date any more... Launcher says valid input is from 256 to 2047MiB but Arma could use up to ~4GiB in theory. So I would force this setting to 4096. If it works then fine, if not... well... it should not matter. But maybe a dev (dwarden where are you :)) can tell you more. By the way... I am sure right after they released the new launcher the tool tip said someting about way more than 4GiB but that has been removed. Maybe the launcher devs dont know about the limitations of their game, no offense of course :D

- Video memory limit: Launcher says anything over 2GiB may result in unforeseen consequences... I set this to 4096MiB... works.

 

I think thats it...

 

 

still really not clear on how those work if they limit the memory or allow a certain amount to be used.

I am not sure if I understand correctly. But I think if you set a limit to 1024MiB than the engine would simply not request any more memory when it reaches 1024MiB. The engine would try to release "unused" (cached) memory first before it requests more.

 

Hope this helps you :)

[Greenfist 1863]

- System memory limit: This one can be interesting. If you believe https://community.bistudio.com/wiki/Arma_3_Startup_Parameters then the game should auto select someting between 512-1536MiB if you leave this box unchecked. But I could imagine that this page is not up to date any more... Launcher says valid input is from 256 to 2047MiB but Arma could use up to ~4GiB in theory. So I would force this setting to 4096. If it works then fine, if not... well... it should not matter. But maybe a dev (dwarden where are you :)) can tell you more. By the way... I am sure right after they released the new launcher the tool tip said someting about way more than 4GiB but that has been removed. Maybe the launcher devs dont know about the limitations of their game, no offense of course :D

-maxmem=2047 limit is 'user defined portion' of memory used by engine

arma2oa and arma3 is 32bit LAA process able to allocate up to 4GB on x64 OS

e.g. you can see the usage can go easily over 2GB on client

...also the engine indirectly allocates more thru OS filemapping API (so it's cached by OS itself in memory) ...

-maxvram isn't limited to 2047 (not even 4095) and it's ignored if engine detects the VRAM size properly

Source: Dwarden on Reddit

 

For a long time maxmem could be set up to 32GB IIRC, but that was confirmed to be just a bug.

[Dwarden 1125]

ye the custom memory block can be defined up to 2047MB atm ...

the test with 3GB didn't go well (random issues on clients exhausting the memory space (way worse than actual memleak trouble)

and for more than 3GB it will need x64 (wip, no eta)

just. note official TBB4 allocator is now on same 2017 update 1 as base, for profiling branch (next will be also DEV)

https://forums.bistudio.com/topic/160288-arma-3-stable-server-164-performance-binary-feedback/page-85?p=3102705#entry3102705

[das attorney 858]

Hey Dwarden,

 

Thanks for info on mem allocator.

 

I noticed a new one popped up in the Dayz Standalone dll folder:

tbb4malloc_bi_x64.dll

I copied to Arma folder and tried to use, but game defaulted to tbb4malloc_bi.dll

 

Presumably, it's DayZ specific, but is there anything useful in it that can make it over to A3?  Or is there anything new in memory handling that you would make a 64bit platform version of the memory handler?

[Dwarden 1125]

will not work, x64 libraries needs 64bit process https://msdn.microsoft.com/en-us/library/windows/desktop/aa384231(v=vs.85).aspx

also that x64 allocator in DayZ is 4.0.x from 2014 so quite outdated ...

[lordprimate 159]

ok so there was an update for this posted with NO change log... and i do Not see ANY information Here about this update....

 

Where did this update come from??