fred41

With the latest stable update 1.68, large page usage is simpler than ever before. Basically large page mapping is possible and useful for 1. code and 2. data for 1.(code): use regedit to create/set DWORD value 'UseLargePages' = 1, for key [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\arma3_x64.exe] for 2.(data) either use BI's default allocator and simple add -hugePages to your arma start line (i think the launcher provides a checkbox for this already), or use blub's 'xtbbmalloc.dll' for some interesting features like diagnostics, etc, the performance advantage should be the same In any case, your (windows) useraccount must have the 'Lock pages in memory' privilege enabled (probably already done, if not use secpol.msc).

Project terminated. ASM did his job a long time now and i hope you enjoyed it a bit. This monitoring project was originally mean't as a temporary solution and as an example. What most users don't know is, that ASM is based of a lot of very tricky interfacing implementations, like interprocess communication based on shared memory, API hooking, callextension, e.t.c, just to present the user some data, that are already available inside the engine. Months ago i have made some really practical suggestions to BI, how a simple but professional monitoring interface could be implemented directly in arma server. I got always the answer 'developers are to busy' to do that. Meanwhile arma is 2 years old and there is still no professional solution provided by BI. Considering this above, i hope you understand, that i really see no reason to support arma server any longer. Best regards, Fred41 ASM is a little program, for monitoring some interesting values from multiple server (or HC) instances. Performance related values like: simulation cycles per second average (FPS) simulation cycles per second minimal (FPSmin) condition evaluations per second (CPS) number of alive players (PL#) number of alive server local AI units (AIL) number of alive server remote AI units (AIR) number of mission objects (OC[0..2], free configurable) amount of allocated physical memory (MEM) network traffic incoming in kbytes/s (NTI) network traffic outgoing in kbytes/s (NTO) disk reading in kbytes/s (DIR) name of running instance name of running mission are displayed, simultaneously for multiple (up to 4) server instances, in a window. The idea behind it, is to provide a fast and easy "state of health" overview at one go. more details here: https://github.com/fred41/ASM/ library and server application for NATIVE LINUX, provided by KILLSWITCH, are available and described here: Arma Server Monitor for Linux Update: Arma Server Monitor values are now accessible from remote via TCP (inspired by terox, as usual:) Changelog: 01.06.2013 Changed the install/launch method to mini-addon (thanks terox for inspiration) 01.06.2013 Fixed: instance occupied additional slot at mission change 02.06.2013 Changed bar graphs for CPS & FPS to logarithmic scale to show states more intuitive 03.06.2013 Added history graphs, customizable (individual visibility, update interval), transparency switch 08.06.2013 custom build binaries, for use with DS as service added ("Global\" namespace for MMF access, usage see my post 08.06) 11.06.2013 fixed: AI counting now works correctly (thanks terox for reporting) 13.06.2013 support for run DS as service, now merged in default build (custom build removed) 17.06.2013 remote monitoring now available via additional ArmaServerMonitor instance, configurable via start params (see github readme.md for details) 18.06.2013 value for amount of allocated physical memory in [MB], MEM added 04.07.2013 max player number adapted from 50 to 100 05.07.2013 history graph extd. (200 -> 800 samples), better lag visualisation via FPSmin (activation in popup menu) 09.07.2013 some performance related improvements, to update replace ALL related files (ASM.pbo, ASMdll.dll and ArmaServerMonitor.exe) 11.07.2013 profilName now displayed additional to missionName; forced=1 -> preInit=1 in cfgFunctions 15.07.2013 counter for all allMissionObjects added (helps to check if cleanup routines works correctly) 18.07.2013 Memory (MEM) changed from GB to MB 31.07.2013 object counting interval configurable in asm.ini (see readme.md), additional to local AI, remote AI is counted now too 31.08.2013 userinterface improvements, use popup-menu to check it out (thanks zach72 for ideas) 26.09.2013 Values AIR & AIR rescaled from 400 to 800 max. 22.10.2013 UI settings saved/restored to/from registry at session end/start 14.11.2013 FPS,CPS,FPSmin graph changed to lin. scale, MEM bar scale to max. 4GB now 24.11.2013 Souce of ASM.dll added to git hub repo, feel free to use it in your projects 02.12.2013 Historygraph extended to 86400 seconds (24h), record to RAM ringbuffer, scroll via LMB, reset via dbl.click, timedivisor via popup menu (thx to dwarden, for idea ;)) 06.12.2013 simple log feature added (-lfilenameprefix, -tinterval) 07.12.2013 OBJ value added as optional value for logging to file 08.12.2013 fix: instance slot blocked, caused by arma server crash (full update required) 13.12.2013 three customizable object counter added (set interval and .sqf command in asm.ini) 20.01.2014 added a little hint for current PID (process ID) of server instance (hoover over or click profileName string value to show) 08.02.2014 vert. gridlines and some hints to variable fields added, width of history view scalable (-wwidth, default 900) 15.02.2014 'bold'mode added, use additional -b param in commandline to activate 23.02.2014 sampling interval (1 second) now synced via timeGetTime(), to minimize longtime drift 16.03.2014 memcpy replaced by strcpy_s in ASMdll.dll (thanks killswitch for report) 22.03.2014 added API values NTI, NTO & DIR, some fixes like: logging not stopped when mission ends 04.04.2014 fix: AIR sometimes returned negative values (thanks dwarden for bug report) 17.04.2014 enableAPImonitoring added to asm.ini to switch of API monitoring feature (NTI/NTO/DIR) 26.04.2014 fix: bug related to enableAPImonitoring, caused double slot usage with HC (thanks tankbuster, for report) 28.04.2014 fix: monitoring NTI/NTO values, works now with A2, set enableAPImonitoring=2 in asm.ini (thanks Viba, for reporting and testing) 14.05.2014 for more then 4 instances (up to 16) a new commandline param, -o is introduced (for 4 blocks, of 4 arma server instances each) 22.05.2014 removed old debug code from ASMdll.dll, DIR-value counting related fix 24.05.2014 profile prefix based slot selection added, usage: arma3server.exe -name=xxwhatever, where xx is the prefered ASM slot (zero based) 23.07.2014 changed library name for callextension commands, in fn_ASM.fsm, to lowercase (thx Killswitch, for hint) 29.08.2014 library and server application, by KillSwitch, now available for native linux °° 14.12.2014 current date added to log file name 30.12.2014 timestamp algorithm "tightened" (to prevent double slot occupation in very low fps conditions) !!!please always update ALL files together (data structures of interface are sometimes changing)!!!

... since this thread is now clearly related to tbbmalloc, an experimental custom memory allocator for arma, i will rewrite this initial post ... (please notice, the /LARGEADDRESSAWARE flag is set correctly for all current arma editions, this old topic just can't be edited now) The current version of 'tbbmalloc for arma', including a short description, can be found here: https://github.com/fred41/tbbmalloc_arma (Note: to get a file from github, you can ether: download the full repository (Download ZIP) and extract the needed files locally OR download one specific file by selecting the file and click RAW button) The following improvements are currently implemented in 'tbbmalloc for arma': - supports the use of 'large pages' (if correctly configured and available), which results in ~ 25% better memory access performance for all read/write operations to allocator memory - some internal optimizations are improving the overall allocation performance (please check your malloc_xxxx.log file and verify that large pages are available and used correctly, 'Alloc LP'). If your OS version don't have secpol.msc, you can use GMF.exe tool in the tool folder in tbbmalloc github repository. See readme.txt for usage details. Please let me know your results, Fred41 BTW: Did you know, that mapping a3 image file to large page memory region, significantly accelerate execution performance? Look for 'GimmeMoreFrames' in my github repository, for a very simple, but powerful registry tweak. If you already have the 'Lock pages in memory' privilege set correctly, it is a very simple tweak. https://github.com/fred41/GimmeMoarFrames

@dlegion, i think i would actually buy such an DLC. This idea sounds a bit crazy first (and maybe gentle provocative too), but somehow it is inspiring. Thanks for your effort to bringing some fresh ideas and spirit back here ;)

I am assuming you use the 64 bit version of arma (are you?), just look here: How to check pagefile size. With 8 GB RAM, a pagefilesize of ~12GB should be sufficient.

your pagefile size is ...?

this registry entry is defined as a 32-bit DWORD and this is where your OS is looking, but yet a funny idea :)

No, like this very thread.

@abudabi, simple spoken, this both advantages are based of large page mapping for 1. code and 2. data for 1.(code): use regedit to create/set DWORD value 'UseLargePages' = 1, for key [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\arma3_x64.exe] for 2.(data) either use BI's default allocator and simple add -hugePages to your arma start line (i think the launcher provides a checkbox for this already), or use blub's 'xtbbmalloc.dll' for some interesting features like diagnostics, etc, the performance advantage should be the same In any case, your (windows) useraccount must have the 'Lock pages in memory' privilege enabled (probably already done).

I think OP has requested legal things and there is nothing wrong doing so. There are a lot of outstanding fixes/improvements that can't be fixed, or at least not efficiently, without access to arma's source code. Since some of this things are requested for many years, it probably makes sense to remember from time to time.

I think, monitoring a server is not only useful, it is essential for tuning a server and to keep things properly running. Hence it is hard to understand, why BI still does'nt provide a serious (and easy to use) interface to support this requirement. Using callextension for this purpose was a very limited, tricky and ressource consuming approach. I nevertheless did it in ASM, to show how monitoring could basically look and there was no other way to do it. But now, years after armas release? I guess you guys have to bug them a bit harder, to get such an interface ;)

@abudabi, actually this is very unlikely, since BE isn't allowing any modification/investigation at binary level and arma is still closed source. Anyway, there is meanwhile a '-hugePages' launch param for client and server and my tests are showing that it is working well. Binary large page mapping (this very registry tweak) works flawless for 64 bit server and client too. Just use this both advantages, it is still totally free and easy to use.

... yes. arma_x64 is now able to access data cache directly, instead via file mapping api. The large file mapping, used by 32bit process, doesn't contribute to process virtual memory usage, hence this difference.

@nikiforos, if i feel i have something serious to contribute again, why not, but currently this is not the case. Regarding your home edition's lack of 'secpol.msc', i remember there was a guy at guru3d, hijacking this idea here and offering a tool to setup binary-LP mapping for many different games, including arma. I didn't try it, but perhaps this is an solution for you.

Hello @archibald tutter, i am still on my good old 2500k@4GHz (bought used, btw), so i am not a victim of any cartel's :) But yes, loocking forward to AMD's ZEN incarnation too. As for arma's performance problems, i think it's memory bandwidth is still way to hi. Cache locality is the first thing you have to care about on today's cpu's, for optimal performance, even since DDR4. There is still a lot of potential lurking around, so buying more and more powerful hardware, just to get 33 instead of 30FPS, is not the way, imho.

I just now benchmarked with helos 0.60 and the improvement is like from 75.0 to 77.5, compared to 32bit version a very small difference. With yaab i don't see any consistent difference. At the end not enough to make a tool for that again. However, this probably means BI did his homework better now.

@NoPOW ... lurking, of course, especially if waked up by 64bit announcements ;) While modern cpu's, like 'skylake', are equipped with larger TLB (translation lookaside buffers) and hence reducing the number of costly TLB misses, the TLB's are still a bottleneck and even more for 64 bit applications, because they usually have to access more memory pages. So yes, large page usage for code and data (in this order) will still help arma to run better. BTW: If you remember the good old 'GMF tweak', a totaly free and very efficient way to improve memory performance, this thing seems to work perfectly with the 64bit arma binarys again, without any executable patches. Just the 'Lock Pages in Memory' privilege and the registry entry adapted to 'arma3_x64.exe' works flawless for me.

First congratulation to this important step, there is already a nice difference compared to limited 32 bit version. This should open the door for further improvements, requiring large amount of process virtual address space. A question/hint though, related to file/data cache implementation: A closer look reveals, that windows file mapping API is still used, just with a larger section object (~4GB) initially created. There are still frequently MapViewOfFile/UnmapViewOfFile calls to map 64kb windows in process virtual address space. While this approach did make sense in 32bit implementation, i would say this is unnecessary complicated (and a wasting of resources), considering the huge 64bit address space. If not the caching system should be generally reworked, at least a simple adaption could be done as following: - replace the initial CreateFileMappingA(...) by a VirtualAlloc(...) with identical size - replace all related subsequent MapViewOfFile/UnmapViewOfFile(..., FileOffset) calls, by simple pointer arithmetic Perhaps, to prevent windows from early paging to disc, increase the min/max workingset size, respecting systems physical memory availability. I wish you all a nice winter holiday :)

... from my (earlier) experience and in respect to large page usage, backing your physical memory pages by a pagefile (preferable OS recommended size, fix) is helpful/necessary for windows OS, especially if you have to defragment your physical memory later to improve availability of large pages. However, a very clean windows system (less background memory activity, no memory leaks) can survive without any pagefile and still have some large pages available, though. BTW on linux huge page support is significantly better (transparent huge pages, huge page daemon, huge page file system ...)

How much can this 64bit executable help? ... i would say it depends ... Being still a 32bit application, is limiting arma already, in multiple ways: 1. the well known 3 FPS 'bug' is nothing else, as a symptom of a overused (full) virtual address space (< 4096MB for 32bit process on 64bit OS). Memory intense scenarios/mods are affected more and earlier. 2. the current approach, to use some physical RAM outside the 32bit address space (via file mapping API, limited to 1,7GB now) as data cache, is responsible for the stuttering while mapping/transfering data in armas address space (stuttering while turning, zooming, etc.) 3. a 64bit process can hold significantly more of this data in physical RAM, directly accessible, in an practical unlimited linear address space (no mapping/unmapping and especially no/less RAM copying required). Assuming there is enough physical RAM available (not really a limitation on todays gamer machines), i would expect a big potential for performance improvement. 4. before i don't see a working 64bit arma binary, i don't believe on it ;) Regards

... just found this statement in latest SITREP #00166: From user reports, it looks like problems happens, when virtual memory used by arma process is 3000-3500MB. Considering the fact that each user process on windows, has its own virtual address space and that arma is still a 32 bit process, it seems natural that such problems are related to the limitation of 32 bit address space. Even if the total 'mapped' virtual memory is still some hundreds MB below the limit of 4096MB, it is more likely that a request for a larger block of memory fails later, simple because there is no continous block of virtual address left, large enough to map the requested ammount (fragmentation of address space). This seems not to be fully understood and i hope my hint will help to rethink the current 'bug hunting' approach. A recommended tool for address space analysis is VMMAP: https://technet.microsoft.com/en-us/sysinternals/vmmap.aspx Regards

@Nikiforos, just reapply, via LPManager (uncheck and check 'LP ImageFileMapping Client'). If you forgot this step, you are trying to run a 'arma3LP.exe' ver. 1.42, on 1.44 data ;)

... at least not from me ... ASM is designed as serverside only addon and should be used as that. I am not sure, but i think it is not a good idea, to allow each connecting client to have a @mod=asm running. Theoretically, such clients could use "customized" callextension .dll's this way (cheating)! I would say, it is a bit sad, that the HC is still not able to act like a server in respect to signing, but currently you can only sign ASM your self and ensure that nobody except you have access to this key. Anyway i am not a signing expert, so maybe i am totally wrong ;)

... thanks for your feedback, guys ... I think, generally you should get similiar relative improvements like before arma 1.38 (in percent, %). I got the following result, on windows 7/64bit, using helo's altis benchmark (5 runs each, averaged): 83.3 arma3.exe ... 94.1 arma3LP.exe -malloc=tbbmalloc ... ((94.1/83.3) - 1) * 100% = 12.96% ~ 13% Considering the fact, that this effect is only caused by large page usage, for most of the data and all main module code, this result is not that bad. Anyway, i think it is still a lot of effort to set it up correctly and therefore primarily interesting for the performance enthusiast :)

... for all GMF fans, here is a little update ... I'am trying to 'reanimate' this tweak for all 'non-BE' sessions, like SP or LAN-MP. Since it is an additional patch of arma client nessecary to make this tweak work, you have to bar in mind: DON'T use a patched arma client to join a 'battleye enabled' session!!! Please read the readme in my GMF repository for detailed usage instructions and read it carefully. If you prefer the manual method, i recommend to work with a copy of 'arma3.exe' to reduce the risk of unintentionally joining a battleye enabled server. For the less experienced users, there is an updated 'LPManager' available again, to save you some work and stress. This tweak must now be reapplied with each arma update to work, i am sorry. Enjoy :)