Freghar

Thanks for the link. I kind of figured out he was a troll few posts ago, but I still wanted to learn a bit more about Windows memory management and this was a nice opportunity. Turns out it's pretty similar in basic concepts to how we do things in the Linux kernel. The funny thing is that I actually did the benchmarks without pagefile or ramdisk in the end, both of which he seems to recommend. :) FTR - SSD would help initially, because the mapped files aren't yet in the page cache. Assuming the page cache is big enough (MSDN seems to mention some problems Linux doesn't have thanks to the "swappiness" tunable :) ). The interesting thing is that NT kernel apparently shows page cache as "free" memory. Go figure.

Yes, of course. Because that's how Arma 3 memory management seems to work, regardless of whether you have it enabled or not - it uses the file mapping API and the OS deals with it somehow, whether it has a paging file or not. I also tend to be skeptical about programs showing one RAM value without saying what it actually measures - allocated virtual memory or pages with data? I don't exactly know the impact, I haven't measured it and I presume you haven't either. Game shuttering isn't a valid measurement of RAM usage - one would have to record the shuttering separately and match it against RAM access data, collected at much finer granularity than I do (and at MMU level). Then, one would have to look at the cause for the shuttering (nr. of draw calls?) and prove that RAM speed is indeed the limiting factor. It might very well not be. Yes, I know that DRAM/SDRAM is. I have soldered a few. I also am aware of what is memory refresh and that it becomes less significant with higher memory usage (ironically). ;) And yes, I know the difference between 32 and 64bit virtual addressing space. You seem to misunderstand what virtual memory is - it's not pagefile, it's just an abstraction above RAM+pagefile, so that the application doesn't have to deal with where its data is stored (simply said). And yes, for some weird reason, I don't get as much shuttering ingame - hard to say why. It may be some specifics of Windows memory management or how Arma works with the file mapping API. It's not magic and probably has some explanation, but I don't have enough time, interest or source code to dig deeper. Without paging file, I get http://i.imgur.com/Kk9KiDY.jpgfor a few seconds when I quickly look around. With the pagefile, I get it only once per mission. You're kind of right that it doesn't make much sense as (on Linux, but Windows may do it similarly) you can only use "swap" (paging file) for anonymous memory, not file-backed, kind of due to how page tables work. But again - the MMU just calls the page fault handler, which differs per OS. Well, it's kind of related to everything due to how memory-mapped files work as they're (presumably) the reason behind this topic. The game is not loading 10GB of virtual memory as it has only (up to) 4GB. Similarly, as the pagefile can only contain paged out anonymous memory for the process, it can never have more than (up to) 4GB of data for that one 32bit process (unless Windows does mm in some very weird way). Right, because it's not. IF indeed 10GB of physical pages is used, it's not loaded by the game. The game can only map small chunks of files into its memory at a time as it needs to fit the (2-3GB) free virtual memory addresses. When it unmaps the chunk(s), it's up to the OS what it does with the data. We don't know the RAM is being used, we know that some program displayed a nice big green number without telling us what it is. We don't have a clue because we (you?) haven't investigated enough into what the memory is actually used for. I didn't say I can load (play) the game, I said implying that I used to run (Windows) with pagefile disabled. Actually, I was running Operation Flashpoint back then and without Windows Aero and other visual mess, 2GB was very much enough. It was more of a philosophical statement, as in - any game can use whatever hardware resources it wants, but its commercial success will be dictated by how many people can actually run it. As in - the game knows best how much RAM/VRAM it needs for the textures and if you want the best ones it has, you'll need to meet the RAM requirements (or suffer slowdowns). No, I don't. Page cache seems to be working without it and SuperFetch (according to Microsoft) seems to be just boot-time preloader + some algorithm that tracks what you do and pre-loads files into page cache based on time of day and that kind of stuff. As I use Windows only for games, I don't start it regularly enough for such heuristic to have a meaningful effect. PS: I've done some additional measurements, both with pagefile disabled, the first one with my 6GB ramdisk still active, the second one without it (~16GB RAM free initially). To make things interesting, I added per-process statistics (arma3.exe) for both (cuts into the first one as I added the counters, the second graph has them already set up). As you can see, even without the ramdisk and pagefile, only about ~3GB of memory was really used. The rest is reclaimable for other purposes by the OS and is "used" only for optimization purposes (which is a good thing). And even accounting for this, only about 8GB of physical pages was actually used overall, incl. processes other than Arma. So no, I cannot reproduce your "issue" no matter how hard I try.

Okay, so I managed to (mostly) localize perfmon to English by copying a few files around and did a few more measurements, ("Měřítko" means multiplier.) Apparently, Windows has one additional "layer" between allocated ("reserved") and actually used (COW'd pages) memory - "commited" memory, defined as The amount of Commited memory did indeed go up to ~15GB, however note that this is not memory that's actually used (and also that it was at 9GB before Arma started, probably the ramdisk). Am I reading something wrong? edit: https://support.microsoft.com/en-us/kb/2160852 So it's unclear to me whether the commited memory is used (but paged out) or just "more reserved". In any case, (RAM total - RAM available) + pagefile usage are still below what's considered commited, so I'm inclined to believe the space is not actually used.

I use SoftPerfect RAM disk software, which allows me to use a "boot" ramdisk, apparently set up before Windows initializes paging files. I have never investigated why it makes repeated mission load times faster for me, there are multiple possibilities, but no time to dig deeper, though. :( As long as it's reproducible, great. The more pressure we can make for using 64bit, the better. :ph34r: bratwurste's "issue" might also be a real one, the thing is there's a lot of presumptions about what causes performance issues and what doesn't and that high usage might actually be by design, to speed things up. If it is real and not imagined by MSI - would be useful to know *what* that number measures. It's mentioned in the other thread, perfmon.msc. It's unfortunately localized to the language of Windows, which is why I posted text translation of the legend instead of including it in the screenshot. I was really surprised what you can measure - we have "perf" on Linux which can do similar things, but I never really expected something like that from Windows. :)

On the bug / feedback tracker? Any proof of that? Normally, high memory usage indicates heavy caching = less waiting for disk I/O or page faults. At the very least, it's not an indication of "performance" as in "FPS". That's not weird at all - the memory is not actually loaded by the .exe, it's cached by the system. ARMA actually uses the pagefile API to make the system do that (instead of using the memory locking API or actually being a 64bit app), read more on https://www.bistudio.com/blog/breaking-the-32-bit-barrier . Not true, see above. That's why I have my pagefile on a ramdisk and can definitely see faster mission load times than without pagefile (still on rotational HDD here). Any game can use any amount of ram it wants to get the job done - if you selected Ultra everything, expect Ultra resource usage. Timings (as in memory timings) have absolutely nothing to do with this - the game cannot possibly use all the RAM at once. Actually, frequent memory access would be a much bigger problem than lots of RAM used. I used to run with 2GB of RAM on win7 with pagefile disabled. :) My personal findings can be found in that other thread, I repeated them with everything set to max distance and Ultra (even under the post processing tab), after a game restart (to be sure): red = read/written pages per second (unit = 1 page) blue = free memory (unit = 10^-7 bytes, eg. 800 = 8GB) green = page file usage (unit = 10^2 %, eg. 100 = 10%) The first graph is from me trying the Zeus 4+1 bootcamp, so I could fly around. Turns out the mission had some overcast which cannot be disabled via Zeus, so I loaded an editor mission with a littlebird and spent a few minutes flying (second graph), with full 12km distance. Both high and low altitude. I even spawned several full squads of units fighting. As you can see, with my 16GB RAM (6GB of which is dedicated to ramdisk with pagefile), I cannot reproduce the issue. I was able to get (in the worst case) 3.4GB of RAM usage and only few % of pagefile above that (6-8% change out of 6GB). Similarly to my previous graphs - you can see Windows clearing up RAM and putting the pages in the pagefile, at a rate of ~256 pages/sec (red line). These readings are from the native measurement tool that comes with windows and uses kernel hooks to get the data. You cannot get more exact than this. Where exactly do you see that? I'm no Windows expert (more of a Unix one), but that's nonsense - there's no reason for "paged out" pages to be showing as the virtual memory - the former is a level higher than the latter. It would make sense with pagefile disabled, though, as Windows has to somehow simulate the paging API even when there's no paging file. Eh, what? I am also tired of people, who pretend to know low level memory signalling and try to somehow tie it to used memory as shown by the OS. Do you even know how page tables work on x86? Besides, even if the high memory usage is real, there's probably nothing BI can do about it - it's the Windows kernel doing its own thing. Who knows what that extra usage is? It might be page cache, eg. essentially free memory used for write-through caching disk contents (IIRC Windows calls this SuperFetch) and MSI counting that incorrectly as "used". True, it might be ARMA textures loaded "just in case" in the background. The point is - you need a pretty reliable reproducer, a proof that what MSI shows is a correct value, find out what the usage actually is and prove that it has a negative effect on anything, .. before developers start investigating your issue, I imagine.

Thanks for the replies, I thought about overriding the UI element, but that would be incompatible with some mods that simply override the UI, without additional syncing logic (ie. ACE3). I could definitely make a background loop what would sleep a lot and be virtually nonexistent performance-wise, to re-add the markers without _USER_DEFINED - again, my use case is not to prevent all deletions, just ones that would allow players to easily communicate over distance and if marker creation is limited to short range, some 1-5secs of enabled deletion for the marker is no big deal. However in the end, I might take the easy way out and just disable the delete button as it (IIRC) cannot be rebound and most mods don't touch it on the map screen (display 12). Thank you.

Hello, is there any way to override the "delete" functionality for player-placed map markers? If not, is there any way to transform player-placed markers to some "global" markers (editor or curator placed) that can't be deleted (though still local, on one client)? I'm trying to make a "guerilla" mission and magical map synchronization over distance is kind of a problem - I can get range-limited marker placement by removing all channels except Direct Communication, but players can still communicate by removing markers (which is synchronized over any distance/channel). Thanks for any tips.

The short answer is - it depends on where you measure it. https://en.wikipedia.org/wiki/Gravitational_acceleration

Honestly, rather than embracing the NIH syndrome even more, let's just use - just this one time - something actually sane. You know - like LUA/Python/Perl/C++/.. Even better, let's make an official C/C++ based API for the engine and build scripting support on top of it, for whatever language one's heart desires - even SQF. The community could even do it! BI wouldn't have to spend development time implementing basic language features (Christ, SQF doesn't even have '+=') and could spend the extra time on something more productive. Oh boy, wouldn't that be great? (Probably not going to happen, though.) :(

Detailed interior (in some form) might be soon possible thanks to https://community.bistudio.com/wiki/Geometric_Occluders .

Indeed, it's been known for some time that cheap Pentium dualcores are great for singlethreaded performance if you overclock them. They don't have huge amounts of cache, but have the price advantage. (Also, games still cannot use the integrated GPU in addition to a dedicated one, so you don't benefit from i3/i5/i7 GPU.) On the other hand, the AMD Bulldozer (and Piledriver) has its share of (mainly cache) problems, significantly affecting single-threaded performance, so it cannot really compete with Intel in that area at the moment. It will be interesting to see what Zen brings along in 2016 Q4 - wouldn't be the first time for AMD to give Intel a run for the money, but we'll have to see.

A piece of advice to existing programers considering SQF: _a = 1; { _a = 2; } forEach [1]; hint str _a; // 2 _a = 1; { _a = 2; }; hint str _a; // 1 And a lot more of things like this. Think twice.

Here's my measurement, using mostly High options, view/object distance set to 12000, measured using Microsoft's built-in perfmon.msc. In both of these, red = read/written pages per second (unit = 1 page) blue = free memory (unit = 10^-7 bytes, eg. 800 = 8GB) green = page file usage (unit = 10^2 %, eg. 100 = 10%) First an interesting sample of me just loading in the world, after previously quitting the game (so that there's tiny remainder of something in the paging file): Notice that Arma3 indeed eats up 2.75GB, but that's likely not the actual process itself, looking at allocated && used virtual memory for arma3.exe, I can see 1679MB, likely at the limit of its memory space. The extra RAM is probably allocated using the paging file API, but Windows is smart enough to not actually page it if not necessary. Also note what's happening after the world load - it seems that Windows is making use of the "free time" by slowly (256 4K pages / second) freeing up the memory, moving it to the file, up to about 10% of the 8GB paging file (800MB). The second sample shows me going back to the menu and selecting a Zeus 4+1 Altis mission (as I didn't have a zeus mission at hand), waiting for it to settle (red line goes down), and flying around (red line up again), high and low, as much as I could. No noticeable free memory decrease or paging file growth. Actually, it seems that flying near the ground in heavily object-populated areas (low fps) helps both memory and the paging file. As such, I cannot confirm any high memory usage on High settings. Maybe Ultra makes the extra 13GB difference?? Or maybe MSI AfterBurner doesn't measure correctly?

The "update failed" errors have been popping up for me recently as well, with ACE3 dlls being blocked. The only way to "fix" this I managed to make up is:make sure arma3 (or arma3battleye) and beservice is not running (task manager) delete C:\Program Files (x86)\Common Files\BattlEye delete C:\Users\YOURUSER\AppData\Local\Arma 3\BattlEye delete/backup the contents of steamgames\steamapps\common\Arma 3\BattlEye download all 4 binaries from http://www.battleye.com/downloads/under Arma 3 place them in steamgames\steamapps\common\Arma 3\BattlEye start the vanilla arma3 launcher, make sure BattleEye is checked (enabled), click Play the game will take a while to start, easily ~5 minutes, let it start (watch the task manager for arma3battleye.exe if you want) after it loads the main ingame menu, quit it start (modded) game via arma3sync (Obviously, the last 2 steps are specific to arma3sync, configured with with arma3battleye.exe as the game binary.) This fixed the dll blacklist for me, at least it seems to. Parts of this list can be found on the official battleye website.

Hey Taosenai, is this mod still in active development? I would like to request the paper map to be foldable only on certain edges instead of centering on the player position - IOW it would show the "fold segment" the player is in and re-fold when the player moves to a different segment, but the fold offsets would be relative to the (0,0) corner of the big map instead of the player position. (Like real-life folding maps with pre-made folds.) This would also significantly reduce the GPS-like nature of the paper map. Alternatively, don't rely on the player position at all and make the player shift folds left/right/up/down to completely detach it from the player position. I was looking at the code as the idea seems rather simple, but I see you have different map scaling (?) for each map and the 100m/1km squares wouldn't probably align correctly to the folds if I did it the naive way. Thanks.

This. AMD Bulldozer/Excavator CPUs aren't as bad as people say if you really manage to use all cores (such as on a server) - we use Opterons next to Xeons and they're still very capable. However most games are few-threaded, so ... That being said, don't trash away AMD just yet, we'll see what the Zen architecture can do about single-threaded performance. That was just a scam - the AMD CPUs have 8 integer units, just 4 FPUs. Historically, "N cores" usually means "N INT units", so really nothing more than a guy/company trying to get some compensation money out of another company. That's a bit misleading. First, you have DMA mapped addresses, which take a significant portion of the 4G space. Then there are global pages and while you could not mark them as global (so-called 4G/4G split we used to have on enterprise Linux in ~2003), it would result in a TLB cache flush of those addresses whenever you wrote to the CR3 register, ie. on a context switch. In other words, up to 30% (IIRC) performance loss. Since you map the same pages to the virtual 32bit space on a 64bit OS, the impact would be similar. (Or am I missing something?) I'm not aware of Windows ever having that feature though, the 2G/2G (or 3G/1G) split is in effect to my knowledge. Actually, the worst possible thing you might want is the load having spread evenly amongst all the cores if all you're doing is just selecting which core the next chunk of data is going to be processed, without any parallelism - this implies significant cache misses as well as context switch overhead. So unless the work is parallelized, you want to keep it localized to a single CPU core as much as possible - it's why "cpu pinning" exists, even though it often shouldn't be needed as CPU schedulers should have sufficient heuristics for the most part. Ah, that would explain the green triangle crashes I'm getting sometimes - I run with 16G RAM with the paging file disabled. Oh developers, always trying to get around the need to port their apps to 64bit. :)

Guys, remember that you can configure the 1.54 stamina system however you like it. It's still far from ideal, but it's at least usable. Here's an example I've written for my community (CNTO): class CfgPatches { class CNTO_Stamina { units[] = {}; weapons[] = {}; requiredVersion = 1.54; /* new stamina system needed */ /* since we want to override maxSoldierLoad defined by ACE3, * we need to load after it, hence the dependency */ requiredAddons[] = {"ace_movement"}; /* if you wish to run this stamina tweak without ACE3, comment * out the above requiredAddons and uncomment the following */ //requiredAddons[] = {"A3_Modules_F"}; }; }; class CfgMovesFatigue { /* make sway recovery faster when changing stances, offset by weight */ aimPrecisionSpeedCoef = 20; //default 5 /* disable sprinting for this amount of secs when stamina runs out */ staminaCooldown = 5; //default 10 /* "amount" of stamina, how fast it drains (higher = slower) */ staminaDuration = 90; //default 60 /* restore stamina from 0 to full in this amount of secs */ staminaRestoration = 90; //default 30 /* when terrain gradient prevents sprinting (hill icon), * add this value to stamina gain */ terrainDrainSprint = -0.6; //default -1 /* when terrain gradient forces you to walk (steep hill), * add this value to stamina gain (warning: too low negative values * actually restore stamina due to the character walking) */ terrainDrainRun = -1.4; //default -1 }; class CfgInventoryGlobalVariable { /* how much a soldier can carry (weight bar width) */ maxSoldierLoad = 1400; //default 1000, ACE 1200 }; (some references: Stamina wiki page, CfgMovesFatigue)Put it in a file called config.cpp, pack it in a .pbo (ie. using cpbo), put that .pbo inside a new Addons directory and create mod.cpp alongside it, containing something like name = "CNTO Stamina System Tweaks"; author = "Freghar"; hideName = 0; overview = "Various tweaks of the vanilla Stamina system, to make it more bearable."; Finally, put both mod.cpp and Addons inside another directory and call it ie. @cnto_stamina. Or really whatever name you want.Since my community uses the ACE3 mod, which already modifies maxSoldierLoad, I had to put a dependency on ace_movement to override that definition - if you don't use ACE3, simply use the other requiredAddons line. Please do customize those values, mine are just an example of a sort-of-eyeballed-1.52 and some other coefficients might suit you better. This configuration tweak doesn't make the stamina system any good, but IMHO it makes it .. umm .. less broken. To keep huge weapon sway at bay, use loadouts with "weight bar" below 50% - sway is much more affected by carry weight rather than stamina. Alternatively, look at CfgImprecision.