brightcandle

Reading through the siterep I come across something that kind of irks me in regards to performance. Not only has the profile build now disappeared and all we again have is development stable but we are soon going to get a boolean switch so they can tell if a game is modded or not. Its not which mods are running, nor is it tooling to tell us which mods are costing time per frame, no its a simple switch of vanilla or not. They seem to be setting up the blame game and pointing at mods rather than actually a) helping people improve the performance of the mods and b) actually addressing the problems of performance generally. This massive delay and this feature as well as the type of testing they are doing tells us a lot about what they think the problem is, they think its all the modders fault. Yet I can open vanilla and go to altis and get the game into the 20's on a high end machine, I don't understand why they don't just do that test themselves. I would say keep up the pressure but they haven't done anything about it yet, and right now all the evidence is they are going in the opposite direction of "its all fine its you and your modded games" without actually helping fix the darn issues.

Maybe, it depends on the reason why the rendering process is taking a long time. It might be because of the API but it could also be BIS code setting up the calls and actually the game isn't really running up against DX limits, from the information we have and the cryptic names of the calls its hard to tell. The simulation presumably is entirely their code and model but that could also at the bottom of things turn out not to be the case. I can't for example tell if its the scripting engine that takes the time or something else, its obscured to an extent to make such analysis hard.

Yes but in software its always faster to take the battle hardened software you have and evolve it into what you need. As much as adding concurrency to the existing engine will be hard, rewriting an engine and game that took over a decade to build will be harder.

If the community was bigger it might support a wider set of paid community projects, but they also need to be projects people would pay for. I haven't yet seen one I wanted that suited our style of play so its of no surprise I haven't donated. The Arma community certainly isn't used to paying for its mods and all that means and its certainly not used to having its community split between the old game and new due to persistent issues in A3 with performance. So I think personally its the wrong environment to be asking for such things, its just not big enough or optimistic enough about the base game to be willing to invest in further content and mods, because they half expect BIS will just break it next week anyway.

The game is basically single threaded by its design (this is the time of a single frame captured, 12 grey lines at the top represent all my cores 6c12t 3930k@4.4Ghz): https://dl.dropboxusercontent.com/u/3638175/Capture%20Palagia.png This is why you see core 0 so high on load and everything is low, the game is basically single threaded except for the bits of rendering that are a bit parallel and I suspect that is the graphics driver or DX doing that and not BIS.

We are right about the game world being the issue. Just today someone posted AI performance tests in the VR world showing 1000 fps. The engine is capable of very high FPS. If you play the game on Cherno you can get 30-60, on altis 20-60 and on some of the other object dense maps (like Bornholm) you can be down in the teens or below. Simply put its object count or its technical equivalent of draw calls that is dominating the performance from the rendering front. DirectX is intrinsically single threaded and there is just a limit on how much you can do, which Arma 3 exceeds or puts a lot of overhead around. The other big time sink per frame is the simulation and here you are right there needs to be a great deal of multi threading as its both quite a lot of the frame time (I have seen 25ms) and its single threaded, but its also the bit that is divorced from the rendering engine. I think people think that Arma 3 ought to run better for how it looks but I think its the simulation and complexity of the worlds dominating here and it underutilises the GPUs as a result, if you play on the Arma 2 ported maps the experience is a lot better because its +10-15 fps and that is just low complexity all around. If you play on the smaller less complicated maps it runs a lot lot better. I agree that the game needs engine upgrades, the rendering needs world as does the scripting and the core underlying simulation all needs to be panellised at the very least. But a new engine isn't something you can just stick into Arma 4 and expect results, because the game has so many unique engine elements. Its just more complicated than getting Cry engine 3 and ramming it in there in a patch and going. Arma has its own unique engine for a reason, because no other games do it remotely like they do. They don't need to start again they need to focus on fixing the very real client side issues. (Recommend people look at my profile capture https://dl.dropboxusercontent.com/u/3638175/Capture%20Palagia.png to understand why I say these are the issues).

I don't think its currently all that obvious without a lot of work what you need to do to get the various AI mods working together and what potential bad interactions they might have. We run MCC (gaia) with ASR_AI and TPWCAS and that seems to give a reasonable experience but its all designed to work reasonable well together. I would like to try some of the others but I am not going to drop MCC as its how I make most missions so really compatibility with that is super important to me and since none of these projects directly talk about interoperability I kind of assume they aren't. So the question is what do I need to do to get AISS working with MCC and is that even possible?

This whole problem is due to steam, I am not kidding. Back in the Arma 2 days we applied patches manually. That meant when putting together a modded game we could choose the base version of the game we were using, then make mods for it and use mods designed for it. If BIS released a new patch or beta then we could test our mods with it, if it worked then all was well, if it didn't then we could continue playing the old version until we had repaired the problems in the mods. It might take 5 minutes it might take 5 weeks, it didn't matter because we could keep playing. But Steam makes everything worse because it just auto updates your game. So the day BIS pushes out an update a number of mods and dependencies of your mods break. So people start fixing it as soon as they can and the fixes start coming in at different paces, some of them very quickly, others take a lot longer. With 20+ mods this takes a while to filter down to the top mods and in the meantime random things are just broken and you don't have a choice. A tool like Play with six makes this worse for also assuming you want all the latest versions, so you have to tell everyone to roll back when the mod breaks something. They only hold onto 2-3 old versions and then its gone and if all the new versions still have the same broken thing introduced you have to live with it or move to a custom repository. But you can't reproduce a reliable dependable modset anymore because of Steam. Even if you fix all your mods in place and test all your updates to make sure they basically work you still get the base game just pushing whatever updates it wants. So while I see why people are requesting that BIS give a heads up in reality that isn't going to fix the problem, the issue is about dependencies. Mods depend on versions of games and other mods, "the latest" of everything doesn't always work together and since we can't version everything properly its all in a constant state of different types of broken. Steam just doesn't support modding properly, it assumes the publisher wont break them which obviously isn't how BIS and many others operate. The situation is broken and the core underlying issue of dependency needs to be addressed to resolve the problem.

I must be the only person that would prefer a series of smaller maps rather than a big one. The bigger it is the slower it will run and Arma 3 already has severe performance problems. Most AOs don't need to be more than a few kilometres so maps designed around a few specific interesting locations are more useful as they run better verses a massive map with lots of areas but where when you only operate in a small part of the map you seem to pay in frames for all the rest you aren't using. I don't want them too small but really big maps aren't sensible considering how poorly Arma 3 runs with them.

The community has been complaining about this as the number 1 issue since the first day alpha was released. Its been the number 1 issue with the game for nearly 2 years now. A petition isn't going to change a thing, they know we care they just don't themselves find fixing it economic. One of the problems here is that the assets of the game are likely too complicated. There is too much stuff in the game world for DX11 to actually handle, there are more draw calls than a modern CPU can handle with DX11. BIS might be able to crush some draw calls together or reduce their CPU overhead but it looks like its just good old fashioned limits on the number of things you can actually render due to CPU overheads. So it needs DX12 or a downgrade of the number of assets that can be seen at any one time and since DX12 isn't available yet to fix this problem likely means a significant reduction in the games image quality, it needs half or less objects. The simulation is single threaded due to the immense difficulty of multi threading it. BIS has spoken before about a way of panellising but its so far the project has brought us nothing tangible and I don't know if they are still even pursing it. It was meant to arrive with Arma 3 (because Arma 2 had the same problems) but obviously it didn't. This is hard, not everything can just be run concurrently, its not easy to convert serial code and many algorithms can't be run on all cores anyway because its mathematically impossible. This is not something any games company can do as a quick patch to a game. Its something they might try to do before the next one but its an immense job that no company does it to an existing product in its twilight years. The problems are fundamental. Without a cutting back of the complexity and number of assets in the game at any one time (world, units, vehicles etc) and without a massive change in the simulation aspect of the game you aren't going to see a change in performance, and indeed its no surprise that the efforts we have seen BIS taking have provided no relief, as its not a series of little minor issues its 2 really big ones. They are trying to give us pounds by saving pennies and its not working. I would bet money on the fact this wont be fixed in Arma 3's lifetime at all.

People like to say lack of optimisation a lot, but as a programmer I can tell you its a not exactly a well defined term. What I have done is said what sort of optimisation is required and why the game is running poorly. Lack of optimisation just isn't a useful term, especially when in the case of Arma 3 much of the problem looks to me to be with the number of draw calls they are making and hence its partly that they made the models and world with too much in it. Maybe they can fix that but they probably can't, that isn't really poor optimisation (in the sense of algorithmic complexity or not taking shortcuts for rendering that make little difference) but rather the game is ahead of what DX11 can actually handle. Their simulation isn't multithreaded but then its also a 100% redesign of the core logic of their game to fix that, these aren't optimisation decisions they are design ones and hence I feel are extremely unlikely to be fixable for A3. So I don't agree its a lack of optimisation, that term meaning very little anyway its more fundamental than that.

I took a profile picture in Palagia in Altis a while ago and captured the following from the game to explain the basic issues with the game on a 3930k running at 4.4Ghz: https://dl.dropboxusercontent.com/u/3638175/Capture%20Palagia.png (2808 kB) Each of the bars at the top represents a core (12 in my case since I have a 3930k) over the time of one frame. Ignoring the text at the bottom its pretty obvious that the game is mostly single threaded except at some points in the rendering where it can go very parallel, but performance tests show that its of minimal impact so despite the moments of parallelism the multithreading in this game has limited impact in practice. We can see that the game is dominated by simulation time (the time it takes to update the world on user updates and network received updates and run any scripts) and on rendering time on the CPU. That suggests two things to me about the game: 1) The simulation needs to be parallel or a lot more performant. Its taking over 9ms in this case to do all the various bits of updating the world and AI and such. 2) The game seems to be heavily dependent on draw calls to the graphics pipeline. Its spending nearly 12ms just doing this and that would suggest there is just too many draw calls and CPU overhead is dominating. In addition sound and physx are all running in the same thread which a lot of games have them run separately which would save a couple of milliseconds as well. I also think that if this is performance at 4.4Ghz on a sandy bridge you aren't going to do much better regardless of your CPU. Worse that that the game seems to be hyperthreading insensitive. We can see it loads core 0 and 1 at the same time and yet they are the same underlying core, and then the rendering process threads seem to run cores next to each other as well while other real cores aren't being used at all. That is just really poor generally and its going to hurt performance. So the game is very heavily dependent on a single CPU core and clockspeed, it also benefits from memory bandwidth but its also fair to say that the issue is mostly the game and the way its written. There is nothing any of us can do. I hope the others have answered your questions so I wont address that but wanted to show you what you are up against to try and get the game running well.

No, for me its been gradually getting worse since the middle of last year. Broken engine is still broken with no fix in sight. Doesn't matter how important it is the player base we'll get a marksman DLC instead of a working game.

We just did a test with two of us and these are the issues we faced: - Reverb off small buildings seems to cut off around 40-45 metres. Its very jarring when it suddenly cuts off. - There is no generic reverb or reverb from the hills and its really noticeable not having that distance bounce of the noise compared to 2.2. - The guns sound very quiet when fired by the other person. At 25metres if they aren't shooting directly at you then without the crack its hard to know they are firing at all. - The grenades when thrown first person sound fine, but to other people on the server they sound muffled, under water and far away. This occurs with RGOs and M203s. - The sonic cracks were a little loud but I see that has been fixed. Certain things were really good like the reverb itself, but the apparent randomness is jarring and the differences from perspective are also problematic. Hope that helps.

Performance wise my feeling is that smaller is better at this point, altis is just too large and cluttered for its own good and runs poorly. Bigger is not better, I would rather run 4 maps of 1/4 of the size than one big map most of the time.

I remember that patch, the game was 60-90 fps for me with a pair of 680's and it was the best performing week of Arma 3 I have ever played. Its not happened since however, its got a little better or a little worse with each patch until now the range is 25-60 or on a big game 15-30. Would love to have the magic that was that beta patch back frankly, the game was a lot better running smoothly.

Lets be honest about the Performance shall we because 200 AI unit its not my problem. My game is typically: 6 players 30 AI 1-5 vehicles on Altis Performance result on a 3930k with 970 = 20-35 fps. The only time we see 60 is when we aren't anywhere interesting. If you go near a town or a forest it drops. You don't even need anything in the map for this to be the case. I was concerned that maybe our mods were causing the issue, so I launched the game vanilla and went to the same places with nothing there and sure enough performance tanked. GPU usage is below 20%, one CPU core is maxed out and the frame rate was awful. The game has a problem, a really serious one. 30 fps is bad in this game because its an FPS, it needs a minimum of 45 really and because most people don't have gsync like I do 60 is what it needs. 20 just isn't good enough and you can see it just walking around altias in the editor on your own without even playing multiplayer. If you go play altis life or wasteland or any of the other multiplayer modes with 60 players you can get down into the 10-15 fps range on a regular basis. That isn't playable, I am astonished people are still around playing the game at this, its a slide show. I am not complaining about 200 AI and 60 player performance, I am complaining about 1 player on the editor performance. The game needs to be fixed asap.

BI has serious quality control problems with its games. The main one in Arma 3 being performance. Its been a problem since the first day of the Alpha and it remains the games number 1 problem. As far as I can tell from the profiler builds its entirely due to CPU performance problems, the game is single threaded and the simulation takes a long time, regularly over 20ms. Its not just that, most new features they introduce I dread, because they are likely going to be bad to begin with. That just speaks to the quality control issues they have generally, but also how long it takes them to fix it. As a company they have some good qualities, they are aligned with their players on some aspects. But they certainly aren't aligned on all of them, DRM being IMO a blight on their record of normally pro customer actions. The recent microtransactions and nagging is also a bitter pill to swallow from them after years of great behaviour. I would put them as a company about in the middle of the evil companies scale. They aren't Ubisoft but they also aren't perfect either and their issues go directly to game playability and thus drive a lot of future customers away.

I view it as a mixed bag. On the one hand mod support and a game that gets continuous updates is great. But at the same time because they are a small studio problems persist for very long periods of time, performance is a real issue with BI games as is quality control and these are just as important as mod support and updated features. Their community support is also very mixed, they mostly do their own thing regardless of what the community asks them to do. So we find a lot of problems they introduced are just worked around with mods. Then there is the DRM. BI still likes to use DRM in its games, online checks and even limited installs for Arma 2. This is distinctly anti consumer. With Arma 3 we now have in game nagging for DLCs. We have more than just poor quality textures we have helicopters and cars we can't drive and which nag us to buy. Its not exactly good. So BI has some things right, they are after all a PC focussed developer and this game genre just wouldn't work without the myriad of control inputs available on the PC, so we aren't getting console shovel wear from them. But its hard to ignore the quality control issues and DRM slant from the company. They aren't Ubisoft or EA in terms of poor treatment of their customers, but if we feel these are the best then the industry is sicker than I think it actually is. They are hovering in the middle on balance, slowly sliding down the scales with every release since OFP, its just the big publishers went down faster.

There are plenty of things we fix with mods, what we can't fix is the performance. Until that gets fixed this game will forever be annoying. Personally I think it ought to target 60fps average and never below 30fps. Right now its more average 35 fps dropping to low 20's with small games and that just isn't good enough.

A long long time ago my community and I produced profiler traces of multiplayer games and showed two types of serious CPU limitations in the game that contributed to performance problems. We had builds that were enabled with diag_captureFrame that allowed us to capture the profiles that the developers were asking us for and we captured a lot of data (http://forums.bistudio.com/showthread.php?169484-What-I-found-about-performance-and-I-what-I-want-to-help-fix-it). Having captured the data and provided it as request we haven't heard a thing since. Performance hasn't changed either, if anything its gotten worse over time. Given that people are still concerned about performance and given what we determined before I propose once again we look into getting profiles from real multiplayer games with highly modded gameplay so that you can look into the major contributors to poor performance and do something about. I can't say I am entirely happy with the radio silence last time so a condition of us helping again would be to be told what issues you find and the list of bugs/features you have added so we can track as and when they get improved. But neitherless I know after we did this last time there was a notable jump in performance and I think its time to do it again now that performance has become unacceptable again. We need you to provide a diag_captureFrame call in the development branch again unless you now have a better approach. Its a pain for us to do these, takes a lot of effort to get everyone to download and install development branch and test all the mods and such but I want my Arma 3 game to run at the very least above 30 and mostly 60 fps and I am not finding 45 at mission start and 25 in mission to be acceptable where not more than a few months ago that was 90 and 50. So please let me know when we can do this exercise again.

Not to mention the gun sway that even the average 30 something airsofter can dramatically outperform. Then there is the recoil pattern which seems to match something a 5 year old would achieve and not a trained rifleman. Heck even I can control the recoil of an m4 in real life better than the dude in the game by a considerable margin. This is why we need mods. They aren't mostly about adding stuff to the game, its about fixing all the stupid stuff BIS does in the name of "realism", which are the exact things that make it more game like. Its even stupid stuff that COD does like making suppressors do less damage and reduce the velocity of the rounds, that isn't even remotely reasonable. In the real world they marginally increase velocity and have no bearing on the accuracy or damage of the gun. Instead what they do is get hot and eventually they fail unless they get time to cool down. They also get in the way of doors making it harder to corner tightly in CQB. But BIS decided not to model any of that so instead they just throw in the COD mechanic of balancing it with damage. What makes this game realistic is a tonne of mods fixing all these bad calls. BIS doesn't make the game the community plays, the community fixes all this stuff every release (Arma, Arma 2 and now Arma 3). We do the best we can to improve performance and we do it all with mostly just hindrance from BIS.

Its not just about their scenarios, its about real world games. The problem has always been that performance in some particular circumstance can be made fine but real games use mods, scripts and different combinations of AI. Simply put the results of the program to improve performance using BIS servers has netted zero benefit for most groups out there. They can improve a few public games running limited scenarios but not real missions. Its simply not sufficient what they are doing and having done it for over a year the benefits have been nonexistent. Its not like this testing just started, it was happening during the beta. Frame rates were actually higher then than they are now. BIS do this to show they are doing something about performance when they actually have no intention to fix it. Its just a ruse.

Just bare in mind they have been doing this exercise for well over a year with no real results. We provided detailed analysis of a 35 player game that had fps dropping into the 20 fps range and they didn't do a thing with that information, not a thing. We try and arrange getting captureFrame reintroduced into development so we can do it again and get no response. These servers are there to give the impression that they are willing to do something about performance, not to actually do something about it. When we first were showing performance problems in Alpha it was apparent they appeared in single player as well. You didn't need multiplayer to break the frame rate, the single player showed some of the worst performance possible in the game. Yet there they were trying to setup servers and do this when if they just played the single player they could see the poor frame rate for themselves reliably and repeatable. So I am sorry that people don't get this yet, but its simply untrue that BIS has any interest in fixing the issues with this game. They just aren't ever going to do so. The evidence is that after a year and a half performance is basically the same as it was on the very first release, there is minimal improvements. They have ignored community input and failed to give us the tools to fix the script performance that could be contributing. In the end its a waste of time going onto these servers and its a waste of time trying to contact BIS about the issues. They couldn't care less, they already got your money.

So officially then BIS has stopped trying to improve performance then. They aren't willing to do anything to find the problems and fix them as is clearly apparent. Offer is now withdrawn.