nkenny 1057 Posted February 8, 2019 The effect of weapon on AI accuracy by nkennyAbstract* Testing the effect of weapon on AI accuracy. Introduction The weapon an AI soldier is equipped with greatly affects killing power. Even when the class of weapon is nominally the same, weapon configuration plays enormous part. This is especially evident when similar weapons from different mods are used together. To explore the effect of weapon on AI killing power I built an experimental suite. Experiment The test was conducted while running only CBA, CUP Weapons, RHS Russian Armed Forces and NIArms. The player profile was newly created and no AI skill tweaks were made. Description I placed a single shooter on flat terrain, back turned towards a four man group of enemies. The four man team is concealed from the shooter (his back is turned). Each soldier is identically equipped with no body armour or helmet or weapon. After five seconds the shooter is given a weapon and given perfect information about the enemy. This results in him turning and engaging the enemy. The targets are spaced at two meter intervals and the test was run on a 100m and 200m range (normal combat engagement ranges). Shooter and targets were forced to remain static and remain in a standing position. My point of comparison was the AKM rifle. Ubiquitous in enemy hands and mod packs alike. I provide data for the modern MX rifle with a HAMR RCO as a frame of reference. The exercise is over once three minutes has passed, the shooter has expended all ammunition, or all targets are dead. Measurements are printed on screen and recorded on the clipboard. I paste that information into a document. Measurements I ran the exercise six times with each weapon. While not enough for statistical evaluation, I nonetheless deemed it sufficient to get a sense of the performance characteristics of each weapon. - Hits, each bullet that impacts a target. - Shots fired, maximum is 6 magazines: 180 rounds. - Accuracy, hits divided by shots fired. - Time, time in minutes and seconds. Record After each round I recorded weapon class name, Accuracy, (hits / shots fired) and time. Each measurement is averaged (in red). Expectations Mod configuration will vary somewhat. With CUP weapons being more accurate than RHS counterparts. Data @ 100 Meters Spoiler // 100M Vanilla wpn: arifle_AKM_F - acc: 50 percent (5/10) - time: 00:05 wpn: arifle_AKM_F - acc: 57 percent (4/7) - time: 00:04 wpn: arifle_AKM_F - acc: 57 percent (4/7) - time: 00:05 wpn: arifle_AKM_F - acc: 46 percent (6/13) - time: 00:07 wpn: arifle_AKM_F - acc: 38 percent (5/13) - time: 00:07 wpn: arifle_AKM_F - acc: 61 percent (8/13) - time: 00:07 51% accuracy -- 10.5 shots -- 6 seconds// 100M CUP wpn: CUP_arifle_AKM - acc: 50 percent (4/8) - time: 00:07 wpn: CUP_arifle_AKM - acc: 38 percent (5/13) - time: 00:10 wpn: CUP_arifle_AKM - acc: 50 percent (4/8) - time: 00:06 wpn: CUP_arifle_AKM - acc: 50 percent (4/8) - time: 00:07 wpn: CUP_arifle_AKM - acc: 30 percent (4/13) - time: 00:09 wpn: CUP_arifle_AKM - acc: 41 percent (5/12) - time: 00:08 43% accuracy -- 10.3 shots -- 8 seconds // 100M RHS wpn: rhs_weap_akm - acc: 27 percent (6/22) - time: 00:11 wpn: rhs_weap_akm - acc: 19 percent (9/47) - time: 00:28 wpn: rhs_weap_akm - acc: 12 percent (6/48) - time: 00:29 wpn: rhs_weap_akm - acc: 14 percent (7/47) - time: 00:29 wpn: rhs_weap_akm - acc: 25 percent (10/40) - time: 00:26 wpn: rhs_weap_akm - acc: 12 percent (4/32) - time: 00:21 18% accuracy -- 39.3 shots -- 24 seconds// 100M NIArms wpn: hlc_rifle_akm - acc: 40 percent (4/10) - time: 00:06 wpn: hlc_rifle_akm - acc: 16 percent (4/24) - time: 00:09 wpn: hlc_rifle_akm - acc: 40 percent (4/10) - time: 00:06 wpn: hlc_rifle_akm - acc: 50 percent (4/8) - time: 00:05 wpn: hlc_rifle_akm - acc: 36 percent (4/11) - time: 00:05 wpn: hlc_rifle_akm - acc: 23 percent (4/17) - time: 00:08 34% accuracy -- 13.3 shots -- 6.5 seconds Data @ 200 Meters Spoiler // 200M VANILLA wpn: arifle_AKM_F - acc: 9 percent (4/41) - time: 00:47 wpn: arifle_AKM_F - acc: 27 percent (6/22) - time: 00:23 wpn: arifle_AKM_F - acc: 15 percent (7/45) - time: 00:49 wpn: arifle_AKM_F - acc: 14 percent (8/56) - time: 00:59 wpn: arifle_AKM_F - acc: 14 percent (4/27) - time: 00:28 wpn: arifle_AKM_F - acc: 10 percent (7/69) - time: 01:17 15% accuracy -- 13.3 shots -- 47 seconds// 200M CUP wpn: CUP_arifle_AKM - acc: 13 percent (4/30) - time: 00:31 wpn: CUP_arifle_AKM - acc: 33 percent (4/12) - time: 00:15 wpn: CUP_arifle_AKM - acc: 50 percent (4/8) - time: 00:09 wpn: CUP_arifle_AKM - acc: 23 percent (5/21) - time: 00:22 wpn: CUP_arifle_AKM - acc: 18 percent (4/22) - time: 00:25 wpn: CUP_arifle_AKM - acc: 28 percent (7/25) - time: 00:26 27% accuracy -- 19.6 shots -- 21 seconds// 200M RHS wpn: rhs_weap_akm - acc: 4 percent (6/131) - time: 01:57 wpn: rhs_weap_akm - acc: 3 percent (5/156) - time: 02:05 wpn: rhs_weap_akm - acc: 8 percent (5/62) - time: 00:50 wpn: rhs_weap_akm - acc: 5 percent (8/158) - time: 01:51 wpn: rhs_weap_akm - acc: 5 percent (7/130) - time: 01:46 wpn: rhs_weap_akm - acc: 2 percent (5/180) - time: 02:36 ** RAN OUT OF AMMO ** 4.5% accuracy -- 136 shots -- 111 seconds// 200M NIArms wpn: hlc_rifle_akm - acc: 60 percent (6/10) - time: 00:12 wpn: hlc_rifle_akm - acc: 30 percent (4/13) - time: 00:15 wpn: hlc_rifle_akm - acc: 38 percent (7/18) - time: 00:22 wpn: hlc_rifle_akm - acc: 66 percent (6/9) - time: 00:12 wpn: hlc_rifle_akm - acc: 71 percent (5/7) - time: 00:09 wpn: hlc_rifle_akm - acc: 77 percent (7/9) - time: 00:12 57% accuracy -- 11 shots -- 13.6 seconds// 200M VANILLA MX with HAMR RCO wpn: arifle_MX_Hamr_pointer_F - acc: 40 percent (4/10) - time: 00:12 wpn: arifle_MX_Hamr_pointer_F - acc: 36 percent (4/11) - time: 00:13 wpn: arifle_MX_Hamr_pointer_F - acc: 42 percent (6/14) - time: 00:14 wpn: arifle_MX_Hamr_pointer_F - acc: 31 percent (5/16) - time: 00:17 wpn: arifle_MX_Hamr_pointer_F - acc: 50 percent (4/8) - time: 00:09 wpn: arifle_MX_Hamr_pointer_F - acc: 70 percent (7/10) - time: 00:11 45% accuracy -- 11.5 shots -- 12.6 seconds Findings @100 Meters At close range the vanilla AKM and CUP AKM are closely matched. With a slight edge to the vanilla gun. RHS weapons are considerably less accurate than all other counterparts and require four times the amount of time and rounds fired for similar effect. The numbers for the NIArms AKM belie their effect on target. Toadie's AK tended to fire in bursts, but with a highly accurate first shot. In effect the gun killed with the first shot, but burned off a few rounds into the air-- wasting time and bullets. @200 Meters At mid range the differences between the weapons become evident. CUP weapons are almost twice as effective as their vanilla counterpart. RHS weapons are considerably less accurate than the others. Which is not to say ineffective. Rate of fire is high and spread provides greater coverage-- area of fire-- than the other counterparts. As for NIArms: the rifle comparatively is laser accurate, out shooting all other alternatives. Contrasted to RHS it provides almost eight times more killing potential. It even compares favourably to the vanilla MX with a optic. Conclusion Not all AKMs are created equal. The killing potential of each gun varies wildly from mod to mod. The most deadly close quarters AK is the vanilla, while in most combat ranges, the NIArms gun outperforms all other alternatives. One obvious take away is that there are radical differences of capability with two presumably identical weapon systems. This is one reason why I would advice consistency in mod sets. Another is that this vast difference in capability need not match the players performance, expectations and experience, with the same weapon system. In the case of the AKM, I believe most players find them very similar-- aside from minor differences in sight picture and sound. In fact to test that very thing I used the same suite to record my own times. I won't post the results here, but they were largely similar across mods. The test itself remains neutral to the intended design intention of the style of configuration each mod adopts. The comparable lack of accuracy of RHS weapons need not be read as a flaw. There are arguments both from the perspective of realism and game play to prefer less accurate weapons. - Firstly, the sort of 'white room' engagement against a non-responsive, non-moving target is hardly applicable. Handling performance on sterile ranges fail to convey the stress of combat shooting. Most studies into the effect of combat on marksmanship show a massive decrease in effective accuracy. RHS weapons simulate combat shooting, rather than target plinking. - Secondly, more shots fired and fewer one-shot-kills, is generally good for game play. At 100 meters vanilla, CUP and NIArms equipped soldiers can eliminate an inattentive fire team in less than 10 seconds. While demanding some caution is good, such a quick end also means a quick end to the session for the player in question. - Thirdly, RHS comes with an entire dimension of weapons. Accuracy comes with modern quality. A RHS M4A1 is more accurate than an AKM. So is a an AK74. See references. Many communities depress AI accuracy values. The considerable accuracy of CUP and NIArms weapons need not be a flaw. Follow the links in the references and one can see the extreme degradation of accuracy based suppression levels. The considerable default accuracy of vanilla weapons, the presence of multiple stances, all are suggestive of a very specific type of gameplay. Soldiers should be cautious and reveal their positions only momentary. Suppressive fire is very effective. It is more important to suppress all dangerous enemies than it is to pursue time consuming killing shots! When the AI is accused of laser accuracy, it will in some cases be due to failure to adequately suppress-- through fire superiority-- the enemy position. Finally, I suspect, particularly in regards to the user made modifications there are considerably differences. Test are done in different AI setting ecologies-- tweaking AI skill is easily (if somewhat obtusely) done. If I shall brave a conjecture I expect CUP weapons, being added more along open source model will show greater variance in configuration. NIArms weapons are developed with an eye towards delivering an uncompromising experience for the shooter. The AI comes much later. All of this means that mod makers and mission designers must pay attention to the source of the weapons given to the AI. Future tests 1. The same platform can easily be used to test other weapons or family of weapons 2. In the future I would like to test ranges out to 300 and 400 meters** References - Original thread on nopryl.no, further numbers, weapons and experiments are here - Experimental mission, links to mission for your own testing -k * Dear mods: Feel free to move this thread. While relevant to mission makers and mod developers equally. It seems sufficiently meta to concern general players, but it is not pointed enough to belong to the mod discussion forum. ** My initial run with standing RHS weapons yielded one hit across 180 rounds. In other words. If faced by a lone, standing gunman armed with a RHS AKM. At 300+ meters you might as well remain stationary and standing yourself. 16 3 Share this post Link to post Share on other sites
froggyluv 2136 Posted February 8, 2019 Fantastic. This kinda testing is just so important - i had no idea different mods had different dispersion for same weapons. That was one of my turnoffs to the game a while back in which low skilled Insurgents using AK's were completely dominating high skilled spec ops (5.56). Really giving no reason not to arm everyone with 7.62 arms which was kinda depressing Share this post Link to post Share on other sites
mmm 35 Posted February 9, 2019 I don't know jack about scripting to build a proper test but I do have an interest in learning what makes bot kill faster/slower. As far as I know even vanilla weapon has some wildly inconsistent configuration for AI accuracy. I guess aiDispersionCoef probably has the most significant effect, and they're configured to have such massive gaps between weapon classes(sometimes within nominally the same class) that it totally outweighs base weapon dispersion. Marksman rifles generally have a aiDispersionCoefX/Y value of 2*3, original vanilla rifle of 4*6(4x greater area than markman rifle!), however most APEX DLC addition has a 6*6 for reasons unknown, including SPAR-17 which you might consider a marksman rifle otherwise. Machineguns are on the otherhand in a league of its own with aiDispersionCoef on the order of 20s in both axis, in turn means they have rather limited actual killing potential under vast majority of circumstances. At higher unit skill/AI aim level it starts to equalize across the board as aiDispersionCoef eventually goes to 1 in which case weapons are treated as exactly what they are. Machineguns are still hindered by their burst behavior which reduces the fire rate of actual aimed shots, since only the first shot within a burst has chance to hit(I don't observe any meaningful recoil compensation even at higher skill). But relatively speaking some become quite a bit better like SPMG with impressive dispersion. The information regarding to suppression is new to me but from the interactions between AIs I've seen, all else being equal machineguns will be consistently beat by average rifles in the hands of AI, in a straight up duel type of scenario at least. Whether they could provide any benefit that outweighs the abysmal lethality in a unit is unclear to me. But if I were to play dress up doll with my AI team, I'd favor very heavily on weapons configure as marksman rifle(CMR-76/Mk14 in, SPAR-17/MXM out). Short of that weapons with favorable ballistics in the assault rifle category. Just from gameplay I was kinda under the impression that optics on AI doesn't really meaningfully affect their accuracy/"time to kill", but mostly the range they're willing to engage, and their ability to maintain track on a already detected contact. Will you be able to point to me where in the config does it specify it's effect on AI accuracy? And did you investigate stances? resting(don't believe its a feature AI can make use of)? I'm under the impression they don't really matter to AI anymore. Share this post Link to post Share on other sites
joostsidy 685 Posted February 9, 2019 A tip for the investigator: see if you make graphs from your quantitative data. They are easier to read, you can spot differences immediately, and hopefully draw more people to the discussion. Nobody (or is it just me?) likes reading lists of numbers.. 🙂 1 Share this post Link to post Share on other sites
nkenny 1057 Posted February 9, 2019 @mmm Interesting. I did not delve into the dark arts that is the skill settings of Arma3. Instead I wanted to run a baseline of sorts were ostensibly identical weapons were compared in similar situations. Repeating the experiment with the AI skill set to 0.25, 0.5, 0.75 and 1.0 would perhaps yield interesting data. To respond to some of your questions and input. 1. Suppression dramatically decreases the effective accuracy of the soldier. Follow the references for more numbers. In regards to automatic rifles vs marksman rifles. I ran some tests where I compared a standing AKM shooter to a standing machine gunner. While I did not perform a full experiment, the general trend was that standing at 100-200 meters, assault rifles are more effective. Machine guns will consistently overshoot the target. Of course, this is a white room experiment, and not a duel environment with cover, concealment and multiple stances. I will agree to your other assessment. My limited tests with marksman rifles show that these, particularly paired with optics, are very deadly. 2. The effect of weapon optics on AI is considerable. An MX rifle equipped with a HAMR optic is nearly twice as efficient as a bare bones rifle (Source). As I'm not sat before an Arma capable computer at the moment, I can't point you to the config values. But I do recall there are different shooting (or recoil?) patterns for optics. Let me get back to you on that. 3. I have not checked the stances! It is a very interesting area. Still this is easy to do within the testing suite. It is my belief that weapon resting and bipod use are not considered for AI. But the prone stance in particular is. I did run some tests at 300 meters, unrecorded, but stopped because the RHS AKM was simply unable to score more than a single hit. Setting the unit to prone resulted in two hits. Repeating the test across a wider range of distances and stances is next on my todo list. Share this post Link to post Share on other sites
nkenny 1057 Posted February 9, 2019 (edited) @froggyluv Thanks! I suspected seeing some data would be useful for mission and mod designers and players alike. It isn't as bad as you might think. Within an ecology of vanilla weapons, the AKM is markedly inferior to both the MX and SPAR15 rifles, on all ranges. Add RCOs and the modern rifles pull out ahead. Note also that the AK12 is intrinsically more accurate than the AKM. That said, vanilla weapons are very accurate overall. It may be difficult to distinguished between being one shotted in six seconds rather than four point five... @joostsidy Good point. I will think about it. My reasoning for avoiding it has been this: the sample size is quite small. I therefore wanted to avoid giving the impression that this experiment is suitable for statistical numbers mathemagics. -k Edited February 9, 2019 by nkenny clarity 1 Share this post Link to post Share on other sites
mmm 35 Posted February 9, 2019 @nkenny Well according to THIS aiDispersion will only truly equalize at 1 skill. Lower than that I will assume it will use a multiplier still larger than 1 proportional to the aiDispersion value. In my test to be safe I put the AI aim in difficulty to 1 as well. Also since it's a multiplier to weapon dispersion it informs you more about how weapons are configured for AI use, rather than weapons themselves. Basically the "unfairly" configured weapon catches up at 1 skill because the artificial inaccuracy is removed. One thing I have no mean to properly investigate is recoil compensation for automatic fire. Maybe you can try some tracer effect at different skill level/stances? Edit: With regards to scope, I don't mean to dispute your test result as it is more scientific than I could ever be. And I'd like to believe scopes make some difference to AI accuracy. But I wonder if it is actually in the config, maybe it's pulled number from the optics FoV and hard coded to apply bonus to AI? Again I know nothing about it just spewing things I don't understand. Just don't seem the weapon's "AI firemode" in the cfgweapons has anything indicating an accuracy change. As for stance effect. I could say, from my experience in gameplay, prone definitely HAD an effect in earlier days of ARMA 3. The focused streams of tracer bullets coming your way from an AI automatic riflemen in prone position, watched in slow motion was quite unforgettable. Even assault rifles delivered more rapid semi auto fire from prone. However It may have changed at the time or sometime after the Marksmen update(along with the excessive dispersion multiplier), as of today I don't observe any tangible effect from stances. 1 Share this post Link to post Share on other sites
Defunkt 431 Posted February 10, 2019 Good work. I think the next thing would ideally be to work out exactly what config values influence these variations such that a script might be run on the whole config tree and spit out a large result set (which could be graphed as suggested above). In as much as you've identified a potential 'problem' - this might yield its solution, whether by allowing mod makers to see how their configs compare with others or mission makers to see what weapon sets inter-operate well. Share this post Link to post Share on other sites
SwissArmy1984 8 Posted February 14, 2019 This is a great start. I'm having trouble also with the distance at which AI shoot. Have you tried AI shooting beyond 750 meters, which seems to be a hard stop? AI with sniper rifles, DMRs, and machineguns should easily fire beyond 750 meters, but they do not. I've been shooting AI beyond 750 meters and they never fire back, even if you're standing in plain site. It seems the cfgWeapons files have distance settings. These were modified for a few weapons by the "Duckhunting" mod on Armaholic. But that mod is 3 years old and doesn't seem to work anymore. Share this post Link to post Share on other sites
nkenny 1057 Posted February 16, 2019 AK12 in various stances So I have rerun the original experiment with the vanilla AK12 across a wide variety of parameters, stances and distances. The number one thing demonstrated was the unsuitability of the experimental framework. While adequate for demonstrating categorical differences between ostensibly identical weapons between mods, it has proven unhelpful in showing the nuisance of stance and weapon attachment. The sample size being small, also the main reason I do not provide data in pretty columns and graphs, simply does not present an adequate foundation for consistent comparison. That said, the test still showed some things. Hence this post. Method and measurements The experimental suite is unchanged from what was presented in the first post. To make sense of the data I decided to make a measurement of aggregate potential. The formula I settled on was this: effective kills per second multiplied by time left (180 seconds) This measurement, while not perfect, gives some indication of how the weapon could perform in perfect conditions. The conclusion will cover my findings in more detail. Data @ 100 Meters (TL;DR in conclusion) Spoiler // 100M Standing wpn: arifle_AK12_F - acc: 50 percent (4/8) - time: 00:05 wpn: arifle_AK12_F - acc: 57 percent (4/7) - time: 00:05 wpn: arifle_AK12_F - acc: 40 percent (4/10) - time: 00:06 wpn: arifle_AK12_F - acc: 56 percent (9/16) - time: 00:09 wpn: arifle_AK12_F - acc: 60 percent (6/10) - time: 00:07 wpn: arifle_AK12_F - acc: 62 percent (5/8) - time: 00:04 54% accuracy --10 shots --6 seconds // 100M Crouched wpn: arifle_AK12_F - acc: 66 percent (4/6) - time: 00:04 wpn: arifle_AK12_F - acc: 44 percent (4/9) - time: 00:06 wpn: arifle_AK12_F - acc: 45 percent (5/11) - time: 00:06 wpn: arifle_AK12_F - acc: 80 percent (4/5) - time: 00:03 wpn: arifle_AK12_F - acc: 57 percent (4/7) - time: 00:04 wpn: arifle_AK12_F - acc: 50 percent (4/8) - time: 00:05 57% accuracy --7,5 shots --4,6 seconds // 100M Prone wpn: arifle_AK12_F - acc: 40 percent (4/10) - time: 00:07 wpn: arifle_AK12_F - acc: 44 percent (4/9) - time: 00:05 wpn: arifle_AK12_F - acc: 50 percent (4/8) - time: 00:05 wpn: arifle_AK12_F - acc: 60 percent (6/10) - time: 00:06 wpn: arifle_AK12_F - acc: 45 percent (5/11) - time: 00:07 wpn: arifle_AK12_F - acc: 62 percent (5/8) - time: 00:06 50% accuracy --9 shots --6 seconds Data @ 200 Meters Spoiler // 200M Standing wpn: arifle_AK12_F - acc: 37 percent (6/16) - time: 00:16 wpn: arifle_AK12_F - acc: 55 percent (5/9) - time: 00:10 wpn: arifle_AK12_F - acc: 36 percent (9/25) - time: 00:27 wpn: arifle_AK12_F - acc: 29 percent (7/24) - time: 00:26 wpn: arifle_AK12_F - acc: 33 percent (7/21) - time: 00:22 wpn: arifle_AK12_F - acc: 41 percent (5/12) - time: 00:14 39% accuracy --18 shots --19 seconds // 200M Crouched wpn: arifle_AK12_F - acc: 28 percent (4/14) - time: 00:15 wpn: arifle_AK12_F - acc: 35 percent (5/14) - time: 00:15 wpn: arifle_AK12_F - acc: 25 percent (5/20) - time: 00:22 wpn: arifle_AK12_F - acc: 60 percent (6/10) - time: 00:11 wpn: arifle_AK12_F - acc: 41 percent (5/12) - time: 00:14 wpn: arifle_AK12_F - acc: 40 percent (4/10) - time: 00:11 38% accuracy --13 shots --14,6 seconds // 200M Prone wpn: arifle_AK12_F - acc: 38 percent (5/13) - time: 00:14 wpn: arifle_AK12_F - acc: 54 percent (6/11) - time: 00:12 wpn: arifle_AK12_F - acc: 23 percent (5/21) - time: 00:23 wpn: arifle_AK12_F - acc: 31 percent (5/16) - time: 00:19 wpn: arifle_AK12_F - acc: 21 percent (7/33) - time: 00:38 wpn: arifle_AK12_F - acc: 24 percent (8/33) - time: 00:38 32% accuracy --21 shots --24 seconds Data @ 300 Meters Spoiler // 300M Standing wpn: arifle_AK12_F - acc: 15 percent (6/40) - time: 02:11 wpn: arifle_AK12_F - acc: 23 percent (10/43) - time: 02:17 wpn: arifle_AK12_F - acc: 21 percent (9/42) - time: 02:16 wpn: arifle_AK12_F - acc: 15 percent (4/26) - time: 01:25 wpn: arifle_AK12_F - acc: 17 percent (6/35) - time: 01:53 wpn: arifle_AK12_F - acc: 22 percent (6/27) - time: 01:28 19% accuracy --35.5 shots --115 seconds // 300M Crouched wpn: arifle_AK12_F - acc: 21 percent (7/32) - time: 01:42 wpn: arifle_AK12_F - acc: 23 percent (9/39) - time: 02:07 wpn: arifle_AK12_F - acc: 25 percent (10/40) - time: 02:10 wpn: arifle_AK12_F - acc: 5 percent (3/54) - time: 03:00 * FAILED with 2 kills * wpn: arifle_AK12_F - acc: 13 percent (6/43) - time: 02:22 wpn: arifle_AK12_F - acc: 19 percent (7/36) - time: 01:59 18% accuracy --41 shots --133 seconds // 300M Prone wpn: arifle_AK12_F - acc: 20 percent (7/34) - time: 01:52 wpn: arifle_AK12_F - acc: 24 percent (6/25) - time: 01:23 wpn: arifle_AK12_F - acc: 29 percent (5/17) - time: 00:56 wpn: arifle_AK12_F - acc: 21 percent (9/42) - time: 02:18 wpn: arifle_AK12_F - acc: 29 percent (5/17) - time: 00:58 wpn: arifle_AK12_F - acc: 18 percent (9/49) - time: 02:38 23% accuracy --31 shots --101 seconds Data @ 300 Meters w Optics & Attachments Spoiler // 300M Prone + bipod -- #bipod was not deployed wpn: arifle_AK12_F - acc: 18 percent (6/32) - time: 01:44 wpn: arifle_AK12_F - acc: 17 percent (6/35) - time: 01:54 wpn: arifle_AK12_F - acc: 36 percent (7/19) - time: 01:01 wpn: arifle_AK12_F - acc: 23 percent (6/26) - time: 01:24 wpn: arifle_AK12_F - acc: 31 percent (7/22) - time: 01:11 wpn: arifle_AK12_F - acc: 25 percent (8/32) - time: 01:46 25% accuracy --28 shots --92 seconds // 300M Prone + HAMR (opticType = 1) wpn: arifle_AK12_F - acc: 41 percent (5/12) - time: 00:38 wpn: arifle_AK12_F - acc: 11 percent (5/43) - time: 02:19 wpn: arifle_AK12_F - acc: 19 percent (9/47) - time: 02:36 wpn: arifle_AK12_F - acc: 15 percent (7/45) - time: 02:26 wpn: arifle_AK12_F - acc: 19 percent (9/47) - time: 02:35 wpn: arifle_AK12_F - acc: 17 percent (7/41) - time: 02:14 20% accuracy --39 shots --128 seconds // 300M prone + optic_LRPS (opticType = 2) wpn: arifle_AK12_F - acc: 16 percent (6/37) - time: 01:57 wpn: arifle_AK12_F - acc: 30 percent (10/33) - time: 01:45 wpn: arifle_AK12_F - acc: 15 percent (5/33) - time: 01:47 wpn: arifle_AK12_F - acc: 23 percent (5/21) - time: 01:07 wpn: arifle_AK12_F - acc: 46 percent (6/13) - time: 00:42 wpn: arifle_AK12_F - acc: 30 percent (4/13) - time: 00:43 27% accuracy --25 shots --80 seconds Findings The effect on stance on accuracy is slight, but present. In particular around the 300 meters mark the effect is more pronounced. The nature of the experiment introduces an unintended penalty to stance modifications. because of the way weapons are added and that the soldier begins back turned to the enemy, the initial shot (sometimes two shots) fired from the crouched or prone stance will miss due to animation changes. There are two categories of optics, type 1 and 2, and these have an effect on weapon performance. The effect may not however always be beneficial, as in the case of the AK12 adding a HAMR (type 1) optic will at 300 meters negatively affect performance. A bipod has no relevant effect on AI accuracy, and when fired from a prone position. The bipod is not engaged. At 300 meters the AK12 rate of fire is distinctly slower than at other ranges. At all tested ranges the rifle fires in a semi-auto manner. Conclusion There is a small effect on AI weapon accuracy based on stance, but generally only between standing and prone. At closer ranges, with small margins, changing stance or other animation interactions play a far greater effect on accuracy loss. An AI that frequently shifts between crouched, prone or standing will be considerably less deadly. I was surprised to find that the HAMR optic did not increase accuracy at 300 meters. When equipped on a 5.56 or MX rifle there was a clear improvement in accuracy at 200 meters. It is worth noting that the LRPS, a type-2 optic config wise, did see some accuracy improvements. To answer @mmm, I suspect the reason is that optics alter the ranges where the weapon is consdiered to be short, mid or far ranges. Perhaps there is some intrinsic accuracy loss when the weapon is fired at further ranges. The AK12 fares poorly, mostly for the accident of precise ranges, accuracy values and bullet damage. The biggest takeaway is of course the inadequacy of the testing suite. It was designed to compare weapons across mods, not the precise nuances between weapons and stances. A new, more easily scalable suite would need to be constructed. Preferably one which can develop numbers in a quicker and more statistically significant manner. Abstract potential As mentioned in the introduction, to make sense of the numbers I created a sort of abstract potential. This created a nice string of numbers, but I must stress that the small sample size and considerable variety in test results (a single good set can skew the numbers quite considerably!), means that these should be viewed as guidelines and not final. (Higher is better) Ak12 standing 100M: 95 Ak12 crouched 100M: 101 Ak12 prone 100M: 79 Ak12 standing 200M: 68 Ak12 crouched 200M: 64 Ak12 prone 200M: 51 Ak12 standing 300M: 12 Ak12 crouched 300M: 10 Ak12 prone 300M: 20 Ak12 prone 300M with #bipod : 24 Ak12 prone 300M with HAMR : 14 Ak12 prone 300M with LRPS : 29 From this it can be read that there is no real difference between standing and crouched stances. Prone initially fares poorly, because of the aforementioned time spent changing stance, but comes out slightly ahead as distances being to increase. The numbers are however so close that no real firm result can be read. Overall, this test is best viewed as a failure with some interesting implications and lessons learned. Future improvements - Run the test with set AI skill 1 - Run the test in a manner which produces more numbers (and faster) - Create a better aggregate number which can be used to compare weapons 2 Share this post Link to post Share on other sites
mmm 35 Posted February 18, 2019 I dunno... I'll call it inconclusive? This is a good deal of testing done by you already nkenny it''s simply unfair to ask you do another 100 to get closer to statistical average. But with the extra AI dispersion a good deal of randomness is involved, it's totally within what RNG could do to you. I'll stick by my assumption that optics and stances don't have a significant impact on AI accuracy, if any. But arguably your test made a good case for avoiding close combat with AI at all costs... I think gameplay has already allowed us to develop such instincts but you properly quantified it. 1.5s is your life expectancy after AI starts engaging you at 100m(that explained my fear for towns and Tanoa). Given how AI handles detection at such range you're pretty much instantly engaged as soon as you get within line of sight of an AI. 300m range didn't reduce AI's hit probability to exactly 1/9 of 100m, but the time to successfully engage a target is extended by even more than that. Actually funny enough we discussed all the potential factors but none affect AI's ability to kill more than range., well at least for assault rifle. I'd say weapons and their configuration for AI use is another one, and depending on type maybe scaled differently between ranges. 1 Share this post Link to post Share on other sites
mmm 35 Posted March 30, 2019 Looking back I think the suppression test was also quite informative. As far as dealing with AI goes, given that you have identified the position of your attacker(which may not always occur before you die...), you should develop the habit of dumping as much fire power towards the target instead of trying to landing the killing shot with your first few bullet, while dropping to prone to minimized your target size, if possible. I think it's especially important to adopt this behavior when you're caught in unexpected engagement at relatively close range when there's no cover in your immediate vicinity. I'd go as far as saying that dumping your entire 30 round magazine quickly, reload under fire, then try to score hits with the second magazine is better than trying to score hit initially when fighting excessive sway from the weapon inertia caused by your earlier 180° turn. It's not gonna be comfortable to be caught helpless with an empty gun, but the suppression should have mitigated the risk of actually taking a hit and bought you time to steady your aim. 1 Share this post Link to post Share on other sites
nkenny 1057 Posted June 13, 2019 Sorry, I have been busy with other projects and have therefore not followed this thread to the extent I would have wanted. I have in mind a more robust testing scenario which allow collating more numbers in an easier manner. That will happen once I have set up my Arma3 capable computer again. :) @mmm Your comments in regards to suppression match my findings quite precisely. I refer to the data collected in the final posts at LINK. Against a 6.5 Katiba equipped AI, at 200 meters, effective suppressive fire decreases AI accuracy dramatically. Successfully maintaining 20-25% suppression, comparable to near hits by two to three round bursts, changes the life expectancy of a four man fireteam from 25 seconds to 3 minutes-- with not all members killed. This is clearly an intended behaviour. Which is in turn unfortunately marred by Bohemias poor record of player feedback. It is difficult to determine if an AI is in a state of suppression-- no animations, no sounds, no immediately obvious behaviour is reported. Achieving 80 to 100% suppression on an enemy effectively neuters the shooter. As you correctly surmise, responding to AI contacts with aggression is key. It translates into increased survivability. Interestingly, this is best achieved by working closely with your own team. Players trained only to go for killing shots are in the style of Arma3 fire fights contributing less to their team in certain situations. I further suspect that the AIs reputation for laser accuracy, especially in urban or jungle environments is actually an expression of poor use of suppression fire. Hammering the position of a known, or suspected enemy, vastly increases life expectancy. I am unsure if suppression values have any bearings on any class of vehicle. RHS vs all others Is the high initial level of AI accuracy, especially seen in vanilla, CUP and (under certain circumstances NIArms) good design? I will begin by saying that the data I have collected can only inform mission designers to be aware of how these weapons behave very differently. Read the following as a small exposition on my own thoughts and preferences. My experience with playing with different communities, both public and closed, is that the AI accuracy is generally tweaked. Sharp and deadly gameplay is all well and good, but becomes an annoyance in a world filled with distractions: Radio comms, latency issues, chain of command, interpreting mission scripts, and dealing with admin (technical issues, respawning, dying and so on). To the point where I see most communities reduce AI accuracy to values comparable to the RHS default ones. This only becomes a problem if RHS and vanilla/CUP/NIArms are used interchangeably by the AI. I see RHS cutting out the middle man. Giving very plausible and playable accuracy settings, without necessitating extensive tweaks to the default values. In addition the relative low accuracy of a RHS AKM does not mean all RHS weapons fare equally poorly. RHS M4A1s with optics perform comparably to bare bones MX rifles. This adds another important element to a mission makers toolbox-- contrasts between modern and older weapons. Finally, there is a simulations side of thing. Most investigations into combat marksmanship suggests it is very low indeed. Stress, adrenaline and fatigue takes its toll. I see vanilla weapon accuracy modelled on bench accuracy tests, whereas RHS accuracy values are based on expectations of combat marksmanship. This does not mean RHS settings are perfect. For very urban, cqb styled operations in the vein of Rainbow Six or Ground Branch, I find vanilla weapons work better. At sub-30 meters AI armed with vanilla weapons will behave and react in a convincing and deadly manner. For these three reasons I see the RHS solutions as superior to the others (practical, adding contrasts, and realism). 4 Share this post Link to post Share on other sites
nkenny 1057 Posted June 20, 2019 Updated experimental suite by nkenny Introduction Having a bit of time on my hands I have rebuilt my experimental weapons testing suite. The previous testing facility was calibrated to demonstrate the large difference between different mods configuration of the same weapon. With the ability to generate more numbers more easily, comes the opportunity to perform more refined tests between otherwise similar weapon systems. Experiment I ran the experiments using no mods and the latest Development Release candidate 1.95.145769. The tests were run on a fresh, default profile with no tweaks made to AI skill or settings. The goal was to test every assault rifle present in vanilla Arma 3 (with DLCs) to create a point of comparison and reference. Description The new testing suite is built around ten walled lanes. Each lane is reused for testing cycles where shooter and targets are spawned and despawned. The shooter is equipped with a weapon and six magazines. The shooter begins facing four targets~ a fire team. The targets are unarmoured and unarmed. Each target is presented on a line with five meter intervals. The distance between shooter and targets can be configured. All actors are forced to remain static for the duration of the cycle. Each cycle runs until all targets are dead or the shooter runs out of time (three minutes). Typically running out of time means the shooter has expended all ammunition. Once either condition is fulfilled, the lane is cleared-- shooter and targets despawned-- and a new cycle is initiated. The results of each cycle is recorded. The end of the exercise has the user manually pasting the data into a larger file. Measurements Each weapon was tested across four ranges: 25, 100, 200 and 300 meters. Each testing cycle ran 100 times, with a reset and reload of the mission between weapon systems. The record contains figures for each round, as well as an overall average of the current figures: Weapon classname Accuracy, hits divided by shots fired Hits, both total and per cycle Shots, both total and per cycle Time spent, measured in seconds and minutes Cycle Success or Time out. (In the case of Time out, the number of survivors is recorded) Expectations Assault weapons to perform in a homogeneous manner with a few outliers. Data There is a massive amount of data. For persons interested in each test cycle in detail I provide a link to the raw data in the references. To give an example of how this data looks, and is read, check the SPOILER tag in the references section. The data presented is the 6.5 MX rifle. First spoiler contains the data, second provides a screenshot of the new VR suite. Below that is a summary of all weapons organised by range categories. Source: link Findings The test revealed clear trends in weapon configuration. As a rule of thumb the Average Time measurement is the best indicator of efficiency. The number reflects the time in seconds necessary for the AI to kill four unarmoured soldiers at the given range. (1) This confirms and earlier finding. The AI will in perfect conditions kill four soldiers with considerable speed. Cover, concealment, body armour and suppression will all mitigate these numbers. But being spotted is very dangerous. (2) A surprise finding is that accuracy is at the highest at the 100 Meter mark. At 25 Meters weapons are configured for automatic fire, and this has a toll on marksmanship. In fact some weapons suffered Time outs at close range. (3) As noted in earlier tests. When presented with four targets the AI will not focus on a single enemy. The AI will shift fire, seemingly randomly, to cover all four. As discussed elsewhere, while this compromises the timing, this is not a flaw in game or simulator terms. (4) I will now go through each range category and list, roughly, the best and worst. At 25M the best weapons are the AKM, CAR95 (CTAR) and mk20. The Katiba trails very closely behind, losing only a little to accuracy. The AK12 is also interesting, because it has a kill time similar of the top weapons, but does at a cost of ammunition-- 600 rounds more than its older relative the AKM. The worst weapon of the bunch is the MX rifle which suffers the greatest numbers of timeouts and longest time (nearly double!) At 100M the stand out weapons are the SPAR-16, MX-rifle and Promet (MSBS65). Throughout the test the TRG21 will closely match the SPAR-16 in performance. Already the AKM is outmatched and will continue to suffer. At 200M the MX and Promet remain top dogs, with the 5.56 rifles SPAR-16 and TRG21 offering equal time to kill. The AK12 and Katiba and are starting to suffer. With the CAR-95 falling closer to the AKM in time to kill. At 300M the MX simply dominates in time to kill. The Katiba has a time to kill of 66 seconds, but Time outs betray inconsistent performance. Interestingly the SPAR-16 is now starting to flex its superior range to the TRG21, though I expect the 5.56 caliber is making both weapons suffer. The Mk20 is now beyond effective range. Low accuracy, low time to kill, and many timeouts. The CAR-95 is notable for tremendous high rate of fire: 5433 rounds. The AKM proved unable to engage targets beyond 250 meters. Conclusion The large number of weapons and wealth of data makes for a potentially confusing dataset to work with. I will here present a way of understanding the numbers: Weapon pairs. Finally I will discuss some aspects the numbers do not suggest. The Mk20 and TRG21 The case of the Mk20 and TRG21. These weapons provide a good starting point to understanding how weapons are configured. Notably the weapons feature very different performance characteristics: the Mk20 is an excellent CQB weapon-- high rate of fire and accuracy at 0-100 Meters, whereas the TRG21 comes into beyond this range. It is notable that I can find no simulation or realism reason why the weapons perform as they do. As far as I can tell both are reasonably modern bullpup rifles slinging 5.56x45 caliber bullets. There is no self explanatory reason why one should be better at CQB than the other! It seems to be a decision made purely for GAME DESIGN reasons*. Bohemia wanted AAF forces to be equipped with a close quarters weapon, and handled a mixture of both to the guerilla counterparts. This is not necessarily poor game design. Having weapons with strongly configured differences makes for more colourful scenarios. It also makes AI behaviour more consistent. Weapon pairs What is also interesting is that the weapons pair mentality is repeated! Consider the default CSAT and NATO weapons: the Katiba and MX. These weapons reflect the characteristics of the Mk20 and TRG21, only to a greater degree: The MX rifle is an excellent (if not the best) long range assault rifle. It is amongst the worst at 25 Meters or less, which is where the Katiba dominates. Had the testing targets featured armour, I expect the powerful 6.5 rounds to have skewed results more. Another weapons pair is present in the Apex DLC. The SPAR-16 and CAR-95 (CTAR). Here the difference is more nuanced, but still present. The SPAR-16 is a superior ranged weapon, whereas the CAR-95 is better at closer ranges. NATO has the better long range weapons, CSAT has the better close in weapon. An interesting development is that the CAR-95 has such a high effective rate of fire. At 300 Meters, where it ostensibly is the worse choice, the weapon will still put out a very large amount of bullets (5433, almost 2000 more than the next in line)-- the spray of bullets still matters in range, as the suppressive effect has considerable effect on AI accuracy. Of course not every weapon is organised so easily as NATO = long range, CSAT = short range, there are a few odd ones out. Amongst them is the new Promet assault rifle. I expected this to be ahead of the pack in all categories. Certainly in player hands its high mechanical rate of fire and good iron sights can make it a favourite-- but barring any changes to AI configuration, it actually features a very distinct band of effectiveness: at 100 to 200 meters it operates at its peak**. Another set is the AKM and AK12. The AKM is clearly intended as a low tier weapon, best suitable for guerilla forces. The AK12 is an upgrade in all ways. Whereas the AKM loses effectiveness at 50 meters, the AK12 remains relevant out to 200M, whereafter it stops being in the game. Like the CAR-95, the high rate of fire will play into suppressing the AI. The high rate of fire also introduces some volatility, it oscillates between fast kill times and and simply missing. Best weapons (a) What to read from this: Going by numbers alone, the MX rifle is by far the superior assault rifle in Arma3. Aside from the 25M range category it is either in the lead or completely dominates. At 300M there is no real contest. (b) The SPAR-16 is close behind. It performs consistently across all ranges, being near the top or middle at all ranges. This reflects the SPAR-16 being in many ways simply a better TRG21, held back by its 5.56 caliber. (c) That does not mean that the Katiba is a slouch. It is amongst the best CQB rifles and remains relevant out to longer ranges. At 300 meters it featured the second best time to kill. As I will get to later, the Katiba also exists within a different ecology of small arms than does the MX. Missunderstandings What not to read from this. The best AI weapons of each range category need not reflect player ability or preference. The AI is bound by different rules than players are. The idiosyncrasies of AI handling have little bearing on players experience of the weapons. Especially concerning the practical application of full-auto fire in a high ping environment-- or when applied actively to shoot through cover. It is also important to remember that each weapon exists within different ecologies of equipment. Optics play a major part in improving weapon accuracy-- all of these tests have been performed using no attachments. The availability of armour on both own and enemy troops and the presence of support weapons within the squad all greatly affect the overall fire power expressed by a squad. Logistical challenges, such as the weight of weapons and ammunition, or the benefits of full ammunition compatibility, are also not accounted for. In fact the presence of inferior or lower tiered weapons only add to the simulation. That the older AKM is an inferior rifle (aside from CQB) only adds meaning to the superiority of the ultra modern MX-series. Final words Contrary to the expectation that all weapons to perform largely identical-- subject only to the whims of caliber, I found strong thematic tendencies. CSAT weapons are geared towards close combat, whereas NATO weapons are better at longer ranges. The weapon configuration show cleverness and intelligence: Weapons perform intuitively, with only a few unexpected hard limits (250M for the AKM) or uncommunicated contrasts (the Mk20 is a CQB rifle whereas the TRG21 is long range rifle). It is also clear that the assault weapons have been considered not isolated but in context to available equipment. Future tests 1. In the future I want to test consistency of Global Mobilization weapons to their Vanilla counterparts. 2. An accuracy and killing power test comparing Standing and Prone, and the effect of various default armour classes. 3. Some glaring omissions are: the AKS in 5.45-- which handles more like an SMG in any case. The Type-115 and the new carbine version of the AK12. These may be explored in the future*** 4. A look at the light machine guns and marksman rifles present in Arma3 References - This document in Goggle Docs format, may be easier to read - Original thread on nopryl.no, further numbers, weapons and experiments are here - Experimental mission, links to mission for your own testing - Raw data, to draw your own conclusions MX Rifle data at 200M & New VR course Spoiler WEAPON: ARIFLE_MX_F RANGE: 200M CYCLES: 100 1 wpn: arifle_MX_F - acc: 57 percent (4/7) - time: 00:06 - SUCCESS 2 wpn: arifle_MX_F - acc: 50 percent (4/8) - time: 00:07 - SUCCESS 8 wpn: arifle_MX_F - acc: 66 percent (4/6) - time: 00:06 - SUCCESS 3 wpn: arifle_MX_F - acc: 33 percent (5/15) - time: 00:17 - SUCCESS 5 wpn: arifle_MX_F - acc: 30 percent (4/13) - time: 00:13 - SUCCESS 7 wpn: arifle_MX_F - acc: 35 percent (5/14) - time: 00:15 - SUCCESS 4 wpn: arifle_MX_F - acc: 40 percent (8/20) - time: 00:21 - SUCCESS 9 wpn: arifle_MX_F - acc: 50 percent (5/10) - time: 00:13 - SUCCESS 6 wpn: arifle_MX_F - acc: 30 percent (6/20) - time: 00:20 - SUCCESS 10 wpn: arifle_MX_F - acc: 54 percent (6/11) - time: 00:14 - SUCCESS 11 wpn: arifle_MX_F - acc: 41 percent (5/12) - time: 00:14 - SUCCESS 14 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:11 - SUCCESS 15 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:11 - SUCCESS 17 wpn: arifle_MX_F - acc: 44 percent (4/9) - time: 00:09 - SUCCESS 18 wpn: arifle_MX_F - acc: 57 percent (4/7) - time: 00:10 - SUCCESS 12 wpn: arifle_MX_F - acc: 26 percent (5/19) - time: 00:21 - SUCCESS 19 wpn: arifle_MX_F - acc: 55 percent (5/9) - time: 00:10 - SUCCESS 13 wpn: arifle_MX_F - acc: 38 percent (7/18) - time: 00:20 - SUCCESS 21 wpn: arifle_MX_F - acc: 77 percent (7/9) - time: 00:10 - SUCCESS 16 wpn: arifle_MX_F - acc: 26 percent (4/15) - time: 00:18 - SUCCESS 20 wpn: arifle_MX_F - acc: 50 percent (6/12) - time: 00:16 - SUCCESS 24 wpn: arifle_MX_F - acc: 60 percent (6/10) - time: 00:10 - SUCCESS 22 wpn: arifle_MX_F - acc: 33 percent (5/15) - time: 00:15 - SUCCESS 23 wpn: arifle_MX_F - acc: 33 percent (4/12) - time: 00:14 - SUCCESS 26 wpn: arifle_MX_F - acc: 54 percent (6/11) - time: 00:13 - SUCCESS 27 wpn: arifle_MX_F - acc: 41 percent (5/12) - time: 00:13 - SUCCESS 31 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:10 - SUCCESS 30 wpn: arifle_MX_F - acc: 50 percent (5/10) - time: 00:13 - SUCCESS 29 wpn: arifle_MX_F - acc: 46 percent (6/13) - time: 00:16 - SUCCESS 28 wpn: arifle_MX_F - acc: 21 percent (4/19) - time: 00:19 - SUCCESS 34 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:09 - SUCCESS 25 wpn: arifle_MX_F - acc: 30 percent (7/23) - time: 00:25 - SUCCESS 32 wpn: arifle_MX_F - acc: 33 percent (4/12) - time: 00:14 - SUCCESS 33 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:12 - SUCCESS 36 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:11 - SUCCESS 35 wpn: arifle_MX_F - acc: 30 percent (4/13) - time: 00:15 - SUCCESS 37 wpn: arifle_MX_F - acc: 54 percent (6/11) - time: 00:11 - SUCCESS 41 wpn: arifle_MX_F - acc: 57 percent (4/7) - time: 00:09 - SUCCESS 40 wpn: arifle_MX_F - acc: 50 percent (5/10) - time: 00:12 - SUCCESS 38 wpn: arifle_MX_F - acc: 54 percent (6/11) - time: 00:13 - SUCCESS 43 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:11 - SUCCESS 44 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:13 - SUCCESS 45 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:08 - SUCCESS 42 wpn: arifle_MX_F - acc: 33 percent (5/15) - time: 00:17 - SUCCESS 47 wpn: arifle_MX_F - acc: 71 percent (5/7) - time: 00:08 - SUCCESS 39 wpn: arifle_MX_F - acc: 30 percent (6/20) - time: 00:24 - SUCCESS 46 wpn: arifle_MX_F - acc: 38 percent (5/13) - time: 00:16 - SUCCESS 52 wpn: arifle_MX_F - acc: 85 percent (6/7) - time: 00:08 - SUCCESS 51 wpn: arifle_MX_F - acc: 55 percent (5/9) - time: 00:10 - SUCCESS 53 wpn: arifle_MX_F - acc: 100 percent (7/7) - time: 00:09 - SUCCESS 50 wpn: arifle_MX_F - acc: 46 percent (7/15) - time: 00:17 - SUCCESS 55 wpn: arifle_MX_F - acc: 50 percent (4/8) - time: 00:08 - SUCCESS 54 wpn: arifle_MX_F - acc: 54 percent (6/11) - time: 00:12 - SUCCESS 49 wpn: arifle_MX_F - acc: 41 percent (7/17) - time: 00:21 - SUCCESS 48 wpn: arifle_MX_F - acc: 37 percent (9/24) - time: 00:27 - SUCCESS 56 wpn: arifle_MX_F - acc: 53 percent (7/13) - time: 00:16 - SUCCESS 58 wpn: arifle_MX_F - acc: 46 percent (6/13) - time: 00:14 - SUCCESS 57 wpn: arifle_MX_F - acc: 31 percent (6/19) - time: 00:21 - SUCCESS 62 wpn: arifle_MX_F - acc: 50 percent (7/14) - time: 00:15 - SUCCESS 64 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:13 - SUCCESS 65 wpn: arifle_MX_F - acc: 71 percent (5/7) - time: 00:10 - SUCCESS 60 wpn: arifle_MX_F - acc: 36 percent (7/19) - time: 00:20 - SUCCESS 61 wpn: arifle_MX_F - acc: 29 percent (5/17) - time: 00:20 - SUCCESS 63 wpn: arifle_MX_F - acc: 43 percent (7/16) - time: 00:17 - SUCCESS 59 wpn: arifle_MX_F - acc: 28 percent (8/28) - time: 00:31 - SUCCESS 66 wpn: arifle_MX_F - acc: 33 percent (5/15) - time: 00:18 - SUCCESS 71 wpn: arifle_MX_F - acc: 62 percent (5/8) - time: 00:10 - SUCCESS 67 wpn: arifle_MX_F - acc: 60 percent (9/15) - time: 00:18 - SUCCESS 69 wpn: arifle_MX_F - acc: 53 percent (7/13) - time: 00:17 - SUCCESS 72 wpn: arifle_MX_F - acc: 41 percent (5/12) - time: 00:15 - SUCCESS 70 wpn: arifle_MX_F - acc: 31 percent (5/16) - time: 00:18 - SUCCESS 75 wpn: arifle_MX_F - acc: 87 percent (7/8) - time: 00:10 - SUCCESS 74 wpn: arifle_MX_F - acc: 43 percent (7/16) - time: 00:18 - SUCCESS 73 wpn: arifle_MX_F - acc: 29 percent (5/17) - time: 00:20 - SUCCESS 68 wpn: arifle_MX_F - acc: 39 percent (9/23) - time: 00:27 - SUCCESS 77 wpn: arifle_MX_F - acc: 60 percent (6/10) - time: 00:10 - SUCCESS 80 wpn: arifle_MX_F - acc: 80 percent (4/5) - time: 00:06 - SUCCESS 76 wpn: arifle_MX_F - acc: 28 percent (4/14) - time: 00:17 - SUCCESS 78 wpn: arifle_MX_F - acc: 40 percent (6/15) - time: 00:17 - SUCCESS 79 wpn: arifle_MX_F - acc: 41 percent (5/12) - time: 00:13 - SUCCESS 82 wpn: arifle_MX_F - acc: 55 percent (5/9) - time: 00:10 - SUCCESS 81 wpn: arifle_MX_F - acc: 63 percent (7/11) - time: 00:13 - SUCCESS 83 wpn: arifle_MX_F - acc: 50 percent (5/10) - time: 00:11 - SUCCESS 84 wpn: arifle_MX_F - acc: 44 percent (4/9) - time: 00:11 - SUCCESS 85 wpn: arifle_MX_F - acc: 90 percent (9/10) - time: 00:10 - SUCCESS 87 wpn: arifle_MX_F - acc: 46 percent (6/13) - time: 00:14 - SUCCESS 89 wpn: arifle_MX_F - acc: 55 percent (5/9) - time: 00:10 - SUCCESS 86 wpn: arifle_MX_F - acc: 35 percent (6/17) - time: 00:19 - SUCCESS 92 wpn: arifle_MX_F - acc: 50 percent (5/10) - time: 00:11 - SUCCESS 91 wpn: arifle_MX_F - acc: 45 percent (5/11) - time: 00:12 - SUCCESS 94 wpn: arifle_MX_F - acc: 44 percent (4/9) - time: 00:11 - SUCCESS 90 wpn: arifle_MX_F - acc: 57 percent (8/14) - time: 00:17 - SUCCESS 88 wpn: arifle_MX_F - acc: 28 percent (6/21) - time: 00:25 - SUCCESS 93 wpn: arifle_MX_F - acc: 42 percent (6/14) - time: 00:16 - SUCCESS 97 wpn: arifle_MX_F - acc: 75 percent (6/8) - time: 00:10 - SUCCESS 96 wpn: arifle_MX_F - acc: 36 percent (4/11) - time: 00:12 - SUCCESS 95 wpn: arifle_MX_F - acc: 33 percent (6/18) - time: 00:19 - SUCCESS 98 wpn: arifle_MX_F - acc: 40 percent (4/10) - time: 00:12 - SUCCESS 99 wpn: arifle_MX_F - acc: 41 percent (5/12) - time: 00:14 - SUCCESS 100 wpn: arifle_MX_F - acc: 35 percent (7/20) - time: 00:22 - SUCCESS FINAL REPORT: arifle_MX_F - average accuracy: 44 percent (6/13) - average time: 00:14 - total cycles: (100/100) - range : 200M TOTAL HITS: 557 TOTAL SHOTS: 1262 TOTAL TEST TIME: 03:46 Spoiler - Experimental Suite -k * In fairness the Mk20 iron sights are very suggestive that this weapon is better at closer ranges. The TRG21 in contrast has excellent adjustable sights. ** It is possible the Promet rifle should be paired, contrasted to the new AK12 carbine. *** As a matter of fact these experiments have already been performed and are available in the source material. Both in RAW data form and the linked Google document. I chose to keep them away from this analysis because these rifles are in the case of the AKS, much closer to SMGs, the AK12U, essentially the same as the AK12, and the Type-115; a special, very limited use Assault Rifle. 5 Share this post Link to post Share on other sites
moondawg 12 Posted June 20, 2019 Great work! I like the "Game design, not simulation" theory. Will be interesting to see how this data stacks up against mods. 1 Share this post Link to post Share on other sites
mmm 35 Posted June 21, 2019 Not surprisingly both top performer are configured with best dispersion value in class, and in the case of SPAR-16 less AI dispersion than counterparts IIRC. Even if both suffers from some of the worst external ballistics in their own class. Also TBH Type 95 LSW and HK416 14.5" are assault rifles as far as AI's concerned, as they're configured with rifle fire discipline and AI dispersion. You'll see when and if you move to LMG test, and probably have their results displayed along side rifles than LMGs. Share this post Link to post Share on other sites
nkenny 1057 Posted July 16, 2019 @mickeymen Testing of optics. Repeat of experiments according to the parameters listed above, save for the inclusion of a 400 meter stage. At the 400m stage the AI stance is set to "AUTO". Optics background The AI have three types of Optics configured. None, OpticType 1 ~ RCO, MRCO combat optics, and OpticType 2 ~ SOS, LPRD sniper optics. These optics add extra fire modes to the AI. These fire modes come with unique settings for engagement range, rate of fire and dispersion (generally dispersion is the same for all ranges however). Data Source Source data Conclusions Within 400 meters the effect of optics is negligible. The core weapon class is more important. That said optics add to the maximum range by with which a weapon can engage. This testing scheme does not explore that to further detail, but looking at the MXM config the following becomes apparent. MXM, single * aiRateOfFire = 2 * aiRateOfFireDistance = 500 * MaxRange, 450 * MidRange, 350 * MinRange, 120 MXM, single medium Optics 1 * aiRateOfFire = 6 * aiRateOfFireDistance = 700 * MaxRange, 750 * MidRange, 450 * MinRange, 120 MXM, single far Optics 2 * aiRateOfFire = 8 * aiRateOfFireDistance = 900 * MaxRange, 900 * MidRange, 550 * MinRange, 200 Optics should therefore extend the maximum range which the AI will engage. This also accounts for why there is no immediate increase of potency. Even when an optics fire mode is used, the weapon is still limited by external ballistics-- and rate of fire in these longer range modes are generally slower. By vanilla configuration, Arma2 and Arma3 has tended to give Ranged Combat Optics to squad leaders. I would speculate that the reason for this is to get the squad to engage targets at greater range. Interestingly, given other variables, this may be counterproductive. Ideally the AI squad leader should be trying to move his element as close as possible where a larger number of squad weapons are accurate and able to engage. edit: To clarify. Giving squad leader a short ranged weapon-- like an SMG, might not be a bad idea. -k 2 1 Share this post Link to post Share on other sites
nkenny 1057 Posted July 18, 2019 Intro So I have continued to explore the effect of optics.This led to two discoveries. I realised that 500 meters was a breaking point, config wise for weapon fire. Theoretically one should see a sharp distinction, between fire with and without optics. This was supported by looking at the Time outs or failed rounds. At 400 meters the Type 2 sniper optic came out, barely ahead. Figures In short I repeated the test according to all previous specifications, but at 500 meters. The test was done on Development version 1.95.145895. Each full cycle took about 30 minutes. - Full source here as usual. Conclusions & findings At 500 meters more nuance becomes visible. In the case of the MXM there is little difference in killing efficiency. Save that by pure numbers, (1) Ranged combat optics, like the HARM, actually decrease efficiency. Rate of fire decreases for very little increase in killing power. In the case of (2) sniper optics, there is also very little gain, though overall greater consistency is achieved. This is seen by fewer time outs, but overall more targets left alive on each cycle. A curious discovery is that at 500 meters ironsighted MXM shooters remained standing 8 out of 10 times. All other shooters preferred prone stance. The overall winner is ironsights. Higher overall rate of fire pays of in the specific environment created in the test scenario. Perhaps against moving targets, or targets with shorter periods of visibility numbers will be skewed more in favour of magnified optics. If my methods are correct, I can only conclude that the MXM is poorly configured in regards to optics. This may be generalised to affect other weapons. (While remembering that external ballistics of the fired round is of great importance). Secondary experiment I proposed that optics allowed weapons to engage targets at further ranges. Running full 30 minute tests was time prohibitive, so I ran much faster 10 cycle attempts. The test was set to 800 meters. This led to the following finding (RAW DATA): Spoiler // 3 rifles - 10 cycles -- 800 METERS WEAPON: ARIFLE_MXM_F RANGE: 800M CYCLES: 10 1 wpn: arifle_MXM_F - acc: 0 percent (0/4) - time: 03:00 - TIME OUT (4 alive) 2 wpn: arifle_MXM_F - acc: 20 percent (1/5) - time: 03:00 - TIME OUT (4 alive) 3 wpn: arifle_MXM_F - acc: 33 percent (1/3) - time: 03:00 - TIME OUT (4 alive) 4 wpn: arifle_MXM_F - acc: 0 percent (0/3) - time: 03:00 - TIME OUT (4 alive) 5 wpn: arifle_MXM_F - acc: 25 percent (1/4) - time: 03:00 - TIME OUT (3 alive) 6 wpn: arifle_MXM_F - acc: 100 percent (1/1) - time: 03:00 - TIME OUT (4 alive) 7 wpn: arifle_MXM_F - acc: 0 percent (0/1) - time: 03:00 - TIME OUT (4 alive) 8 wpn: arifle_MXM_F - acc: 0 percent (0/4) - time: 03:00 - TIME OUT (4 alive) 9 wpn: arifle_MXM_F - acc: 0 percent (0/1) - time: 03:00 - TIME OUT (4 alive) 10 wpn: arifle_MXM_F - acc: 0 percent (0/4) - time: 03:00 - TIME OUT (4 alive)FINAL REPORT: arifle_MXM_F - average accuracy: 13 percent (0/3) - average time: 03:00 - total cycles: (10/10) - range : 800M TOTAL HITS: 4 TOTAL SHOTS: 30 TOTAL TEST TIME: 03:24 WEAPON: ARIFLE_MXM_HAMR_POINTER_F RANGE: 800M CYCLES: 10 1 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 2 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 3 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 4 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 5 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 6 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 7 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 8 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 9 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive) 10 wpn: arifle_MXM_Hamr_pointer_F - acc: 0 percent (0/0) - time: 03:00 - TIME OUT (4 alive)FINAL REPORT: arifle_MXM_Hamr_pointer_F - average accuracy: 0 percent (0/0) - average time: 03:00 - total cycles: (10/10) - range : 800M TOTAL HITS: 0 TOTAL SHOTS: 0 TOTAL TEST TIME: 03:25 WEAPON: ARIFLE_MXM_SOS_POINTER_F RANGE: 800M CYCLES: 10 1 wpn: arifle_MXM_SOS_pointer_F - acc: 4 percent (1/22) - time: 03:00 - TIME OUT (4 alive) 2 wpn: arifle_MXM_SOS_pointer_F - acc: 14 percent (3/21) - time: 03:00 - TIME OUT (3 alive) 3 wpn: arifle_MXM_SOS_pointer_F - acc: 9 percent (2/21) - time: 03:00 - TIME OUT (4 alive) 4 wpn: arifle_MXM_SOS_pointer_F - acc: 0 percent (0/20) - time: 03:00 - TIME OUT (4 alive) 5 wpn: arifle_MXM_SOS_pointer_F - acc: 5 percent (1/20) - time: 03:00 - TIME OUT (4 alive) 6 wpn: arifle_MXM_SOS_pointer_F - acc: 9 percent (2/21) - time: 03:00 - TIME OUT (4 alive) 7 wpn: arifle_MXM_SOS_pointer_F - acc: 9 percent (2/21) - time: 03:00 - TIME OUT (4 alive) 8 wpn: arifle_MXM_SOS_pointer_F - acc: 14 percent (3/21) - time: 03:00 - TIME OUT (4 alive) 9 wpn: arifle_MXM_SOS_pointer_F - acc: 0 percent (0/21) - time: 03:00 - TIME OUT (4 alive) 10 wpn: arifle_MXM_SOS_pointer_F - acc: 10 percent (2/20) - time: 03:00 - TIME OUT (3 alive)FINAL REPORT: arifle_MXM_SOS_pointer_F - average accuracy: 8 percent (2/21) - average time: 03:00 - total cycles: (10/10) - range : 800M TOTAL HITS: 16 TOTAL SHOTS: 208 TOTAL TEST TIME: 03:26 The raw data is fairly easy to read. Only the Sniper Optic equipped MXM performed with any consistency, actually scoring 16 hits. Second place is again Ironsights which scored 4 hits. Loser is RCO equipped MXM which failed to fire a single round. While this supports my general finding, that optics enable accuracy at longer ranges, the strong showing of the ironsighted MXM rifle suggests that some config tweaks are in order. -k 1 Share this post Link to post Share on other sites
nkenny 1057 Posted March 10, 2020 Restested AKM rifles by nkenny Introduction This is a repeat of the entry experiment where AKMs from different mods were compared. It utilises the new testing suite developed mid-thread. The report coincides with the latest @CUP release, which seemed as fitting time as any. Experiment 1. This test evaluates three larger mod/DLC packs and the vanilla AKM as control: RHS, Global Mobilisation and CUP. 2. It develops and compares numbers from 25 to 400 meters. 3. Because this is primarily an accuracy test, and the new Magwell technology allows it-- the same magazine is used for all weapons : "30Rnd_762x39_Mag_F" (vanilla 30 round 7.62x39 magazine). I will immediately recognise that this may slew some numbers as weapons should in some case be considered within their mod/conversion/DLC infrastructure, but as the original post suggest. This is not commonly how these mod packages are actually used. 4. As before the only mods run was CBA and the relevant mods in their most recent Development versions. - Arma version 1.96.146114 - CUP Weapons DEV version r1419 - RHS Dev version Build 83 - Global Mobilisation version 1.1 Description and measurements The measurements and platform of testing is identical to earlier tests save that units were set to "AUTO" stance at all ranges. Expectations As per previous tests: RHS weapons to be the least accurate, CUP weapons to be the most accurate with other AKMs falling somewhere in between. I know that the CUP team have recently been reviewing some of their weapona ccuracy settings according to https://dev.cup-arma3.org/. Data As before I will present a summary of my findings. To access the raw data check the references link below. Findings - The vanilla AKM reinforces and corresponds with numbers developed in the previous test. Peak accuracy resides around 100 meters and the weapon reaches maximum engagement range at exactly 250 meters. The maximum range is not borne from the figures listed above but other tests. - The CUP AKM is the weapon that follows the vanilla gun the most closely. Even so it remains considerably more accurate than the vanilla weapon at all ranges. At sub-100 meters it as similar times to kill, but at 200 meters it is siginifcatly more deadly than its vanilla counterpart. It is also far more ammunition efficient at all ranges. The CUP gun is subject to the same maximum range as the vanilla AKM; tested to exactly 250 meters. - The Global Mobilisation AKM, actually a East-German version of the Soviet AKM, is by far the most accurate weapon at all ranges. At 200 meters the GM gun is 62% faster killer than its vanilla counter-part. It is also the only weapon which can engage with any degree of accuracy out to 400 meters. Without mitigating factors such as ping, suppression or skill settings, a GM AKM will kill a four man fire team at 300 meters in around 30 seconds, dealing damage every four shot. - The RHS AKM reaches peak accuracy at 25 meters at which time it is comparable to all other weapons. From there it has a slower time to kill and much increased ammunition expenditure to the point of being in a different category. It is worth explicitly mentioning is that all weapons, aside from the RHS AKM, are configured with a drop in accuracy at closer ranges and a peak accuracy around 100 meters, with a loss of accuracy further out. - Other findings: (a) Though not timed, all modded and DLC weapons appear to have a slower magazine reload time than the vanilla weapon. (b) Only the GM AKM fired true automatic fire at 25M Conclusion The hierarchy of deadliness goes as follows: - GM AKM - CUP AKM - Vanilla AKM - RHS AKM 1. Referencing numbers from earlier weapons test one can see that the Global Mobilisation AKM is simply in a class of its own and rivals most modern 2035 weapons. This could in-universe be explained as (1) the weapon being closer to factory new and (2) the weapon should be compared only to weapons from its own mod set and ammunition. These are in my opinion poor explanations. If used blindly alongside weapons from other mods or vanilla it will in the AIs hands outclass all other takers. From a personal perspective as a mission maker and server host, I would say that without severe reductions to default server accuracy and intelligence settings, the weapon is unusable for most scenarios. 2. The CUP AKM remains deceptively accurate. While its numbers are close to vanilla stats-- it routinely outshoots the vanilla weapon in terms of 'time to kill' and 'rounds fired'. This is especially noticeable at average combat ranges of 100-200 meters. 3. RHS weapons again stick out as being configured in a very different manner than other weapons. I have defended the approach of configuring weapons for the 'combat accuracy' rather than 'bench accuracy' above and will not repeat that here. 4. It is my personal opinion that both CUP and vanilla AKMs suffer from the lack of pluss 250 meter engagement range. Not only from a simulations perspective, queue any reports of modern firefights, but even more accessibly from player vs player warfare in Arma-- were humans will often engage at ranges where chances of hitting are low for reasons of both weight of fire and incompetence. 5. Final words remain that mission makers and server/mod managers should remain aware of which weapons are used in which scenarios. Attempting to use RHS and Global Mobilisation weapons interchangeably in AI hands may result in effects wildly diverging from expectations. Future tests CUP, vanilla and RHS all feature 10-11 inch HK416 carbines. References- Raw Data folder Most pertinent dispersion values for 'single' shot at maximum range. vanilla dispersion = 0.00203; cup dispersion = 0.00125; rhs dispersion = 0.0021; gm dispersion = 0.0014992; It is not entirely clear to what the basis of these values are. 2 1 Share this post Link to post Share on other sites
froggyluv 2136 Posted March 10, 2020 Love this nkenny - looks like RHS are the weapons i been looking for What were your AI skill settings -default? And do you notice lowering/raising them having any significant impact on the above? Edit: Just to be clear no offense to CUP which i love - this would just make it possible to have low skill troops ie african militia (RHS AKMS) vs some High Level Trained Militia (Cup AKMS) -its the calibrated disparity im always after Share this post Link to post Share on other sites
nkenny 1057 Posted March 10, 2020 @froggyluv As in previous tests my skillsettings are default. Units are set to 0.5 skill. The skill setting system is in my experiences rather opaque and difficult to get a proper read of. The testing suite makes it easy to run multiple tests of it. -k Share this post Link to post Share on other sites
nkenny 1057 Posted March 11, 2020 On 3/10/2020 at 4:15 PM, froggyluv said: Edit: Just to be clear no offense to CUP which i love - this would just make it possible to have low skill troops ie african militia (RHS AKMS) vs some High Level Trained Militia (Cup AKMS) -its the calibrated disparity im always after Actually this is precisely one major design flaws of vanilla Arma3. It was initially released with a lack of dimensionality which make it possible to create such asymmetric scenarios. Within the same mod ecology I think the ideal solution would be to set base accuracy to chronological origin of the weapon. It would make perfect sense to have, given a 2020ish timeframe, that poor quality troops carry older AK rifles, whereas high level troops would carry modern and accurate AKs with RIS and polymer bits and all that jazz. CUP and vanilla configs are quite similar. I wonder how much of a difference the dispersion values above actually make. Perhaps I'll make a tiny mod to test the theory. *shrug* -k edit: I too greatly appreciate the efforts of the CUP team. 1 Share this post Link to post Share on other sites
mmm 35 Posted March 18, 2020 My impression is that dispersion would not have been THAT significant if not for the AI dispersion coefficient that amplify if for a few times for AI usage, so it becomes likely the main source for weapon hit probability disparity. Just for example your previous test here seems corroborate the theory well. Those with strong performance in hit probability department are all configured with superior dispersion. While for real human player a flat trajectory and short flight time brought by higher muzzle velocity is likely more important than a moderate dispersion difference, as exemplified by the SPAR-16 Aka HK416 10.4" which totally defies the expectation and grossly outperforms a CTAR with otherwise mostly comparable external ballistics. Edit: As for the discrepancies between different AKMs, just off the top of my head, it might also be helpful to look at AI dispersion coefficient. I don't have access to GM but I assume it might be configured somewhat different from established standard there. RHS obviously adopted automatic fire for AI usage, which adversely affects killing efficiency as far as current ARMA AI firing simulation goes. I'm certain only the first round within a burst has any chance of hitting the intended target. 1 Share this post Link to post Share on other sites