Machine learning / AI projects using ArmA?

[specialsmith 35]

Hi,

 

Anyone aware of a research project using ArmA as the test bed for some kind of an advanced machine learning / AI implementation? Google DeepMind already beat the best human Go player, anyone created an AI mastermind tactical genius commander for ArmA? :)

 

 

Sent from my iPhone using Tapatalk

[fn_Quiksilver 1636]

Generally no, there are too many variables for the AI. In chess and Go, there are limited options so it is possible to attach a "success/fail" value to each move and store that move/position + value in memory.

 

Simply having a player as Zeus to govern the AI is much easier to arrange with much better results and less CPU cost.

[Rydygier 1316]

Thought about some simple neuron network tests once, few years ago, just out of curiosity, but so far it remained on early stage. I've to better grasp nn concept itself first. Maybe some day, who knows. As for amount of variables - I was thinking about two ways:

 

1. Low level AI, simplistic duel

 

Mostly academic tests. Two AI against each other. Both can do one of several things: shoot at the opponent, change stance, move towards or opposite etc. 

 

2. Commanding AI

 

Determines tactical situation by analyzing map in similar way, OCR works (map divided into squares, distinguished those, where hostile or own presence is known), then applies/advices tactics depending on it. Has to work on highly generalized level to keep variables on conceivable level.

 

Both only concepts, may be not doable/not practical.  

 

Also I recalled this thing:

 

 

[specialsmith 35]

Generally no, there are too many variables for the AI. In chess and Go, there are limited options so it is possible to attach a "success/fail" value to each move and store that move/position + value in memory.
 
Simply having a player as Zeus to govern the AI is much easier to arrange with much better results and less CPU cost.

Thanks Quicksilver. Also what i thought. I would think if something could be implemented it should be limited to certain aspect(s) of the AI command / behavior.


Sent from my iPhone using Tapatalk

[fn_Quiksilver 1636]

EA has been experimenting with FPS AI machine learning in Battlefield 1

 

 

may be of interest

[B_Fox 132]

You may want to look at Genesis's learning AI videos

 

and his Vcom AI and Vcom Driving

 

and if I'm not mistaken modders don't get the same accesses to the AI as the devs and if thats the case it would be up to the devs to do such project.

[specialsmith 35]

You may want to look at Genesis's learning AI videos
[/url]  
and his Vcom AI and Vcom Driving
 
and if I'm not mistaken modders don't get the same accesses to the AI as the devs and if thats the case it would be up to the devs to do such project.

Big fan and a heavy user of Genesis92x’s work. ;) Will need to watch the first video though...


Sent from my iPhone using Tapatalk

[ToxaBes 13]

On 31.12.2018 at 1:13 AM, specialsmith said:

Anyone aware of a research project using ArmA as the test bed for some kind of an advanced machine learning / AI implementation? Google DeepMind already beat the best human Go player, anyone created an AI mastermind tactical genius commander for ArmA? :)

Yep, I use trained (over 2000 fights in training set) neural network based on Fast Artificial Neural Network Library as High Command structure for bots on my PvE server. 

Using it on low level don't make any sense because we have Final State Machines for that (good example is VCOM AI).

Every X mins script get information about known enemies for each group, prepare data and send to Mysql DB via extDB3 as command request (+ get response for previous command requests). PHP script called by cron every minute check DB for new command requests, use Neural network for build command response and save it back into DB. Quite simple.

 

Main things I got after more than year of using this:

1. 90% of players are not tactical geniuses, so if you will use all played battles as training sets for additional learning you faced in degradation of neural network (why complex things needed if simple attack from side is effective)

2. 90% of players don't like pain and humiliation from AI so your server could become empty quite fast

3. 90% of cases are covered by VCOM AI (thanks to Genesis for this) and if you don't think so - you can add needed parts inside FSM logic (I added some things like: if players have defend position in house, bots with RPG 7/RPG42/PCML will fire rockets in this house).

 

So if you planning to use some neural network in Arma, just use VCOM AI first and expand it as needed. It dramatically save your time (and population of players on server).

 

 

[fn_Quiksilver 1636]

15 hours ago, ToxaBes said:

Yep, I use trained (over 2000 fights in training set) neural network based on Fast Artificial Neural Network Library as High Command structure for bots on my PvE server. 

Using it on low level don't make any sense because we have Final State Machines for that (good example is VCOM AI).

Every X mins script get information about known enemies for each group, prepare data and send to Mysql DB via extDB3 as command request (+ get response for previous command requests). PHP script called by cron every minute check DB for new command requests, use Neural network for build command response and save it back into DB. Quite simple.

 

Main things I got after more than year of using this:

1. 90% of players are not tactical geniuses, so if you will use all played battles as training sets for additional learning you faced in degradation of neural network (why complex things needed if simple attack from side is effective)

2. 90% of players don't like pain and humiliation from AI so your server could become empty quite fast

3. 90% of cases are covered by VCOM AI (thanks to Genesis for this) and if you don't think so - you can add needed parts inside FSM logic (I added some things like: if players have defend position in house, bots with RPG 7/RPG42/PCML will fire rockets in this house).

 

So if you planning to use some neural network in Arma, just use VCOM AI first and expand it as needed. It dramatically save your time (and population of players on server).

 

 

 

can we play on your server? always like fighting against different AI flavors

[specialsmith 35]

Yep, I use trained (over 2000 fights in training set) neural network based on Fast Artificial Neural Network Library as High Command structure for bots on my PvE server. 
Using it on low level don't make any sense because we have Final State Machines for that (good example is VCOM AI).
Every X mins script get information about known enemies for each group, prepare data and send to Mysql DB via extDB3 as command request (+ get response for previous command requests). PHP script called by cron every minute check DB for new command requests, use Neural network for build command response and save it back into DB. Quite simple.
 
Main things I got after more than year of using this:
1. 90% of players are not tactical geniuses, so if you will use all played battles as training sets for additional learning you faced in degradation of neural network (why complex things needed if simple attack from side is effective)
2. 90% of players don't like pain and humiliation from AI so your server could become empty quite fast
3. 90% of cases are covered by VCOM AI (thanks to Genesis for this) and if you don't think so - you can add needed parts inside FSM logic (I added some things like: if players have defend position in house, bots with RPG 7/RPG42/PCML will fire rockets in this house).
 
So if you planning to use some neural network in Arma, just use VCOM AI first and expand it as needed. It dramatically save your time (and population of players on server).
 
 

Haha! Wow, cool. It is exactly what I was thinking one could achieve by doing this, and the end result is “funny” as well. :D I can truly see the predictions of yours happening with average (or even with advanced high-humiliation-tolerance ArmA players)

Btw, the RPG-to-house case is exactly what has been missed back in the day when playing OFP and making one puny house an eternal fortress of unbeatable defense against AI. Today VCOM AI already takes care of that, well, if not RPG, in other very effective ways. We just recently got our asses blown away by VCOM coming and planting a satchel explosive. Took all six of us out at one shot.

It will remain interesting to see if some kind of high command AI competition surfaces in future.

Great stuff ToxaBes! Would be nice to experience your AI one day.


Sent from my iPhone using Tapatalk

[ToxaBes 13]

6 hours ago, fn_Quiksilver said:

 

can we play on your server? always like fighting against different AI flavors

I'm in the middle of refactoring mission code and logic so let me finish it first. After that I will be glad to see all of you on server, beaten by my Red Queen, he he.

I will post here or add new thread about server with details when training mode will be ready for public.

 

But here is one problem. Mission is not translated to English. Let me clarify, why.

I try to make server for team training where results of good fights will be used for additional neural network learning. Mission logic divided into two parts: training and qualification.

Training mode is similar to main AO in Invade&Annex mission. Qualification mode is like linked side missions from Invade&Annex but generated by my second neural network (it's like quest engine and it's in hard WIP).

We can't train quest AI in 2 languages in the same time and our team is Russian-speaking (not all guys know english on good level) so we use russian language.

 

So training mode will be good for all no matter of language but for playing in qualification mode you will need somebody who will translate task details generated by AI and other things.

 

[ZackTactical34 86]

On 1/4/2019 at 8:27 AM, ToxaBes said:

Yep, I use trained (over 2000 fights in training set) neural network based on Fast Artificial Neural Network Library as High Command structure for bots on my PvE server. 

Using it on low level don't make any sense because we have Final State Machines for that (good example is VCOM AI).

Every X mins script get information about known enemies for each group, prepare data and send to Mysql DB via extDB3 as command request (+ get response for previous command requests). PHP script called by cron every minute check DB for new command requests, use Neural network for build command response and save it back into DB. Quite simple.

 

Main things I got after more than year of using this:

1. 90% of players are not tactical geniuses, so if you will use all played battles as training sets for additional learning you faced in degradation of neural network (why complex things needed if simple attack from side is effective)

2. 90% of players don't like pain and humiliation from AI so your server could become empty quite fast

3. 90% of cases are covered by VCOM AI (thanks to Genesis for this) and if you don't think so - you can add needed parts inside FSM logic (I added some things like: if players have defend position in house, bots with RPG 7/RPG42/PCML will fire rockets in this house).

 

So if you planning to use some neural network in Arma, just use VCOM AI first and expand it as needed. It dramatically save your time (and population of players on server).

 

 

Did you perform those tests with VCOM? Also, in what particular way did the AI actually improve?

[ToxaBes 13]

14 hours ago, ZackTactical34 said:

Did you perform those tests with VCOM? Also, in what particular way did the AI actually improve?

Yep.

VCOM AI provide good improvement of individual and in-group tactic, but it don't work with high command. For neural network group is smallest operation unit. For VCOM AI group is highest operation unit. So they work on different operation levels and supplement each other. 

Neural network trained for manage limited number of forces on limited territory (defend zone with R~1km). It understand type of terrain/height gradient, types of units in groups,  vehicles etc etc. in both own and enemy forces. 

[Rydygier 1316]

Man, would be great to have such well trained (or still learning) neural network as high level AI opponent mod/script also for singleplayer. 

[ToxaBes 13]

3 minutes ago, Rydygier said:

Man, would be great to have such well trained (or still learning) neural network as high level AI opponent mod/script also for singleplayer. 

I'm afraid for one player in singleplayer mode it just don't make any sense. 

[Rydygier 1316]

Maybe I misunderstood. Doesn't it command chosen AI groups against opposing side?

[ToxaBes 13]

1 hour ago, Rydygier said:

Maybe I misunderstood. Doesn't it command chosen AI groups against opposing side?

You are correct. 

I mean "Lone wolf" tactic is not a tactic at all. Even if you have some amount of friendly bots. Using Neural network in such variant is  overhead. VCOM AI is all what you need.

Also, extDB 3 required for AI will not work in singleplayer mode  (for me it works only in multiplayer).

 

[Rydygier 1316]

You know, player also may use High Command. Also, he may be lead by another high level AI. There are players, that enjoy being thrown into some big battle revolving around, where player isn't a center of events, just one of the cogs in the machine, where armies of both sides are lead by the AI. Years ago I made high level AI commander (but not learning one, sadly) mostly for singleplayer, and it is still in use. 

 

Lastly there are some people, like me, enjoying just watching, how fancy AI does its thing. 

 

Quote

Also, extDB 3 required for AI will not work in singleplayer mode  (for me it works only in multiplayer).

 

Well, that I understand - technical limits. The only way would be solo "MP" session... 

[ToxaBes 13]

1 hour ago, Rydygier said:

You know, player also may use High Command. Also, he may be lead by another high level AI. There are players, that enjoy being thrown into some big battle revolving around, where player isn't a center of events, just one of the cogs in the machine, where armies of both sides are lead by the AI. Years ago I made high level AI commander (but not learning one, sadly) mostly for singleplayer, and it is still in use. 

 

Lastly there are some people, like me, enjoying just watching, how fancy AI does its thing. 

 

 

Well, that I understand - technical limits. The only way would be solo "MP" session... 

I know, we all could play in Arma by own way and this really great thing. Solo MP session will be an option as well as MP with teammates.

But Solo MP make sense only with disabled self learning, it already trained to play against teams (5 or more human players) so learning on solo-enemy lead it to degradation. Even if you create second group of skilled bots it will lead to degradation (I checked it).

Also playing against newbies/casual players in period will leed to degradation (also checked twice, and than restored AI files from backup).

So, Singleplayer/Solo MP modes are not interesting for me because AI can't learn.

 

 

[ZackTactical34 86]

In all honesty, machine learning AI is overkill. All you really need is a good FSM for the AI. Also, @ToxaBes, you should check out Smart AI since you seem interested in complex AI behavior:

Firing from cover

 

Maneuvering in CQB environment

 

Fast and Deadly CQB Firefight

 

Fireteam CQB Movement

 

And no, machine learning was not involved with any of this. Also, behavior varies according to the situation, which makes it even more dynamic.

[B_Fox 132]

5 hours ago, ZackTactical34 said:

In all honesty, machine learning AI is overkill. All you really need is a good FSM for the AI. Also, @ToxaBes, you should check out Smart AI since you seem interested in complex AI behavior:

I heard about this from ChrisB and It works really well but it's a match up of different mods that did not really make it to Arma 3 or died out early most notably SLX and GL4. I also heard there are groups who play Arma 2 with private AI mods which is said to be on of the best if not the best. But I think that the AI is missing a more human quality like, "Hey 0 ammo vs 1 squad with ammo maybe we should surrender since I don't want to get slaughtered"

 

Also maybe a morale system

Spoiler

 

AI morale system:

AI morale should affect how hard they fight, do they flee easily or die hard, if their weapons run dry do they run or surrender or save that one grenade for a fake surrender. Does the sight of a death of the squad leader cause them to panic and flee or take a morale hit but stand firm or the surrender of one cause a domino effect of the squad. I think all this will be a positive addition to gameplay especially SP people like myself.

 

 

[Valken 622]

@ZackTactical34I am using SmartAI in ARMA2CO thanks to ChrisB but AFAIK there is nothing like this on ARMA 3 and VCOM is pretty close. And in heavy fights with lots of AI, it does bog down the CPU.

 

If there was a mod that used compute (OpenCL) to calculate/model/determine AI behavior - making decisions on the fly based on terrain, enemy type, ammo count, teammate count, then I am all for it. That is one area where I would gladly invest money into such as a low or mid end GPU just dedicated to AI decision making. 

 

If you really watch team tactics on MP servers, it is a bunch of "AI" or players feeding information to work together. It is not a bunch of long term decision making but a LOT of short term decision making based weight - chance of success - in a conflict and the end goal to survive while taking out the enemy based on your team's force.

 

I think an OpenCL type accelerator can do that easily or help - AI runs through a bunch of what ifs in real time and decides what do. The great thing about OpenCL is that it is hardware agnostic. It can run on CPU or GPU or any OpenCL compliant device and scale. Imagine a server using the onboard GPU as an OpenCL accelorator just to increase and / or improve AI behavior.

[ToxaBes 13]

15 hours ago, ZackTactical34 said:

In all honesty, machine learning AI is overkill. All you really need is a good FSM for the AI.

If we talking about tactic inside group (finding covers, moving, supressing fire, controlling sectors, garrisoning, mining etc.) I fully agree with you. Situation changed when we have 50-60 groups with armored vehicles/bots/helicopters up to 200-300 units in total placed in zone  ~3.5 square km without human commander. In this case maximum what you can have here with FSM - share found targets between groups and move some groups to this target.

For effective fight we need to answer on next questions:

• which group types should be send? we have tanks, infantry, helis etc.
• should we use infantry from roadblocks/bunkers inside zone?
• should use bots placed in buildings for scouting?
• how many groups to send? we should send effective minimum which will enough to win, so we need to store and use some data about previous fights
• for each group we should decide, in what position we need to move it (in front of enemy, in side etc)
• which group should fire first? 
• what to do if we already fight with enemy group and it still alive and moving forward?
• what to do if we have several enemey groups infiltrated in our zone from different sides?
• what to do if we need reinforcements? call it or retreat?
• what to do if players attack from one side several times? (ambush)
• what to do after fight? just patrol this area or back to previous position or something else?

Each of this question have a lot of sub-questions. With FSM you can, for example, call for friendly mortar but not answer on all this questions because FSM don't store information about previous fights. Good commander (player) can answer on this questions, but this is PvE - we have humans only on one side. That's why I use trained network.

Again, 

• Tactic inside group -> FSM based mods
• Tactic outside groups (HQ) -> trained neural network

As far as I can see from your videos, SmartAI is really good. But this is in-group tactical solution.

[lex__1 422]

Those who have tried to play machine chess know that there is a basis for the positions of pieces on a chessboard and options for moves in this situation.
I think this is a very big job to create a base of moves for movement in a group and movement of groups. It should be borne in mind that this, unlike machine chess, is complicated by the 3D space of the position of groups and AI in groups.
Probably every situation on the map should be considered in a group as one chessboard, scaling the same tasks for several groups as for one chessboard.

[Defunkt 431]

I think the key to smarter AI in A3 would be to offload higher-level functions (i.e. Finding Cover) and particularly prioritization (determining which units require the most frequent updates - because they're under fire or engaged in CQB) to a separate thread or even separate machine. This is more or less what the Headless Client seeks to achieve but the degree to which missions need to be specially written/deployed to utilize it makes it next to useless to the vast majority of players who just want to throw something together in the editor.

 

I've dallied with the idea of trying to create an extension which an AI mod like VCOM could feed with periodic updates to unit state and location which are then constantly prioritized in a separate thread such that the mod could then also query the extension for; who do I most need to deal with next and what should they broadly be doing? Without access to map data this evaluation would be fairly rudimentary based principally on the threat/efficacy/morale/awareness and especially proximity of enemy or friendly units but it's not beyond the bounds of possibility that it could pass some finer-grained evaluation off to another game instance (think Headless Server) to perform LOS/cover checks (in SQF with actual map data). Just 'Blue Sky' thinking at this stage but it's going to be a long while until Arma 4 (and even longer until content compares to what's available in A3) - so, maybe one day.