AI vs AI contest (How to evaluate AI effectiveness?)

[maturin 12]

Gammadust, I think I found another issue with your pet topic: FoV.

Check out this ticket and tell me what you think: http://feedback.arma3.com/view.php?id=13431

Also, if you know why that piece of shit bugtracker doesn't accept PBO, SQM or SQF, please tell me. How are we supposed to post repro?

[Dwarden 1125]

compress it into zip or 7z

[maturin 12]

compress it into zip or 7z

Dwarden, do you think you could get that information onto the HowTo guide of the bugtracker? I'm sure it would help a lot of people, or at least give you something to say when they complain about how devheaven is better.

Edit: In its infinite wisdom, the bugtracker does not allow me to upload files either in notes or when editing an existing ticket.:angryfire:

[gammadust 12]

Awesome work Gamma, gonna follow this one for sure!

/KC

Tnx, hope to keep the interest up :)

Let the fight begin music is missing ! ;)

I was very curious myself, given the new AI improvements. I don't think 6 runs are sufficient to be conclusive either way, specially since blue won overall in other instaces.

I should probably adapt the modification to current AI default behaviour too, currently I have custom Formation fsm (which attempts at neutralizing Provide Cover cycle) and have a custom danger fsm which forces the AI to check for additional targets before it is allowed to move again (no big changes from previous AI vs AI video). Also Skill is 0.5 for both sides and red has set allowFleeing 0 and enableAttack false.

Gammadust, I think I found another issue with your pet topic: FoV.

Check out this ticket and tell me what you think: http://feedback.arma3.com/view.php?id=13431

Also, if you know why that piece of shit bugtracker doesn't accept PBO, SQM or SQF, please tell me. How are we supposed to post repro?

I'll try and look more deeply into this ticket, the reason i'm being cautious, check my full repro video. Following your exact steps up to 2 mins in, then i attempted to reproduce it differently, by using the footsteps sound alone... nothing surpring this way, though then even shots were not sufficient to help AI get my position (at this point one would expect the same AI immediate reaction).

Notice at 1m22secs the angle (try a frame by frame check) at wich the AI actually pinpoint my exact location appears to be within 180 degrees.

Edited August 27, 2013 by gammadust

[maturin 12]

I never meant to suggest that the AI ignores FOV entirely, just that he detects you from a good 90 degree offset. Since this detection principle of checking LoS under fire can operate at long range (as seen in Bad Benson's videos), detecting new targets with the farthest corners of peripheral vision is highly undesirable. It's not how human peripheral vision would work.

Here is a visual aid for anyone who doesn't have time to repro: http://steamcommunity.com/sharedfiles/filedetails/?id=173436858

That is a screenshot of the moment when the AI 'sees' you, using what I believe to be an LoS check just within the outermost bounds of peripheral vision.

Edited August 27, 2013 by maturin

[gammadust 12]

I have moved the player even close to the enemy and without moving at all, I can get a knowsabout of 4 from 6 o'clock

I think i assume rightly you are on current dev branch... may you have left some mod interfering with your results? I can't reproduce the above :/, even from less than 1 meter at exact 6 o'clock. (i also remove the concrete wall)

[maturin 12]

I think i assume rightly you are on current dev branch... may you have left some mod interfering with your results? I can't reproduce the above :/, even from less than 1 meter at exact 6 o'clock. (i also remove the concrete wall)

No mods running, no. But I'm not worried about it anyways because I'm 95% sure he was just hearing me. AI have very good ears at close range, and there is already a high priority ticket addressing the issue.

I've edited all those bits out of my post. I think I was barking up the wrong tree. In short, I think they're spotting us with peripheral vision, and I don't think that should happen. The LoS check needs to be delayed, first of all. Only then can it be subjected to other limitations like nearby clutter and camouflage, and have perceived position area targets imposed.

After all, that is the final prize, and the solution to the single biggest AI spotting issue.

[gammadust 12]

Still on the AI vs AI let me make a correction:

I overlooked something in the contest script. allowFleeing 0 and enableAttack false are tied to the spawn position. So apart from the lower level modifications both AI teams use these settings for 3 times each. Should be corrected in next version.

Actually the version recorded has this updated already.

Edited August 28, 2013 by gammadust

[gammadust 12]

Ok an update that some of you may like. I am going on vacation for a week tomorrow (sunbeach time came a little late for me this year), so i decided to make a limited release of this project for those interested be able to experiment in my absence.

Download link

It is currently so experimental that despite open source so that anyone can take a good look at it, i am for that reason limiting derivative works. Feel free to inspire on it, but for now no modifications are to be distributed (this is under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License). When i am content with it's development the license shall be updated to a Creative Commons Attribution-Non-Commercial-Share Alike

The zip includes:

- Modified AI addon affecting only OPFOR units and dangerFSM debugging colored spheres (this makes units stop when threathened as in default but staying still until no threats are in sight) - Hardcoded

- Demo AI vs AI Contest mission (which puts up the different AI against each other for a number of times swaping sides)

(neither of which very different from those in the videos, just some minor updates and adaptation for current Dev build, should work on Release branch too)

Some notes on usage:

- Objectives and Spawn markers can be moved anywhere (explore different combat places) - see Markers (F6)

- A lot of AI settings are available (check GAM_setupScene function to play with those)

- Take a look towards the end of init.sqf to set some contest params (number of repeats and round timeout)

- Recommend going to spectator mode to observe it has dependent functionality:

press L to declutter the tags

press J to hide everything

- There is some additional functionality that i was exploring namely FSM execution

Point at unit + Press CTRL+SHIFT+Left Mouse Click to select a unit

Point at place + Press CTRL+Left Mouse Click to order it to move where you want

- This is all quite experimental might be easy to break

I should have some minimal access to the forum while away, but won't be able to give much support if at all until i'm back in a week or so.

So hope you enjoy AI experiments as much as i do (it is also interesting to remove my addon and just test different AI settings against each other - requires some scripting knowledge to edit affected AI).

[punkatux 10]

Possible BUG: nearTargets threat is also immediatly updated even when out of FOV (if and only if) it already exists in the nearTargets array (this could explain some exclamations)

that's approximation, by design.

[KeyCat 131]

I only had a quick look tonight before going to bed but it looks very interesting, will toy more later. Thanks for making and sharing!

/KC

[loki 1]

very impressive.. thx for releasing it. keep up the work.. following this thread and project.

EDIT: at 6m:39s in the first video

http://imageshack.us/photo/my-images/163/uly5.png/

is that LOS?.. the ai can track through multiple buildings?

look at the blu_f on the far left.. and his line goes through many objects to the ofp_f on the far right....

Edited September 22, 2013 by Loki

[gammadust 12]

^^ fortunately not, there are surprising detections out of expected fov and maybe los but not that worrisome :)

Those red and blue lines are returns from assignedTarget (blue for blufor units / red for opfor units). BF13_8 has OF13_4 (bottom right) as an assigned target.

Coverage script which would show LOS is deactivated at that point in time. Lines in that case are colored Red (unobstructed / out-of-cover) and Green (obstructed / in-cover)

Edited September 23, 2013 by gammadust

[loki 1]

^^ fortunately not, there are surprising detections out of expected fov and maybe los but not that worrisome :)

Those red and blue lines are returns from assignedTarget (blue for blufor units / red for opfor units). BF13_8 has OF13_4 (bottom right) as an assigned target.

Coverage script which would show LOS is deactivated at that point in time. Lines in that case are colored Red (unobstructed / out-of-cover) and Green (obstructed / in-cover)

thank you :) was worried for a minute.

[gammadust 12]

Just an update. I finally created the script to allow me to visuallize AI statistics in real time.

Preliminary version video demo:

I'll share the script when the major bugs are out. To be included in AIvsAI contest package, but it is pretty much abstracted from it to allow other usages, it will probably get it's own distribution, soon.

Edited November 10, 2013 by gammadust

[Dwarden 1125]

@gamma that's very nice

[gammadust 12]

^^ glad you enjoyed, i did not want to come back empty handed, so here's the update:

fabrizio_t (bCombat) has already an early version running, but i am hoping to release the latest this weekend. Remember this can be used for all kinds of analysis not just AI. So you statistics guys can experiment.

Also wanted to make mention of STHUD and his creator zx64 and Dslyecxi, code upon which showed this was possible and how.

Edited November 16, 2013 by gammadust

[Rydygier 1309]

Wow. I definitelly must reserve time to explore that.

[gammadust 12]

Happy to finally publish the mod. It was a bit demanding this latter code clean up, and a lot more could have been done, sorry to have you guys waiting...

GAM DataPlot (Statistics for all)

I should really create a small video tutorial, the whole setup and data connection is not absolutely trivial. Distribution includes a mission to get you started.

I hope it only requires small updates, but i do have some already in mind, i just can't wait to start messing with its possibilities.