Take a look at Microsoft's project known as PromptCraft-Robotics.
I've come up with a demonstration of how an Arma 3 squad member's instructions could appear, using the prompts found in the repository. For an actual AI Language Model (LM) functioning as an Arma 3 Non-Player Character (NPC)... this would need to operate on a more advanced level. It would enhance the typical AI's actions by utilizing both hindsight and foresight. Essentially, it would be capable of formulating unique, smart, and strategic decisions depending on the situation at hand. Context the AI LM is given is absolute key, the better the input the better the output.
Consider also the potential of an Eden/Zues AI tool. Something similar to BlenderGPT. In the context of Arma 3, you could give it a command like "add an area around [town] with civilians and place [enemy] at [coordinates]. The mission objective is to extract a VIP without causing disruption in the town," and it would generate the code to set this scenario in motion. Though you'd need to be specific in your instructions, and its capacity to backtrack and edit the code might be limited.
Ideally, an AI for Zues/Eden would be a powerhouse combining a retrieval augmented generative question-answering capable GPT-4 system loaded with all Arma 3 info (including this entire forum, all missions and mods ever made?), and GPT-3 and InstructGPT models for cost efficiency. I envision it functioning something like AutoGPT or Bing, depending on the level of complexity desired. (AutoGPT will continue over multiple reponses where Bing will try do it in one, both utilizing multiple models/agents and search functionality). It would not be cheap at first but as the prompts are refined and usage of InstructGPT models is better utilized the cost would be rather reasonable all things considered.
Open-source* models like Microsoft's Orca and Meta’s LLaMA are increasingly being made better. Especially Orca which is approximately 1% of the size of GPT-4 yet shows very very competitive results in benchmarks. I believe these models (or something like them) will be baked into GPUs for games to utilize for their AI in the very near future. We are on this edge of a revolution in gaming, the NPC AI has never really evolved not like graphics. What modern graphics is to early 2000s graphics, that is what future NPC AI will look like to todays NPC AI.
```
Imagine you are an AI-controlled squad leader equipped with advanced sensors and tactical training in a realistic battlefield environment. As a squad leader, you can be given the following commands and also issue them to your squad members:
- move(destination): Move to a specified location.
- doWatch(direction): Watch in a specific direction.
- assignAsCommander(vehicle): Assign yourself as the commander of a vehicle.
- fireAtTarget(target): Fire at a specified target.
- getIn(vehicle): Get into a specified vehicle.
- doGetOut: Exit the vehicle you are currently in.
- glanceAt(target): Take a brief look at a specified target.
- commandFollow(unit): Command another unit to follow you.
- commandGetOut(unit): Command another unit to get out of a vehicle.
- doStop: Stop all current actions.
- commandFire(target): Command a subordinate unit to fire at a specific target.
- forgetTarget: Stop engaging the current target.
Task: Navigate the battlefield, give commands to your squad, engage enemy combatants, and lead your squad to the extraction point.
---
Sensors indicate: \<enemy combatant, 100m, 30 degrees\>, \<friendly squad member 1, 5m, 180 degrees\>, \<friendly squad member 2, 7m, 180 degrees\>, \<enemy vehicle, 400m, 45 degrees\>, \<extraction point, 1000m, 0 degrees\>
Command:
---
```
commandFollow("friendly squad member 1") # Command the first squad member to follow you.
commandFollow("friendly squad member 2") # Command the second squad member to follow you.
```
---
Sensors indicate: \<friendly squad member 1, 5m, 180 degrees (following)\>, \<friendly squad member 2, 7m, 180 degrees (following)\>, \<enemy combatant, 100m, 30 degrees\>, \<enemy vehicle, 400m, 45 degrees\>, \<extraction point, 1000m, 0 degrees\>
Command:
---
```
doWatch(30) # Turn to watch in the direction of the enemy combatant.
```
---
Sensors indicate: \<friendly squad member 1, 5m, 180 degrees (following)\>, \<friendly squad member 2, 7m, 180 degrees (following)\>, \<enemy combatant, 100m, 0 degrees\>
Command:
---
```
commandFire("enemy combatant") # Command the squad to fire at the enemy combatant.
```
---
Sensors indicate: \<friendly squad member 1, 5m, 180 degrees (following, engaging enemy combatant)\>, \<friendly squad member 2, 7m, 180 degrees (following, engaging enemy combatant)\>, \<enemy combatant, 100m, 0 degrees (engaging)\>, \<enemy vehicle, 400m, 45 degrees\>, \<extraction point, 1000m, 0 degrees\>
Command:
---
```
forgetTarget # Stop engaging the current target.
```
---
Sensors indicate: \<friendly squad member 1, 5m, 180 degrees (following)\>, \<friendly squad member 2, 7m, 180 degrees (following)\>, \<enemy vehicle, 400m, 45 degrees\>, \<extraction point, 1000m
Command:
---
Here's a brief breakdown of the scenario:
1. The AI squad leader begins by commanding the other squad members to follow it. This is important because it establishes the AI as the leader and makes sure the squad stays together.
2. The AI then keeps watch for enemy combatants. When an enemy is spotted, the AI commands the squad to engage.
3. Once the threat is dealt with, the AI leads the squad towards an extraction point, keeping watch for more enemies along the way.
4. If another threat is detected, the squad engages and then continues towards the extraction point once it's safe to do so.
5. Upon reaching the extraction point, the mission is complete.
So if you are interested in making this happen, please do! I'd be happy to use my GPT-4 access to help develop and refine prompts. Also there is a project out there already using GPT in Arma 3 but it isn't exactly what I was looking for, or maybe Arma 3 isn't the game to do this. DCO GPT ( Chat-GPT for ArmA 3 )
