[WIP] ZBE_Cache (caching addon/script)

There are a number of caching scripts/addons out there. Some delete/respawn units, some enablesimulation false and hide units and some even go so far as to disableai/enableai. The issue I have with these scripts/addons is they either spam netcode with delete/respawn (and effect addons/scripts like HETMAN/HAL negatively) or they hide the units which make them static (and unable to move unless uncached, further effecting HETMAN/Waypoints).

I mention HAL/HETMAN. This is Rydyger's artificial leader script/addon and I have been following its development for quite some time. Regardless, there are other AI scripts/addons out there that can mount empty vehicles/static weapons (I think, or at least in A2 there was). I hope that by working on this caching module, that other devs can finally start to use "ambient" vehicles for AI.

In Arma, yes. Even if objects are 10,000m away vehicles still simulate Physx, sound eventhandlers loop checking for an opportunity to fire and play sound. Distant AI with no enemy near keep group count up, needlessly eating resources - and so on and so on. Each issue by itself isn't that big of a deal, multiply this by 100-1,000 times and you have some performance issues.

Another benefit to such a caching module, enablesimulation false disables packets being sent for that object to clients. We all know that Netcode is an issue for MP, why should empty vehicles and AI with no enemy near (or near player) continue to send network updates if no one is around to see it or use the object.

Where does ZBE fit in all of this, how does it help performance without spamming netcode or preventing AI movement when cached?

AI:

Reduce groups down to Leader using:

                _x disableAI "TARGET";
               _x disableAI "AUTOTARGET";
               _x disableAI "MOVE";
               _x disableAI "ANIM";
               _x disableAI "FSM";
               _x enableSimulation false;
               _x allowDamage false;
	_x hideobject true;

When enemy unit near or in player viewdistance:

                _x allowDamage true;
               _x enableSimulation true;
               _x hideobject false;
               _x enableAI "TARGET";
               _x enableAI "AUTOTARGET";
               _x enableAI "MOVE";
               _x enableAI "ANIM";
               _x enableAI "FSM";
               _x setPOS (or findsafeplace or other not yet known way) (position leader)

Unneeded AI are stopped in their tracks, disable simulation in all aspects. Once an enemy unit is near or spotted/combat/leader shot/etc(still thinking of uncache reasons) the rest of the group is uncached and moved back to the leader's position.

That in itself sounds like a great way to increase performance for AI. However, thought needs to go into how to assess of a side is enemy/friendly for independent groups as this can change depending on mission settings.

Another method for triggering caching could be diag_fps. Why loop through caching scripts needlessly if server/client fps is greater than X?

Vehicles

I was inspired by TPW_Park for a way to spawn in ambient vehicles for HETMAN use. I didn't like how TPW_Park deleted and spawned in vehicles dynamically (as in MP, this would spam packets) however at the time it was the only way to do so without performance issues.

I found that when spawning in my desired vehicles that Physx and class Sounds killed FPS as they didn't disable when enablesimulation false. Bravo to BIS for fixing this issue in about a week's timeframe (http://feedback.arma3.com/view.php?id=18451). My issue with current vehicle caching methods is that they don't uncache/spawn until a PLAYER is near. When using HETMAN/HAL with these types of caching, the vehicles stay cached and AI who normally could choose to man a empty vehicle simply cannot as the vehicle isn't uncached for them. The same could go for static weapons and so on depending on how the caching script works.

This is a rough proof of concept I have used in my performance testing:

//Caching loop for LandVehicle
[]spawn {
while {true} do
{
	{		
	if (!(count ((position _x) nearEntities ["CaManBase",1000]) == 0) || !(count (crew _x) == 0) || !(count ((position _x) nearEntities ["Air",1000]) == 0)) then {
	_x hideobject false;
			if (count ((position _x) nearEntities ["CaManBase",500]) == 0)  then {
				if (count (crew _x) == 0) then {_x enablesimulation false} else {_x enablesimulation true};
			} else {
				_x enablesimulation true;
				};
	}else{
	_x hideobject true};
	 } count entities "LandVehicle";		
	sleep 1;	
};
};

Vehicles stay cached until a CaManBase (including AI) is near or inside the vehicle. So far this works well. Damage handling hasn't been added, what if one fires artillery at the area? The vehicles will be damaged but won't show (or blow up) until the player is near resulting in undesired death and visual effects.

Some solutions are available like "hit" eventhandlers and so on however one must assess the performance impact before haphazardly adding it into the caching module. Keep in mind, my goals for this module is to spawn in a few hundred vehicles all across the map during mission start and small performance impacts add up quicky.

This is a WIP and I'm dedicated to the project. Stay tuned for updates.

We also have some great new commands in A3:

https://community.bistudio.com/wiki/hideObjectGlobal

https://community.bistudio.com/wiki/enableSimulationGlobal

Nice work so far Zorrobyte! Looking forward to using thousands of AI on the map. How could this be implemented for DAC?

Nice work so far Zorrobyte! Looking forward to using thousands of AI on the map. How could this be implemented for DAC?

It will work for any AI on the map spawned in by any means however DAC already has its own caching methods however it does spawn/despawn units which can impact netcode.

Also, I just got a HUGE head start thanks to Nou! I have permission to build upon his work and so far I have made some tweaks for A3. The biggest addition will be vehicle caching as even if vehicles are local to server, they still simulate Physx and sound eventhandlers crippling FPS.

Will be script and addon versions.

Very nice. I'm sure that BIS will look into the PhysX and sound eventhandlers eventually. Anything that gives high and sustained FPS with huge numbers of AI on the map is a 'good thing'TM

Correct me if I'm wrong then but I can have 10 groups of AI, that will collapse to 10 group leaders (keeping all waypoints, current states etc) that, only when enemy AI are spotted (perhaps cached as well) will pop back into existence and fight (or react appropriately)

Thanks so much for this (and thanks Nou!)

Very nice. I'm sure that BIS will look into the PhysX and sound eventhandlers eventually. Anything that gives high and sustained FPS with huge numbers of AI on the map is a 'good thing'TM

Correct me if I'm wrong then but I can have 10 groups of AI, that will collapse to 10 group leaders (keeping all waypoints, current states etc) that, only when enemy AI are spotted (perhaps cached as well) will pop back into existence and fight (or react appropriately)

Thanks so much for this (and thanks Nou!)

Correct, 10 groups of ai are cached into 10 units (leader). The additional units are simply "frozen" and the engine stops simulation for these (and server stops sending network updates). Once the leader is within 1,000m of another enemy AI (and vice versa) the leader's units are "unfrozen" and teleported back into formation. The only modification so far to AI behavior is that I must setspeedmode "FULL" to the leader after his buddies are cached as otherwise he will slow instead (waiting for his frozen units to catch up). Shouldn't impact behavior too much, upon "unfreeze" the group/leader is set back to "NORMAL" speed. I'd like to explore if one could store the setspeedmode state and then reapply such state once the units are "unfrozen" for higher fidelity. Additional info on setspeedmode: https://community.bistudio.com/wiki/setSpeedMode

WIP:

I added a 3 second wait for uncached units for allowdamage as I found that AI that were uncached on a slope fell to their death. Such is implemented using a []spawn so it won't slow down uncache loop/function. I'm exploring enablesimulationglobal/hideobjectglobal as these commands are new to A3 (which brings interesting situations such as the server could cache units local to a HC or on player machines and older enablesimulation only worked where object was local). With global commands, no code/addon would need to run on client (or slow client for loop) with an addon only on server if one so desired. One must still do performance testing as to balance loops/netcode spamming.

Vehicle caching has MASSIVE improvements to client FPS, still figuring out reliable methods.

Thinking of an "emergency" protocol where if server FPS drops under 20 that the caching module takes more extreme measures to try and keep the server running. Such could cache leaders and vehicles semi-permanently until server FPS improves. Such brings ideas as to dynamic AI cache distance by itself where a averaged diag_fps loop dictates AI spawn distance and how many AI are uncached. Such could even be used for group strength depending on how many players are connected. Lots of ideas, must not let them delay the project for too long ;)

Hey man.

I am currently active in the same sort of sports for caching in MCC. You can check the results so far in the addon. A lot of stuff you type I can relate to and did quite some research (and still am) myself. If you like we can have some teamspeak chat soon to see if we can learn some stuff from each other.

Anyway, have fun on your project man!

*edit*

Check the last part of video about our caching. The caching we do is also 2 parts. First it does some stuff to non leads and then it completely caches/uncaches.

Hey man.

I am currently active in the same sort of sports for caching in MCC. You can check the results so far in the addon. A lot of stuff you type I can relate to and did quite some research (and still am) myself. If you like we can have some teamspeak chat soon to see if we can learn some stuff from each other.

Anyway, have fun on your project man!

*edit*

Check the last part of video about our caching. The caching we do is also 2 parts. First it does some stuff to non leads and then it completely caches/uncaches.

Wouldn't mind speaking with you on TS, MCC has always been a useful mission dev tool for me. PM TS info.

Interesting stuff guys.

Hey, you can find me on : tseu4.vilayer.com:10007. No password. It is not a secret so fully open :) There is usualy me or somebody around that knows when I will be online.

So far I've seen gains of 23-26FPS with 1145 units however much like the (now resolved) vehicle enablesimualtion issue, I feel as if the engine could do better.

Filed a ticket: http://feedback.arma3.com/view.php?id=18734

What it comes down to is:

1. Waiting to see if the enablesimualtion issue is resolved within a reasonable timeframe

2. Switching to a delete all but leader cache type

3. Coming up with a hybrid system which enablesimulation false units then deletes units if server FPS drops below X

The "packet spamming" is a perception that is not yet verified by my own testing however others have been concerned about it since A2. I suppose I need to whip up a module and see.

Progress is being made, now deleting and creating units instead of enablesimulation for the time being.

Working on caching units in vehicles and storing/restoring loadouts.

So it will work with scripts like UPSMON and will cache units when player will not be nearby, like in DAC?

How is this different / better than simulation manager?

How is this different / better than simulation manager?

The difference is the leader isn't cached meaning the group is free to continue way points. The same goes for units in vehicles, the driver remains while the commander, gunner and cargo is cached. This way groups continue moving and are able to be controlled by AI mods/addons such as HAL instead of being frozen in place.

The other big difference is that Simulation Manager is player centric, AI are only uncached for the player. ZBE_Cache keeps the group leaders uncached and uncaches AI for groups near enemy AI. This way AI can continue fighting even if far from player.

As this ticked was fixed:

http://feedback.arma3.com/view.php?id=18451

Refining a method to increase performance by caching vehicles as Physx and sound eventhandlers continue to use client resources no matter where it is local to.

Waiting for:

http://feedback.arma3.com/view.php?id=18734

To be fixed/improved as I would prefer to enablesimulation over delete/create vehicle as enablesimulation doesn't spam packets. Also with AI addons/scripts such as HAL enablesimulation doesn't effect unit count meaning Leader "morale" doesn't drop. Enablesimulation also decreases code complexity as storing unit's gear and reassigning upon createvehicle isn't required.

Clientside FSM done.. Moving onto isDedicated FSM next. This is for Vehicle Caching only, the AI caching is done.

http://i.imgur.com/68axQJl.jpg (187 kB)

I found that one could cache nonlocal vehicles (such as local to server) for additional performance savings.

The client loop only cares if the vehicles have cargo if they are local, else if outside viewDistance they are cached. I'm figuring out the server loop now as vehicles need uncached if near player else client thinks vehicle is at POS while is actually at POS local to server.

Sorry if not explained properly, is complex 0_o

Good man! This is looking very nice!

Sounds like a very, very interesting project you got there zorrobyte :)

Quick question, will SP game be affected to or is this only for multiplayer purposes?

Once more wish you the best:)

This will work for SP and MP

Script and addon versions for use with any mission/campaign, etc. I've yet to test with DAC or other missions that delete/respawn units. Should still be just fine however possibly redundant.

Here is alpha version. NOT for release on armaholic, six, etc yet. Needs testing!

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

https://www.dropbox.com/s/zbfdnps1vdl4ahy/%40zbe_cachealpha.zip

Discarded local/unlocal caching for now in favor of if vehicle has crew and inside/outside draw distance as locality issues.

Need to document/cleanup for an actual release, get the repo ready and setup bugtracker.

Report issues in this thread for now if you could.

Caching is a bit wonky when TL dies when units are cached. Looking into a solution.

Hope you find the solution zorrobyte so this caching is very good.

Will there be option to make AI distribute to Headless Client?

Version 1 available on my repo:

https://bitbucket.org/zorrobyte/zbe_cache/downloads

Still needs testing. This version uncaches the next TM if the TM dies while group is cached.

As for HC, possibly in a future version or another addon however since setowner needs expansion to save WP etc between setowner it doesn't seem feasible to do right now without a lot of effort. One could use HETMAN or another flexible AI management system that would "reinit" the AI back into a system for assignment of WPs and so on but isn't a 1:1 solution.

Have been occupied playing Space Engineers, let me know of any faults!

Thanks zorrobyte. There must have been a pain in the assignment bugging you to release this! Testing now.

Is this compatible with Arma 2/CO?

Not tried TeilX. Works very well for A3 :)

Hi guys,

Sorry for reviving an old thread. Just wondering of the current status of the project? Is it still needed as many official patches were released since?

Also if someone knows if it is abandoned or just switched to another thread?

EDIT: Ok I have found a version from this February so I assume I just need to look further for the new thread.

EDIT2: Found it thx