ZBE_Cache AI & Vehicle caching script/addon

[zorrobyte 30]

ZBE Mod Version only install on Server? I thought this is one requirement and everyone who wants to improve his Performance have to run it, too.

Or do i misunderstand something?

Disregard last, as of v4.4b there is no difference between script and addon. Simply plop addon on server and be done.

Note

ZBE_Cache is still in it's infancy. Information subject to change as it matures and I have a lot of work to do to resolve now known issues.

---------- Post added at 10:14 PM ---------- Previous post was at 10:00 PM ----------

Questions:

Does anyone commonly change the friend/enemy of the independent faction mid mission?

Would it be inconvenient or weird if other friendly AI didn't uncache for players? - Maybe this needs to be a end user option.

---------- Post added at 10:57 PM ---------- Previous post was at 10:14 PM ----------

Latest testing results in 50CPS at 59FPS with 144 groups, 1145 AI which is a 25x improvement in code performance by using nearEntities and the new isEqualTo command in A3 1.16.

Edited November 25, 2014 by zorrobyte

[kremator 1065]

Absolutely fantastic mate ! Well done.

[zorrobyte 30]

Just need to figure out units in vehicle as nearEntities does not return this..

Using (((position _leader) nearEntities ["SoldierEB",_distance]) isequalto []) currently

Could do:

(((position _leader) nearEntities [["SoldierEB","LandVehicle","Air","Ship"],_distance]) isequalto [])

Of course, I will change "SoldierEB using a switch to the appropriate enemy superclass. This isn't optimal as AI will uncache for all vehicles regardless of empty/manned/side etc. Still working on a solution... Ideas?

Edited November 24, 2014 by zorrobyte

[Zriel 12]

I wish I could help, but I don't get what's the problem. Unmmaned vehicles do not have AI.

And about the friendly AI switch....AI uncaches bassed on distante isn't it?

[Rydygier 1317]

Got an idea, instead of building the players and enemy AI list every frame x amount of groups, I can decrease load by x times by writing a separate FSM that builds these lists.

General question. What is more CPU efficient: keeping amount of spawned sqf/fsm threads as low as possible, so we have eg only one big loop for slow calculation (and as much, as possible goes here) and one per frame (and here only, what absolutelly can't be calculated in the slow loop and simply must be per frame, like 3D icons drawing) or lots of small threads? I heard, the first is remommended, and I'm doing that way - one loop doing 100 things one by one, not 100 parallel loops, each doing one thing. But maybe I do not know something here.

Also constructing an array used in parallel theads can lead to dangerous cases, like working with the array, that at the same time is under construction. I'm usually avoiding parallel scripting as much as possible. Risk may be tiny as long CPU isn't loaded too much, but such code design makes the script very vulnerable on any CPU overload, that could cause delays in script execution. FSM are a bit another story though.

[redarmy 422]

Thanks for work on this so far zorrobyte.

Regarding your question: Would it be wierd to see the friendly AI cache/uncache spawn- I would have to say yes.Playing primarily SP,and focusing on building missions incorperating large scale battles,and large troop movements,as part of the ambience,i would feel taken out of the immersion should i see units popping in and out of reality;)

[zorrobyte 30]

So I have been thinking.

1. Always uncache for players within cache distance

2. Uncache for AI to AI encounters, doesn't have to be a set distance as long as they can fight

nearEntities coud work, I can build an array with the near vehicles, check for any units in cargo then find the side of the units. This could be heavy and makes code complex. The last thing you want is a heavy check running per frame multiplied by every group on the server.

I got to thinking though, what's the use of AI-AI uncaching if they are on the other side of a hill within cache distance or otherwise not going to engage anyway?

So I started checking out findNearestEnemy. This will return an array if the group knowsAbout another enemy. Knowing about an enemy is a prerequisite to engaging an enemy. "But what if the AI group would KNOW about the other AI group if his buddies were uncached but now can't as it's only the leader in the AO?" - From initial testing, if I don't run disableAI commands, even if the group AI are cached they can still "see", set targets and so on as the targeting FSMs are still running. For example, set cache range to very low and shoot the leader. Move into uncache range and you'll find the whole group is lighting you up instantly.

findNearestEnemy testing (144 groups, enemies flying overhead - 1108AI):

Youtube(perspective as AIvsAI)

http://youtu.be/u1tgzi3i6XU

Edited November 24, 2014 by zorrobyte

[kremator 1065]

Gimme, gimme !!!

[zorrobyte 30]

Need to figure out this first :P

[zorrobyte 30]

4.4 has been released!

Changelog:

• Epicly faster performance! Rewrote cache and uncache conditions to be 25x faster. Now CPS is almost native to FPS (no more script lag). 144 groups 1153AI with 59.8CPS/60FPS
  
• Bumped -1 auto FPS values to 16 dedi 31 client for a target of 15/30
  
• Players continue to uncache always within distance set but now AI only uncache for other AI groups IF they are aware of each other (using findNearestEnemy). This keeps AI cached if they have nothing to shoot at and will lead to more overall AI being cached in AIvsAI missions
  
• Cache FSM now exits for player led AI groups
  

Download

Edited November 25, 2014 by zorrobyte

[Zriel 12]

Testing XDD

---------- Post added at 00:49 ---------- Previous post was at 00:23 ----------

Much better. And must say dedicated and game are on the same Machine.

---------- Post added at 01:15 ---------- Previous post was at 00:49 ----------

FUUCK...there's a bug

Units chached in buildings do not spawm correctly, they are spawned outside buildings.

This is a photo of units without ZBE_Cache

http://i.gyazo.com/832805238b308680a2a2f4ab0bcdb443.jpg (734 kB)

Same building, cached

http://i.gyazo.com/3f4a3ce97b1015dfa367885e884f41db.jpg (590 kB)

Same building uncached (notice the soldier in the floor)

http://i.gyazo.com/96b9c914f1ae52edb16403787612e13b.jpg (710 kB)

Should say more than "outside", should be "same position, on the floor."

Got some units in buildings, jerking between flor of spawmed building and terrain.

Edited November 25, 2014 by Zriel

[zorrobyte 30]

Testing XDD

---------- Post added at 01:15 ---------- Previous post was at 00:49 ----------

FUUCK...there's a bug

Units chached in buildings do not spawm correctly, they are spawned outside buildings.

This is a photo of units without ZBE_Cache

http://i.gyazo.com/832805238b308680a2a2f4ab0bcdb443.jpg (734 kB)

Same building, cached

http://i.gyazo.com/3f4a3ce97b1015dfa367885e884f41db.jpg (590 kB)

Same building uncached (notice the soldier in the floor)

http://i.gyazo.com/96b9c914f1ae52edb16403787612e13b.jpg (710 kB)

Should say more than "outside", should be "same position, on the floor."

Got some units in buildings, jerking between flor of spawmed building and terrain.

Not so much as a bug but more of a "lack of functionality". The units are setPosASL to formationPosition of the leader while cached so when the group uncaches they aren't wherever they were when the group was cached. This allows cached groups to still be damaged by artillery, bombs, etc.

Building sensitive caching isn't impossible to do but needs work and time, something I'm short of at the moment. I wouldn't mind adding a dev or two.

Thank you for the screenshots!

Edited November 25, 2014 by zorrobyte

[Zriel 12]

XD lol

Though, this was working OK in 4.3. I'm mistaken?

[zorrobyte 30]

XD lol

Though, this was working OK in 4.3. I'm mistaken?

If it did, it would of been a "bug". I didn't change any code other than the cache/uncache conditions for CPS. All versions are located at the download link in the first page

[zorrobyte 30]

v4.4a

• Fixed next leader uncache loop, was repeating indefinitely in some cases
  

[Zriel 12]

If it did, it would of been a "bug". I didn't change any code other than the cache/uncache conditions for CPS. All versions are located at the download link in the first page

Weeeeeell...it is a bug. I've testing with v4.3 and it's working correctly. Units spawn in their original position.

Note that this units are static (have not moved) so it does not make sense they should change their value if you are storing the 3 values of setPosASL.

---------- Post added at 09:56 ---------- Previous post was at 09:41 ----------

And got it working again, you were using setPos in the past for the position of units, not setPosASL. Changed it back to setPos (you should check why it was changed, maybe I'm doing wrong.)

---------- Post added at 09:57 ---------- Previous post was at 09:56 ----------

Looks like the occasionnal stutter/FPS drop is gone!

Very nice work with this - thanks man!

EDIT : I took another look at your script and I saw one tiny thing you could change in main.sqf ^^

You might want to replace the last part :

[] spawn {
while {true} do {
	uiSleep 15;
	if (zbe_debug) then {
			zbe_stats = format ["%1 Groups %2/%3 All/Cached Units %4/%5 All/Cached Vehicles %6 FPS %7 ObjectDrawDistance", count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
			diag_log format ["%1 ZBE_Caching (%2) # %3", (round time), name player, zbe_stats];
			hintSilent format ["%1 ZBE_Caching # %2", (round time), zbe_stats];
		};
};
};

by this :

[] spawn {
if (zbe_debug) then {
	while {true} do {
		uiSleep 15;
		zbe_stats = format ["%1 Groups %2/%3 All/Cached Units %4/%5 All/Cached Vehicles %6 FPS %7 ObjectDrawDistance", count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
		diag_log format ["%1 ZBE_Caching (%2) # %3", (round time), name player, zbe_stats];
		hintSilent format ["%1 ZBE_Caching # %2", (round time), zbe_stats];
	};
};
};

Because as it is right now, if you choose to disable zbe_debug at mission start, you have a loop constantly checking for a condition that probably won't change once the script/mission is launched.

Another thing I noticed : sometimes zbe_cachedUnits shows negative values; in the last run I had something like 10 units for -2 cached units.

The script was launched with [1000,15,true,1000]execvm "zbe_cache\main.sqf", I was running a mission where I delete units further than 1500 m from the player. Note that I was flying fast with a chopper; maybe one of the FSM failed to catch up? Five minutes later I had 7 units and -1 cached unit.

Also, remember to update this part! it seems you have still left it as it is XDDD

[champ-1 40]

Edit: nevermind.

Good stuff though, thanks.

Edited November 25, 2014 by Champ-1

[Zriel 12]

Regarding the issue with positioning in buildings: It is working correctly in the editor (using setPOS) but there are some issues in dedicated. Some units spawn correctly but others not (other even spawn dead??)

I would say that this is because you use setPos (or setPosASL) in both server and machines. those commands have global effect, meaning that everyone is dictating where the unit should "uncache". This probably is making some contradictions when clients and server do not agree on where the unit should be. That's why I asked on "where the script should be running".

I would like to test (this evening) for a change in code, use hideobjectglobal and enablesimulationglobal, and run this only in server (much very like the addon version).

Also, allied AI is not caching? It should be too isn't it?

[whiztler 137]

Looking great. Really impressed with the cycle improvements.

I have fancied-up the debug hint a bit to make it more readable, thought it would share it (updated every 5 seconds):

[] spawn {
   while {true} do {
       uiSleep 5;
       if (zbe_debug) then {
           hintSilent parseText format ["
               <t color='#FFFFFF' size='1.5'>ZBE Caching</t><br/>
               <t color='#FFFFFF'>Caching debug Data</t><br/><br/>
               <t color='#A1A4AD' align='left'>Game time in seconds:</t><t color='#FFFFFF' align='right'>%1</t><br/><br/>                
               <t color='#A1A4AD' align='left'>Number of groups:</t><t color='#FFFFFF' align='right'>%2</t><br/>                
               <t color='#A1A4AD' align='left'>All units:</t><t color='#FFFFFF' align='right'>%3</t><br/>
               <t color='#A1A4AD' align='left'>Cached units:</t><t color='#39a0ff' align='right'>%4</t><br/><br/>
               <t color='#A1A4AD' align='left'>All vehicles:</t><t color='#FFFFFF' align='right'>%5</t><br/>
               <t color='#A1A4AD' align='left'>Cached vehicles:</t><t color='#39a0ff' align='right'>%6</t><br/><br/>
               <t color='#A1A4AD' align='left'>FPS:</t><t color='#FFFFFF' align='right'>%7</t><br/><br/>
               <t color='#A1A4AD' align='left'>Obj draw distance:</t><t color='#FFFFFF' align='right'>%8</t><br/>
           ",(round time),count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
           zbe_log_stats = format ["Groups: %1 # All/Cached Units: %2/%3 # All/Cached Vehicles: %4/%5 # FPS: %6 # ObjectDrawDistance: %7", count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
           diag_log format ["%1 ZBE_Caching (%2) ---  %3", (round time), name player, zbe_log_stats];    
       };
   };
};

[haleks 8212]

[] spawn {
   if (zbe_debug) then {
       while {true} do {
           uiSleep 15; 
           hintSilent parseText format ["
               <t color='#FFFFFF' size='1.5'>ZBE Caching</t><br/>
               <t color='#FFFFFF'>Caching debug Data</t><br/><br/>
               <t color='#A1A4AD' align='left'>Game time in seconds:</t><t color='#FFFFFF' align='right'>%1</t><br/><br/>                
               <t color='#A1A4AD' align='left'>Number of groups:</t><t color='#FFFFFF' align='right'>%2</t><br/>                
               <t color='#A1A4AD' align='left'>All units:</t><t color='#FFFFFF' align='right'>%3</t><br/>
               <t color='#A1A4AD' align='left'>Cached units:</t><t color='#39a0ff' align='right'>%4</t><br/><br/>
               <t color='#A1A4AD' align='left'>All vehicles:</t><t color='#FFFFFF' align='right'>%5</t><br/>
               <t color='#A1A4AD' align='left'>Cached vehicles:</t><t color='#39a0ff' align='right'>%6</t><br/><br/>
               <t color='#A1A4AD' align='left'>FPS:</t><t color='#FFFFFF' align='right'>%7</t><br/><br/>
               <t color='#A1A4AD' align='left'>Obj draw distance:</t><t color='#FFFFFF' align='right'>%8</t><br/>
           ",(round time),count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
           zbe_log_stats = format ["Groups: %1 # All/Cached Units: %2/%3 # All/Cached Vehicles: %4/%5 # FPS: %6 # ObjectDrawDistance: %7", count allGroups, count allUnits, zbe_cachedUnits, zbe_allVehicles, zbe_cachedVehicles, (round diag_fps), zbe_objectView];
           diag_log format ["%1 ZBE_Caching (%2) ---  %3", (round time), name player, zbe_log_stats];    
       };
   };
};

Fixed! :rolleyes:

[Guest]

Thanks for informing us about the updated version :cool:

New version frontpaged on the Armaholic homepage.

ZBE Caching v4.4a

================================================

We have also "connected" these pages to your account on Armaholic.

This means soon you will be able to maintain these pages yourself if you wish to do so. Once this new feature is ready we will contact you about it and explain how things work and what options you have.

When you have any questions already feel free to PM or email me!

[zorrobyte 30]

And got it working again, you were using setPos in the past for the position of units, not setPosASL. Changed it back to setPos (you should check why it was changed, maybe I'm doing wrong.)

I was working from the code optimization guide, trying to squeeze out every bit of improvement. It's GREAT that regular ole setPOS works and will change back immediately. Thank you for pointing this out! I should of known better.

I would say that this is because you use setPos (or setPosASL) in both server and machines. those commands have global effect, meaning that everyone is dictating where the unit should "uncache". This probably is making some contradictions when clients and server do not agree on where the unit should be. That's why I asked on "where the script should be running". I would like to test (this evening) for a change in code, use hideobjectglobal and enablesimulationglobal, and run this only in server (much very like the addon version).

I spend 95% of my Arma time on the BIKI, editor and Notepad++ and really appreciate the feedback as it can be easy to miss things for MP. Your solution makes total sense as setPOS is global and I didn't think about the ramifications. I may kill the dual FSM idea and only run from isserver using enablesimulationglobal, this will also make updates faster to crank out as it's a pain to maintain 4 separate FSMs. I can do a quick check for isserver for disableAI or simply do some testing with ASM to see if these commands make any actual difference now in A3 (as if ran on client, can break animations). As a footnote, I can do the same with empty vehicle caching and let it run from the server so the old addon script difference is moot.

Looking great. Really impressed with the cycle improvements.

I have fancied-up the debug hint a bit to make it more readable, thought it would share it (updated every 5 seconds):

You rock, thank you for the code! I'm glad you are having success with the cycle improvements! I feel awesome having hit two birds with one stone by optimising AI<->AI engagements while solving a seemingly impossible problem with load. It was taking 0.0231934ms to check the old ({((_position distance _x) < _distance) } forEach ([] call BIS_fnc_listPlayers)) while multiplied by 144 times and looping every second - total was 3.33 seconds to complete work that needs to be done in 1 second.

I couldn't thank everyone enough for the feedback and assistance with ZBE_Cache! It wouldn't be anywhere near where it is today without you all and it's exactly why I release my work under an open Creative Commons licence for all to use, modify and redistribute.

[Zriel 12]

Thanks to you for your dedicated work and ideas. I think that you are the only one approaching the idea of caching using hideobjects instead of deleting units.

Which makes me think...why?

For now it's working quite well, so we will have to see in the future.

BTW, still dunno why my allied IA is not being cached.

[Wolfenswan 1]

Thanks to you for your dedicated work and ideas. I think that you are the only one approaching the idea of caching using hideobjects instead of deleting units.

Which makes me think...why?

For now it's working quite well, so we will have to see in the future.

BTW, still dunno why my allied IA is not being cached.

Hideobject isn't that relevant when it comes to caching, especially for MP, as it's the CPU-heavy tasks which will tank perfomance. Disabling simulation and AI or deleting/restoring AI are still the best options here.

I would like to test (this evening) for a change in code, use hideobjectglobal and enablesimulationglobal, and run this only in server (much very like the addon version).

F3 caching is very similar to Zorrobytes (given the same source of inspiration) but has been using the global commands for a while now. From the tests we've conducted there wasn't any difference in the perfomance improvement compared to the non-global variant. A thing to keep in mind is that uncaching many AI at once (i.e. without a miniscule sleep between units) will negatively affect perfomance in any case.

Building sensitive caching isn't impossible to do but needs work and time, something I'm short of at the moment. I wouldn't mind adding a dev or two.

For F3's caching I've simply added a check like this:

// If the group leader is moving, set his group back next to him
if (speed leader _this > 0 && vehicle _x == _x) then {
   	_x setPosATL (formationPosition _x);
};

This assumes that the group leaders of a garrisoned group will be inside a building and thus immobile. It's not the perfect solution but given the alternatives (checking for e.g. nearestObject) it seemed to be the least expensive.

Edited November 25, 2014 by Wolfenswan

[Zriel 12]

BTW2 Notepad++ it's quite a good editor, but when managing multifolder/files projects I feel it lacks of functionality compared to the BI editor /Poseidon Tools.

For example, it comes with a handy "search in folder", which made the change from SetPosASL to SetPos easy and fast.