Hypnomatic's ARMA 3 Scripting Libraries

[Hypnomatic 10]

Hello all! I'm here to finally release the first batch of functions I've written for ARMA 3. I've been slowly accumulating a list of utilities and functions to solve miscellaneous tasks or problems since the release of the Alpha. A little while ago I decided I wanted to clean them up, document them well, and release them for all to use. One simple mini-wiki later and we're here!

The full documentation, including downloads and installation instructions can be found HERE, however allow me to share a few of the highlights:

• Functions to add escape characters to strings, as well as strip them
  
• Functions to manipulate and handle arrays by reference more easily
  
• Functions to emulate a Key-Value (Associative) Array, complete with JSON encoding and decoding
  

I am releasing all of this code under the very permissive BSD license, so you are more or less free to whatever you wish with this code. Use it, modify it, distribute it with your missions, it's just about all good! (Credit is always appreciated however.)

If you have any issues, suggestions, comments, or criticism, please feel free to post it here! This is my first release of anything to quite this scale, so suggestions for improvement on the code, the site, etc. are very much so appreciated!

Enjoy!

[Fenumeher 1]

Thank you for your work and effort! I am sure a lot of people will be helped with this!

[Hypnomatic 10]

Okay, so now that it's a bit more reasonable of a time, I figure it's a good time for me to ramble off about some of the inner workings of this, if for no other reason for the sake of logging it for when I revisit this:

Naming Conventions

I used to follow the general naming convention of hyp_fnc_[functionName] for naming just about all of my functions, however my list of functions grew, and I wanted to make the name more directly reflective of its purpose and behavior. I'm also a sucker for descriptive prefixes/suffixes, so I adopted something to the effect of: hyp_fnc_[category]_[functionName]. This worked pretty well for me, however it was a bit too much typing for my taste. I finally decided that I really didn't care if I had a hyp_ prefix for every function, and decided to settle on [category]_fnc_[functionName]. I realize this technically leaves me more vulnerable to accidentally conflicting with other libraries, however we'll cross that bridge if we ever reach it. functionName and category are both in camelCase, however I may add an additional suffix from time to time, separated from category by _, but then camelCase itself. But that's a special case with reasoning that should be obvious if it ever occurs.

File Structure / Includes / Etc.

File structure is one of my biggest points of indecisiveness, as I very frequently change my mind on how exactly I want to organize my files. That said, I'm fairly comfortable sticking with the form I have now, which is effectively as follow:

-- Mission Root --
hyp_Libraries
    #CfgFunction.h //Includes Config for all component folders
    Arrays
         #CfgFunctions.h //Includes Config for all "Arrays" functions
         #Headers.sqf //All macros or other forms of scripts to be included in every file in THIS particular folder
         fn_push.sqf //Begin files with each function's code
         ... 
    ...

Basically, we have a master #CfgFunctions.h file in the hyp_Libraries folder itself. This one #includes the #CfgFunctions.h of each category. Each of those category-specific #CfgFunctions.h hold the actual config entries for that folder/category.

Each category-folder has a unique #Headers.sqf file that includes any Macros or code that should be repeated in every function.

Macros

#define handleNil(a) ( if true then {private "_v"; _v = a;  if (!isNil("_v")) then {_v} else {nil} } )

This first macro came about when I was working on the array manipulation library. I wanted to make it work for ANY values stored in the array, and that includes nil. Nil is a tricky beast to work with, as assigning a variable to nil (ie _someVar = nil;) makes it indistinguishable from one that has never been declared, and causes an error to be thrown when you attempt to access that variable later on. Enter handleNil(). This is a simple macro that you pass a variable, or any value in general to. If the variable is valid, all is good and the macro simply returns its value. If passed a non-existent variable (a variable that equals nil), it will return nil (as created by the nil scripting command) instead of throwing errors. Simple, but necessary for any manipulations of data with unspecified datatypes.

//param : "a" is an array, "b" is an index, "c" is a default value.  If "a select b" is either out of bounds, or nil, c is returned.  Otherwise "a select b" is returned
#define param(a,b,c) ( if (count(a) > b && b >= 0) then { private "_v"; _v = a select b; if (!isNil("_v")) then {_v} else {c}  } else {c} )

In working on these functions, I noticed that calling code in functions was a reasonable deal slower than directly executing the same code. As such, I wrote param(), a simple macro to get an index from an array. If the specified index is either out-of-bounds or nil, a default value is returned. It serves an awfully similar purpose to BIS_fnc_param, however trades the thoroughness of additional checks for a 90% increase in performance (ie macro is 10x faster than calling BIS_fnc_param); As a matter of fact, a few functions that accepted optional parameters were already smaller and faster than all of BIS_fnc_param itself, so using BIS_fnc_param decreased performance by over 100%.

KeyValueArrays

I swear I've written this same set of functions a dozen times now, but differently every time around. The intuitive design was always an array of 2 element arrays, each as a key and a value. It worked well enough, but searching for a key required a linear traversal of the array, which can reach O(n) in the worst case. I also tried a very simple linked list design, where each node looked something like this:

[
    ["class", "NODE"],
    ["key", "someKey"],
    ["value", 1234],
    ["next", <the next node>]
]

The final design I've settled on is an array of keys, and an array of values, where each key corresponds to the value of the same index. Why this instead of the 2 element arrays? Simple, the engine command "find". I ran a few tests, and the extreme ends of my results were:

• When finding the 30th element in an array, linear traversal was 10x slower than the find command.
  
• When finding the 900th element in an array, linear traversal was 100x slower than the find command!
  

So as you can see, find beat out linear traversal by quite a wide margin. Even a factor of 10x is massive when we're dealing with something that may be manipulated as frequently as an alternative array type.

And finally, you may notice that the actual formatting of KeyValueArrays "objects" is as follows:

[
    ["class","KeyValueArray"],
    ["keys",[]],
    ["values",[]],
    ["tagged",[]]
]

This is remnant of a combination of two main things; debug code to help me read the contents of the array more easily, and leftovers of my attempt at what I'm calling "Array Oriented Programming". It was basically using lots of arrays to sort of emulate OOP. It's on the backburner for now, but if it ever makes a resurgence I'd potentially maintain the same "Key-Value-Pairs" for each field, even though the keys are never actually accessed or read.

Also, I mention the tagging system on the wiki page too, but it's my solution for deleting elements from the KeyValueArrays. Simply put, removing elements from an array by reference is not pretty, as you need to traverse the entire array every time you want to remove elements. That said, you can remove multiple elements in one pass with no additional impact on performance. That's where tagging comes in. Tag multiple elements if they should be removed, but don't *need* to be removed right away, and remove all of them in one well-performing swoop.

So yeah, there's a quick delve into the inner workings of this library. I figure there's at least one or two guys out there who may find this interesting, and maybe a few who can offer suggestions or improvements!

[das attorney 857]

This is a really nice gesture Hypno,

Clever stuff as well :)

Thanks, DA

[Hypnomatic 10]

Got a minor update for today!

2013-12-8

- Added kva_fnc_count and kva_fnc_forEach! (http://arma.dathypno.net/Function/kva_fnc_count/, http://arma.dathypno.net/Function/kva_fnc_forEach/)

- Fixed minor bug with arr_fnc_resize not returning the reference if passed a positive length.

- Added comment to arr_fnc_compare (Basically me rambling about how I found it funny string-comparison works so well on arrays).

kva_fnc_count

This guy's a super simple function, just returns the number of key-value pairs present in the KeyValueArray. Does not exclude anything tagged.

kva_fnc_forEach

Function to make traversing KeyValueArrays easier in certain circumstances, particularly if we have non-specific keys. The code you pass can include the following variables to reference the contents of the KVA:

_x: Contains the current VALUE
_forEachKey: Contains the current KEY
_forEachIndex: Contains the current INDEX

That's it for now! I'm probably next going to optimize the parsing of JSON, then switch gears to cleaning up and releasing two of my bigger functions: getting and setting an object's state via KeyValueArrays! Doing exactly this is probably the initial motivation for writing KeyValueArrays; I wanted a reliable way to completely serialize a player, vehicle, object, etc. as JSON, and pass it out of ARMA via an extension. The latest and greatest version I've written of this is likely coming soon!

[Larrow 2795]

Liking the macro's especially the param idea.

Its actually quite frightening how inefficient CALLing is, I remember a while back me and KK have a convo in a thread here about the difference in speed.

the difference between

_myArray = [1,2,3,4];
_randomIndex = _myArray select (floor (random (count _myArray)));

vs

_myArray = [1,2,3,4];
_randomIndex = _myArray call BIS_fnc_selectRandom;

Calling the BIS function is around 3x slower than doing it manually, and all the BIS function holds is the same as the first code above.

You then go on to read the optimizing thread in the wiki about script length and repeating code where it says things like 'if a script starts exceeding 200~300 lines it may be worth to start separating things into other scripts' (phrasing from memory). Hmmm whats the payoff between file size and repeated code versus CALLing repeated code as functions. Ok maybe not a problem 80% of the time but definitely food for thought when writing your main loops.

Thanks for posting Hypno very interesting

[Hypnomatic 10]

@Larrow:

Yep, I've been looking into what else I can replace with macros for that exact reason. I actually tried writing many of these functions as Macros as well, and they predictably performed noticeably faster. I'll do more testing later, and if it's actually worth the performance I'll rewrite and release these as macros too. The two biggest reasons I put that idea on hold are optional parameters and recursion: There's no real intuitive way I could find to allow parameters to be excluded, so a few functions would become a bit more awkward. Also a few of these functions are recursive, and you can't really do recursion with macros. In hindsight I see a way to do it, but it's not as pretty as I'd like. Will do more testing later on!

Also, one thing I "think" I observed is that functions in the global namespace are a bit slower to be accessed than those that are local. I haven't done any proper testing on this however, so it could have been any number of other conditions responsible. Will also do some testing on this later on and report back with what I find!