Closures in SQF

[vektorboson 8]

For the basics what I will talk about, please take a read at:

http://en.wikipedia.org/wiki/Closure_(computer_science)

In many functional programming languages closures can be used to fake encapsulation.

Thus it is possible to program in a object oriented way with closures.

Although I know functional programming (for example Scheme, OCaml) for a long time, I only recently thought about trying to use closures with SQF.

The point about closures and functional programming is, that functions are first-class citizens of the programming language; you can pass around functions and manipulate them.

The funky thing about SQF is, that you can create functions on the fly (with the compile-command). I took this approach to bind hidden global variables to some 'object methods'.

The best way is to just look at the code

// File: closure.sqf
makePoint2D = {
 point2DCounter = if(isNil "point2DCounter") then {0} else {point2DCounter+1};

 _id = point2DCounter;
 _this call compile format["point2D%1 = _this;", _id];

 point2DDispatch = {
   _id = _this select 0;
   _cmd = _this select 1;
   switch(_cmd) do {
       case "get-x": {call compile format["point2D%1 select 0", _id]};
       case "get-y": {call compile format["point2D%1 select 1", _id]};
       case "set-x": {(_this select 2) call compile format["point2D%1 set[0, _this];", _id]};
       case "set-y": {(_this select 2) call compile format["point2D%1 set[1, _this];", _id]};
       case "hint": { hint str call compile format["point2D%1", _id]; };
   };
 };

 _closure = compile format["([%1] + _this) call point2DDispatch", _id];
 _closure
};


_pointA = [1, 2] call makePoint2D;
assert(["get-x"] call _pointA == 1);
assert(["get-y"] call _pointA == 2);
["hint"] call _pointA;

["set-x", 10] call _pointA;
["set-y", 15] call _pointA;
assert(["get-x"] call _pointA == 10);
assert(["get-y"] call _pointA == 15);

_pointB = [3, 4] call makePoint2D;
_x = ["get-x"] call _pointB;
assert(_x == 3);
_y = ["get-y"] call _pointB;
assert(_y == 4);

// pass reference to _pointA into another script context, modify there x = 20
_pointA execVM "other.sqf";
sleep 0.1;
assert(["get-x"] call _pointA == 20);

// file: other.sqf
["set-x", 20] call _this;

Save those files into a mission, and execVM "closure.sqf". You can also put more ["hint"] call _pointA (or _pointB) into the code, or you change the hint method:

case "hint": {player sidechat str call compile format["point2D%1", _id]; };

So instead of a hint, the value is output with sidechat.

Short explanation:

The global point2DDispatch-function takes an numerical ID as the first argument, the method name as second argument and optional arguments afterwards in the _this variable.

We switch over the method name and execute a method specific to the global point2D%1 (replace %1 with the ID). The IDs are tracked through point2DCounter, that is incremented every time a new point2D-object is being created.

The whole point is, that we don't want to deal with the global variables at all, thus we create the special function _closure. _closure is a function that binds the ID as the first argument to point2DDispatch, and passes other arguments after the ID.

So let's take a look at calling the "set-x" method:

["set-x", 10] call _pointA;

_pointA points to the _closure-function that has bound the ID 0 to point2DDispatch, so what is really happening is this:

([0] + ["set-x", 10]) call point2DDispatch;

If we instead called _pointB, the same call will be this:

([1] + ["set-x", 10]) call point2DDispatch;

When we're passing the _pointA with execVM into another script, we're indeed passing the _closure-function, that has bound the ID. Thus we are accessing the same internal state, but we don't need to explicitly pass around global variable names.

Note:

Please note that you could already bake the ID into point2DDispatch instead of _closure. I didn't do this, since format has a limit of 2048 characters, thus you can't have bigger objects. Second, the dispatch-function would be duplicated for every object, thus taking a lot of memory, if lots of objects are created.

Edited November 2, 2011 by vektorboson

[Spooner 0]

Regarding the limitation with string-length manageable by format[/b], you can avoid this by using + and str.

<table border="0" align="center" width="95%" cellpadding="0" cellspacing="0"><tr><td>Code Sample </td></tr><tr><td id="CODE">

compile format ["frog%1 = fish", _value];

compile ("frog" + (str _value) + " = fish");

(That trivial example would never overload strings, but you know what I mean).

Sadly, the issue with call compile is that you no longer get debug messages with file/line number, which is a major loss. Although the code seems trivial, someone will provide bad data to it and break it soon enough!

Nevertheless, a very interesting exploration of shoe-horning computer science into SQF (which is a sort of computer mysticism ).

Edited April 11, 2009 by W0lle

[vektorboson 8]

Thanks to Spooner here is a much cleaner version:

point2D = []; 
point2DDispatch = {
 private ["_id", "_cmd", "_point"]; 

 _id = _this select 0;
 _cmd = _this select 1;

 _point = point2D select _id;

 switch(_cmd) do {
   case "get-x": { _point select 0 };
   case "get-y": { _point select 1 };
   case "set-x": { _point set [0, _this select 2] };
   case "set-y": { _point set [1, _this select 2] };
   case "hint": { hint str _point };
 };
};

makePoint2D = {
 private "_id";

 _id = count point2D;

 point2D set [_id, _this + []]; // Store copy, so it can't be changed directly.

 // Return the closure.
 compile format["([%1] + _this) call point2DDispatch", _id];
};


_pointA = [1, 2] call makePoint2D;
assert(["get-x"] call _pointA == 1);
assert(["get-y"] call _pointA == 2);
["hint"] call _pointA;

["set-x", 10] call _pointA;
["set-y", 15] call _pointA;
assert(["get-x"] call _pointA == 10);
assert(["get-y"] call _pointA == 15);

_pointB = [3, 4] call makePoint2D;
_x = ["get-x"] call _pointB;
assert(_x == 3);
_y = ["get-y"] call _pointB;
assert(_y == 4);

// pass reference to _pointA into another script context, modify there x = 20
_pointA execVM "other.sqf";
sleep 0.1;
assert(["get-x"] call _pointA == 20);
["hint"] call _pointA;

Edited November 2, 2011 by vektorboson

[t_d 47]

very interesting!

btw: for creating a copy of an array: http://community.bistudio.com/wiki/plus_a

[PhilippRauch 0]

something for my brains to chew on, nice.

Do you programming gurus have any real(ArmA)World examples where that could be useful? Like counting pink dildos, or ... ?

Im just curious to learn more of this stuff, since im only a Designer which has never seen a university from the inside. (maybe except on a couple of freshman partys...)

[UNN 0]

Quote[/b] ]btw: for creating a copy of an array

I might be wrong, but I thought he wanted to pass the pointer to the array, not a copy?

Quote[/b] ]Do you programming gurus have any real(ArmA)World examples where that could be useful?

Command Engine X by Spinor, but he uses configfiles to do pseudo object oriented code.

Edited April 11, 2009 by W0lle

[vektorboson 8]

Quote[/b] ]btw: for creating a copy of an array

I might be wrong, but I thought he wanted to pass the pointer to the array, not a copy?

In this specific case, a copy should be made, so that the internal state of the closure can't be changed accidentally through the array that was passed to the constructor.

If it wasn't a copy, this would happen!

_a = [1, 2];
_point = _a call makePoint2D;
_a set [0, 4]; // oops, I changed the internal value of _point!!!

Of course the constructor (makePoint2D) may be implemented in another way, for example not taking a parameter at all and returning an object with standard values.

Quote[/b] ]Do you programming gurus have any real(ArmA)World examples where that could be useful?

As UNN mentioned, very complex systems like CoC CEX can possibly be programmed in a easier way. Also think of such missions like Warfare and other CTIs.

As my usage example partially has shown, it's pretty easy to let script threads communicate and share data without explicitly using global variables.

I can think of many things that can be implemented in a nice way, like Signals/Slots, delegates or thread synchronization.

Edited November 2, 2011 by vektorboson

[Big Dawg KS 6]

Very nice indeed. IMO OO scripting is something BIS should have implement into ArmA from the beginning. At the least they could have introduced structs or something, using arrays for everything is just painful.

[bn880 5]

I've been away a while, so lets see if I understand this correctly...

Nothing wrong with the idea VB, however essentially the term "closures" is a term that would encompass most advanced scripts in OFP/ARMA, such as most "self modifying" scripts in use at COC. ArmA scripts in general are capable of what most programming languages are not, self modification.

Look even at one of our basic utility scripts; SortBubble.sqf

A very simple example of a "closure" where you see _numArray set [_j,call _eval];

Quote[/b] ]/*

PURPOSE: Sort given array according to given evaluator.

HISTORY: bn880 24/02/2003

INPUTS: _array: Array, objects to sort

_eval: String, Evaluator to sort by

_element becomes each element from _array

-(_eval) sorts in descending order

RETURNS: BOOL, true on success

false on error

ALGORITHM:

END ALGORITHM

CALL EXAMPLES:

SIMPLE ASCENDING NUMERIC SORT

myArray = [2,6,3,7,8,4];

_ok = [myArray,"_element"] call SortBubble.sqf

myArray becomes [2,3,4,6,7,8]

SIMPLE DESCENDING NUMERIC SORT

myArray = [2,6,3,7,8,4];

_ok = [myArray,"-(_element)"] call SortBubble.sqf

myArray becomes [8,7,6,4,3,2]

DISTANCE SORT TO PLAYER (DESCENDING)

_ok = [ myArray , "-(_element distance Player)"] call SortBubble.sqf

DISTANCE SORT TO TEAM MEMBER 2 IN PLAYERS GROUP (ASCENDING)

_ok = [ myArray , "_element distance ((units (group Player)) select 1)"] call SortBubble.sqf

*/

private ["_array","_eval","_numArray","_j","_k",

"_tempVal","_tempNum","_element","_ok"];

_array = _this select 0;

_eval = _this select 1;

// CHECK FOR PASSED IN PARAMETERS

if (count _array == count _array && _eval == _eval) then {

_ok = true;

_count = count _array;

_numArray = [];

_numArray resize _count;

// ACQUIRE NUMERIC VALUES FROM EVALUATION

_j = 0;

while "_j < _count" do

{

_element = _array select _j;

_numArray set [_j,call _eval];

_j = _j + 1;

};

_j = 0;

// SORT

while "_j < _count -1" do

{

_k = _j + 1;

while "_k < _count" do

{

if (_numArray select _j > _numArray select _k) then

{

_tempNum = _numArray select _j;

_numArray set [_j,(_numArray select _k)];

_numArray set [_k,_tempNum];

_tempVal = _array select _j;

_array set [_j,(_array select _k)];

_array set [_k,_tempVal];

};

_k = _k + 1;

};

_j = _j + 1;

};

} else

{

// ERROR

_ok = false;

};

_ok;

Now that we have the semantics out of the way;

Much more advanced encapsulation to emulate OOP has been used throughout OFP projects. Basically what is occurring in your example is one array is being used as a function stack, while self modifying code a.k.a. "closures" are being used to access the specific sub-arrays on the fly. A fairly neat version of this was implemented in my tracers (bn880's tracers) to eliminate array searches and global variables when assigning tracer per side/weapon/man properties with the need for instant low resource access with each fired shot.

Now maybe I am missing something ground breaking here,i have been away, if I am please let me know. Don't get me wrong, the concept is extremely useful and can be viewed to act as an OOP tool, but, it has been used extensively, though perhaps not documented as OOP.

Edited April 11, 2009 by W0lle

[vektorboson 8]

I don't know about your other code, but the code snippet you posted doesn't show closures.

What you are showing is called a 'lambda function' (and dynamic name binding) in many programming languages, which is not necessarily a closure.

In your case the lambda function accesses variables that were declared outside the lambda function (through dynamic name binding). In this case you're absolutely right: This is a very old technique, and even I used this technique, for example in my VB_RungeKutta-function at OFPEC (don't know if it is still up there).

What I wanted is to simulate closures which can be used in functional languages to do OOP; closures access variables that were defined/declared in the closure's local scope, not outside it. I'm faking this through that global array and returning a 'function' with index into the array hard coded.

In case this was already used some time, of course, then it is nothing new. I just have not seen it anywhere, and I didn't see anyone mention it. I looked and searched for something similar but I didn't find it (which of course doesn't mean, it didn't exist before).

Of course, if you know about techniques how to do OOP in SQF, than write about it; I'd be glad to know that there is some nice and elegant way how to handle complexity (or what other techniques there are).  

[bn880 5]

Yes well it goes right back around to the whole concept of OFP/ArmA scripting. When creating variable names or self modifying code, storing it in a variable, then calling it, you are already working with closures. It has really been used heavily.

In your example:

The function makePoint2D

creates and returns a string called _closure

while maintaining one index variable

and inserting it into the string as a string.

This string can then be called as a function

and the index will be retrieved from this string.

Additionally you work with a global array point2D to stack the data for this possible function.

It is faked as you say; globally accessible.

Therefore;

A) you store the index in a string called _closure, to which you later give other names like _pontA

B) you store the data originally associated with this index in a global array point2D

C) you call the string _closure which translates the index back to an integer and also passes along new parameters to another function which than works with the data and point2D

Since we are in this realm of almost all advanced scripts using closures, it may be best to avoid using that terminology to come up with a document to describe this particular hack.

As I referred to the tracers... and would like to prove to you that this really is in use around many complex projects that I have personally seen and/or worked on I will present two functions from them and highlight the relevant lines that I can still see:

addTracerUnit2.sqf

Quote[/b] ]_u=_this select 0;

_in=_this select 1;

_cc=_this select 2;

_d=true;

_veh="man"counttype[_u]<1;

if(_veh&&count crew _u==0)then{

_i=-1;

_i2=-1;

while{_i2!=_i+1}do{

_u removeEventHandler["getin",_i2];

_u removeEventHandler["getin",_i];

_i=_u addEventHandler["getin",{}];

_i2=_u addEventHandler["getin",format[{[_this select 0,_this select 2,%1,%2,%3]call loadfile{\bn_tracer\getInVeh.sqf};},_in,_cc,_i+1]];

};

}else{

if(count bn_trunits==count bn_trunits)then{_d=false;};

_i=0;

_ci=_In;

_wep=primaryweapon _u;

_weps=weapons _u;

if _veh then{

if(count _weps>0)then{

_wep=_weps select 0;

};

};

if _d then{

if _cc then{

call loadfile{tracerSettings.cfg};

}else{

call loadfile{\BN_tracer\tracerSettings.cfg};

};

bn_trunits=[_ci];

bn_trunitswep=[_wep];

}else{

_i=count bn_trunits;

bn_trunits=bn_trunits+[_ci];

bn_trunitswep=bn_trunitswep+[_wep];

};

_c=call format["%2%1",side _u,{bn_trcolor}];

_d=true;

if(count bn_trweapons==count bn_trweapons)then{_d=false;};

if(_wep!=""&&!_veh)then{

_d=true;

if(count call("bn_tr_"+_wep)==count call("bn_tr_"+_wep))then{_d=false;};

if _d then{

call format[{bn_tr_%1=[_in,_c];},_wep];

};

}else{

if _veh then{

{

_d=true;

if(count call("bn_tr_"+_x)==count call("bn_tr_"+_x))then{_d=false;};

if _d then{

call format[{bn_tr_%1=[_in,_c];},_x];

};

}foreach _weps;

};

};

_u addEventHandler["fired",format[{_this+%1 call loadFile{\bn_tracer\tracereh.sqf}},[_c,_in,_i]]];

};

tracerEH.sqf

Quote[/b] ]_u=_this select 0;

_w1=_this select 1;

if(_w1==primaryweapon _u||"man"counttype[_u]<1)then{

_a=_this select 4;_c=_this select 5;_in=_this select 6;_i=_this select 7;

_w2=bn_trunitswep select _i;

_trace=false;

if(_w1==_w2)then{

_cnt=bn_trunits select _i;

if(_in<=_cnt)then{

_trace=true;

bn_trunits set[_i,1];

}else{

bn_trunits set[_i,_cnt+1];

};

}else{

_d=false;

if(count call("bn_tr_"+_w1)==count call("bn_tr_"+_w1))then{

_wd=call("bn_tr_"+_w1);

_c=_wd select 1;

_in=_wd select 0;

bn_trunits set[_i,2];

bn_trunitswep set[_i,_w1];

if(_in<=1)then{_trace=true;};

};

};

if _trace then{

_b=nearestObject[_u,_a];

_n=objnull;

if!(isnull _b)then{

_v=velocity _b;

_s=sqrt((_v select 0)^2+(_v select 1)^2+(_v select 2)^2);

if(_s>150)then{

_p=getpos _b;

_r=acos((_v select 2)/_s);

if!bn_trdedi then{

_n="BN_Tracer"camcreate _p;_n setdir getdir _b;_n animate["tracer_swing",_r/180];_n setvelocity _v;_n setObjectTexture[0,format["\bn_tracer\%1b.paa",_c]];

};

_r1=(_r mod .01)*100;

_r2=(_r mod .0001)*10000;

[_n,_b,_c,_p,local _u,_r1,_r2]exec{\bn_tracer\tracerkill.sqs};

};

};

};

};

In this example local data is stored in an attached event handler, while the rest is stored in other arrays such as bn_trunits or "bn_tr_%1". All data is accessible upon calling the event handler, same thing as saving this to a string _pointA and then calling it later.

Anyway, no matter how you cut it, it's very neat stuff that we can do in this scripting language and engine. If you wish to document or rather even make a tutorial on OOP encapsulation/simulation in ArmA I certainly will not stand in your way and am all for it.

However; what I am trying to express to you is that YES I have read this thread and its quite cool but I would like to say is CoC is well aware of this technique and has used it extensively in various areas without making a tutorial on it. From my perspective I always thought someone would go in and for example examine the tracer code to see how it manages to do all that without complex array searches and thus instant response and no lag.

Edited April 11, 2009 by W0lle

[Worldeater 2]

I think the problem with closures and ArmA script is the fact that you cannot create them in ArmA script. What was done here was to fake a closure-like behaviour with compile. Using an "eval" (and that's what compile is) to "keep the scope" does not count and could be considered cheating! :D

If ArmA script would support closures the following should display "foobar". But it does not because the value of "_freeVar" is not preserved once it runs out of scope:

private ["_fn_createClosure", "_f"];

_fn_createClosure = {
   private ["_freeVar"];
   _freeVar = _this;
   { _freeVar }; // return an anonymous function
};

_f = "foobar" call _fn_createClosure;
hint format [">>> %1", call _f];

[bn880 5]

Yes, we both acknowledge it would be faked, this is why I say the issue at hand is not closures at all, it is OOP. :)

In terms of closures you can get there in terms of integers by hiding them in command strings, or otherwise any other variable which can be expressed as a clear text string can be saved within a function at runtime. I think VB was trying to simplify or expand scripting in ArmA.

[vektorboson 8]

Well it is indeed about faking OOP by faking closures, a double-fake so to speak.

Nevertheless, I'm pretty interested by the concept of "namespaces" in ArmA II. Actually I'm wondering what it actually will be; since it sounds like they want to cut down usage of global variables, this hopefully means some sort of encapsulation.

[bn880 5]

Heh, but do two fakes make it a a real? That is the question.

As far as I understand the namespaces, they will do just that, separate the variable space. I am just not certain how much easier this will make OOP. What, are we going to create a separate namespace for each object? You still need to keep track of the name of the namespace if you will.

[barmyarmy 0]

I don't mean to seem a killjoy, I consider myself more of an engineer, than a scientist.

I am also team leader for a development house in RL, the biggest challenge is getting code which is easy to understand and maintain, esp. when the person maintaining the code might not be the person who originally wrote it. I am not a fan of "clever" code, if that clever code can only be maintained and debugged by the person who wrote it, thats just "bad" code.

The initial example is somewhat trivial and in my opinion a bad example of what can be achieved more by the use of "anonymous methods" than "closures" per-say. All you have done in the first example obfuscate the code, and ended up with data that looks like code and code that looks like data.

consider pointA= [1,2] call createPoint2D;

"hint" call pointA;

Deciding which method to execute to call by passing in a string? Oh dear.

I think there are better methods of achieving OO without closures, or at least you need to be *much* clearer in your naming of variables if you want to use this technique in "production" code.

While proper Object-Orientation in SQF would indeed be nirvahna this is NOT a good way of achieving it.

However I did find bn880's tracer example much more interesting after I had sat down and spent ten minuted reformatting it so I could see what it did, and how it did it. Thanks for taking the time to post that.

Sorry, what the community needs in my opinion is large body of well written easy to understand and maintain code. While this "science" is all very good, unless used correctly what we will end up with is a mess that is easy to misenterpret.

Having said that this thread was interesting reading, I am trying to be cautious as oppose to negative.

Far to often I have seen "interesting" technologies used for the sake of using them as oppose to because they are appropriate, XML was a particulartily good example of this, Linq is becoming another. However it seems to be case that this happens alot with new technologies, they have to be experimented with before their limitations and usefulness can really be discovered.

There are those in the community who are considered leaders, the problem is with being a leader is that people follow, which creates a responisibility to take the care to set a good example for your followers.

BA

Edited April 17, 2009 by barmyarmy