JavaScript for ARMA

[.kju 3245]

Why not provide a function that nils the variable in SQF land but notifies the JavaScript land beforehand/too?

[Milyardo 10]

Your global reference is a more uglier method of doing what I described with a persistent V8 handle. It would still leak the object. Pseudo-code illustrating object return by reference:

_something = "something_id_ref = new Something; 'something_id_ref'" call JS_fnc_exec;
_something = nil;

When and who is going to release the global "something_id_ref" in JavaScript land?

(_something + " = null;") call JS_fnc_exec
_something = nil;

In this particular example(it's a bad example for what we're trying to illustrate). The user explicitly created a global variable, so it should be up to them to manage it. Detecting user explicit globals would be a fairly advanced feature however and beyond the scope of what I was describing.

Returning our earlier example:

_result = "2+2" call JS_fnc_exec //At this step your framework should create a global variable in JavaScript, say "expr1" and return that name
                                           //It's important to note that your framework creates this reference, not SQF, it's up to you to manage it
                                           //and by extension SQF cannot leak references it doesn't create.
"expr1" call JS_fnc_exec //returns 4
_result call JS_fnc_exec //also returns 4
_result = nil
"expr1" call JS_fnc_exec //returns 4
call JS_fnc_reset //A new method to reset the v8 execution context or just collect the list of evaluated expressions your framework maintains
                      //resetting the execution context is more desired because it allows us to undefine classes as well.
"expr1" call JS_fnc_exec //returns undefined

---------- Post added at 01:27 PM ---------- Previous post was at 12:08 PM ----------

;2412370']Why not provide a function that nils the variable in SQF land but notifies the JavaScript land beforehand/too?

Reference binding and synchronization in general would be a nice to have.

Edited June 11, 2013 by Milyardo

[looter 10]

Hey simast, rather than using google's v8 engine directly have you considered using http://nodejs.org/ ? It might have some more overhead, but it would allow users to choose from thousands of pre-made npm packages of libraries instead of implementing direct interfaces between libraries and javascript in C++ in the framework, and I think it would make this addon alot more powerful and useful, especially for server tasks and mission / persistent database work. What do you think the possibilities of this are?

Edited June 17, 2013 by Looter

[Flo 1]

very cool !! mirrored in our script section ! :)

[ylguf 1]

Very good job. I was learning javascript before I started scripting SQF so this is extremely useful. I can see this being used for a database in the future, especially since javascript would be really easy to throw in a webpage and make it public. one limitation I was thinking of tho is, apparently

_x = 1 + 2

_x == 3 will return true

where as

_x = 0.1 + 0.2

_x == 0.3 will return false. in javascript.

now im not sure if that is fixed in googles v8 enginge. but if not would it be possible to get a function that checks for biggest decimal(lets say 0.01 is the largest decimal passed) then times every number passed by 100 and divide anything passed back by 100 to solve that problem. because what happens if I pass a units xyz with 2 decimal places and I do a check and it would always return false or something. it also might just be better to do that on our own but its still useful knowledge.

Edited June 16, 2013 by ylguf

[Sniperwolf572 758]

Very good job. I was learning javascript before I started scripting SQF so this is extremely useful. I can see this being used for a database in the future, especially since javascript would be really easy to throw in a webpage and make it public. one limitation I was thinking of tho is, apparently

_x = 1 + 2

_x == 3 will return true

where as

_x = 0.1 + 0.2

_x == 0.3 will return false. in javascript.

now im not sure if that is fixed in googles v8 enginge. but if not would it be possible to get a function that checks for biggest decimal(lets say 0.01 is the largest decimal passed) then times every number passed by 100 and divide anything passed back by 100 to solve that problem. because what happens if I pass a units xyz with 2 decimal places and I do a check and it would always return false or something. it also might just be better to do that on our own but its still useful knowledge.

You can either round it to the decimal value of your choice or compare the absolute value of the difference between the two.

a = 0.1 + 0.2; // 0.30000000000000004
a == 0.3; // false

// but

(Math.abs(a - 0.3) < 0.000001); // true

// or

a.toFixed(1) == 0.3; // true

It's much easier to work around it yourself for such cases.

Edited June 17, 2013 by Sniperwolf572
Semicolon

[jacob88 10]

_year = "(new Date()).getFullYear()" call JS_fnc_exec;
_month = "(new Date()).getMonth()" call JS_fnc_exec;
_month = _month + 1;
_day = "(new Date()).getDate()" call JS_fnc_exec;
_hours = "(new Date()).getHours()" call JS_fnc_exec;
_minutes = "(new Date()).getMinutes()" call JS_fnc_exec;
_seconds = "(new Date()).getSeconds()" call JS_fnc_exec;
_fulldate = format ["%3" + ":" + "%2" + ":" + "%1", _year, _month, _day];
_fulltime = format ["%1" + ":" + "%2" + ":" + "%3", _hours, _minutes, _seconds];
_fulldatetime = format ["%3" + ":" + "%2" + ":" + "%1" + "  " + "%4" + ":" + "%5" + ":" + "%6" , _year, _month, _day, _hours, _minutes, _seconds];

This might come in handy for some people using this addon.

_fulldate = date on the calender e.g. 5:07:2013

_fulltime = current time (on machine) e.g. 7:54:21 (last ones seconds)

_fulldatetime = date on the calender and current time (on machine) e.g. 5:07:2013 7:54:21

[Simas 12]

Changes:

• Updated V8 to 3.20.2.
  
• Implemented background script handles (returned by JS_fnc_spawn).
  
• Implemented JS_fnc_done and JS_fnc_terminate functions.
  
• Added sleep(seconds) JavaScript function.
  
• Added a common header file (#include "\JS\API.hpp") with JS(code) macro.
  
• PBO and extension DLL files are now signed with bikeys (server-side key is included).
  
• Other internal tweaks and optimizations.
  

[CSLALUKI 30]

WOW, thanks ! :)

[Guest]

Updated version frontpaged on the Armaholic homepage.

JavaScript [bETA] v0.6

===================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

[Simas 12]

Changes:

• Updated V8 to 3.21.11.
  
• Fixed: Serialize JavaScript NaN value as SQF nil.
  
• Fixed: Serialize JavaScript Infinity value as SQF scalar infinity.
  
• Added a total of 32 unit tests (use execVM "\JS\Tests\_Run.sqf" to run).
  
• Other internal tweaks and optimizations.
  

Edited September 9, 2013 by Simas

[mrflay 4]

Nice to see another update!

I've been experimenting a little bit with this addon and in my opinion it has huge potential!

cURL

Since you have made the source available (thank you very very much!), I decided I would attempt to add cURL support for it. I have made my implementation available on github, if you're interested in having a look. The cURL implementation was taken from SilkJS, another open source javascript framework.

It works well enough for my purposes.

SQF from Javascript

Another experiment I did was trying to call SQF from Javascript.

It uses a background SQF thread that listens for and processes requests made from javascript. A request consists of the name and arguments of the command or function to execute together with a unique id. The requests are put in a queue (not thread-safe) in javascript and polled sequentually by the SQF background thread.

// pseudo code...
[] spawn {
while { true } do {
   waitUntil { JS("sqf.hasRequest()") }; // wait until SQF call is made in javascript.
   _event = JS("sqf.nextRequest()");
   _result = _event call JS_fnc_processRequest; // execute the appropriate SQF command or function.
   if (_result) then {
       _result call JS_fnc_addResult; // send result to javascript
   };
};
};

Then in javascript, this module handles the SQF requests and responses.

var sqf = (function () {
var self = this;
self.sequence = 1000;
self.results = [];
self.events = [];

   // non-blocking, returns SQF request handle
function exec2(expr) {
	var id = "" + (self.sequence++);
	var time = new Date().getTime();
	self.events.push({
		id: id,
		type: "exec",
		time: time,
		args: expr
	});
	self.results[id] = { id:id, time:time, valid:false };
	return id;		
}

// non-blocking, returns SQF request handle
function call2(name, args) {
	var id = "" + (self.sequence++);
	var time = new Date().getTime();
	self.events.push({
		id: id,
		type: "call",
		time: time,
		args: [name].concat(args)
	});
	self.results[id] = { id:id, time:time, valid:false };
	return id;
}

   // blocking, checks results for SQF request with specified id (note: result is not currently removed)
function result(id) {
	var done = false;
	var now = new Date();
	var timeout = new Date(now.getTime() + 1000);
	while (!done && now < timeout) {
		now = new Date();
		done = self.results[id].valid;
		sleep(0);
	}
	if (!self.results[id].valid) {
		throw "error: sqf exec timed out";
	}
	return self.results[id].value;
}

return {

	exec2: exec2,

	call2: call2,

	hint: function (expr) {
		exec2("hint '" + expr + "'");
	},

	// blocking
	exec: function (expr) {
		id = exec2(expr);
		return result(id);
	},

	// blocking
	call: function (name, args) {
		id = call2(name, args);
		return result(id);
	},

	hasRequest: function () {
		return self.events.length > 0;
	},

	nextRequest: function () {
		var ev = [];
		if (self.events.length > 0) {
			var el = self.events[0];
			ev = [el.id, el.time, el.type, el.args];
			self.events = self.events.splice(1);
		}
		return ev;
	},

	addResult: function (id, value) {
		self.results[id] = {
			id: id,
			value: value, 
			valid: true
		};
	},

	getResult: function (id) {
		return self.results[id].value;
	},
};
}());

Obviously, this is a naive implementation and there's a tremendous overhead for making calls to SQF. However, if performance is not an issue, this works adequately.

For example, with this it's possible to completely encapsulate the SQF API in javascript and write nice Object Oriented wrapper classes for commonly used stuff. Eg.,

var player = (function (sqf) {
return {
	get pos() {
		return sqf.exec("position player");
	},
	get atl() {
		return sqf.exec("getPosATL player");
	},
	get asl() {
		return sqf.exec("getPosASL player");
	},
	nearest: function (typeName) {
		return sqf.call("JSQF_fnc_nearest",typeName); // needs some kind of object wrapping
	},
	...
};
}(sqf));

Debugging

Finally, I attempted to get the node-inspector debugger working, but it has some issues. Basically, it's possible to set breakpoints, stepping through the code, inspecting variables and calling functions. However, it's not yet possible to see the source code while doing it, so this limits the usefulness quite dramatically.

Anyways, many thanks for making this awesome addon!

Edited September 7, 2013 by mrflay
described sqf background thread

[Simas 12]

cURL

Since you have made the source available (thank you very very much!), I decided I would attempt to add cURL support for it.

...

Always nice to see a fork :) As you can see, wrapping a C/C++ library is not that complicated with V8 (probably the hardest part is compiling and maintaining all the dependencies/libraries). I will investigate a possibility of expanding the "vanilla" JavaScript with some extra functionality in the future.

SQF from Javascript

Another experiment I did was trying to call SQF from Javascript.

...

For example, with this it's possible to completely encapsulate the SQF API in javascript and write nice Object Oriented wrapper classes for commonly used stuff. Eg.,

Interesting! I was brainstorming the idea of calling back to SQF from JavaScript land myself lately. Although, the idea I was thinking about does not involve any kind of loops and constant polling (for performance reasons).

Basically, the key is:

1. During a JS_fnc_exec call you would spawn (and detach) a separate thread for JavaScript code execution (note to myself: use a thread pool)

2. On the main callExtension thread you block and listed for the above JavaScript thread requests (or completion)

3. Then, during the JavaScript code execution you can now do this:

var playerPosition = SQF.position(SQF.player);

// or ..

SQF.diag_log("my RPT log message");
SQF.hint("my hint");

Every time you call something on the SQF.* namespace in JavaScript - the JavaScript execution thread will suspend and notify the main callExtension thread it needs a callback to SQF.

The main callExtension thread now returns back to SQF, but what it basically does is it returns SQF code that calls what the JavaScript requested and then immediately calls back to JavaScript with the results (with a callExtension recursive call).

In theory this should work and could be quite a killer feature.. Will try to get a working implementation in the next release. And some other random notes:

- This will only work in a blocking mode JavaScript call (and not with JS_fnc_spawn).

- SQF objects (and groups) can be represented/serialized in JavaScript with netId/objectFromNetId and should work transparently.

- The above method needs some serious code re-engineering and could impact (performance wise) other simple JavaScript code that doesn't require SQF callbacks.

- The SQF.* calls would be dynamic and will not require any wrapping. Basically if Arma will add any new SQF commands in the future - they will work in this addon without any changes.

Debugging

Finally, I attempted to get the node-inspector debugger working,

...

I believe I need to enable the debug agent from my side and implement some V8 callbacks. Will see what I can do about this in the future.

Edited September 7, 2013 by Simas

[Gudsawn 93]

This looks brilliant. Can't wait to try it out, although I can't really think of any scenarios where you would need JS over SQF. Any examples on what this has been used in yet?

Edited September 7, 2013 by JamieG

[mrflay 4]

I will investigate a possibility of expanding the "vanilla" JavaScript with some extra functionality in the future.

Cool :)

A trivial, but useful addition is to make the addon execute "init.js" when mission is started. Eg.,

// fn_init.sqf
...
[] spawn {
private ["_script"];
_script = loadFile "init.js";
_script call js_fnc_exec;
missionNamespace setVariable ["js_initDone", true];
};

Altough, in dev branch arma will display an error if the init.js file does not exist (should be ok in stable/release build, I think).

Interesting! I was brainstorming the idea of calling back to SQF from JavaScript land myself lately. Although, the idea I was thinking about does not involve any kind of loops and constant polling (for performance reasons).

Basically, the key is:

1. During a JS_fnc_exec call you would spawn (and detach) a separate thread for JavaScript code execution (note to myself: use a thread pool)

2. On the main callExtension thread you block and listed for the above JavaScript thread requests (or completion)

3. Then, during the JavaScript code execution you can now do this:

...

Wouldn't changing the SQF background thread to call a blocking js function (instead of using waitUntil) have the same effect?

[] spawn {
while { true } do {
   _event = [] call JS_fnc_nextRequest; // blocks until next call/exec request becomes available.
   _result = _event call JS_fnc_processRequest; // execute the appropriate SQF command or function.
   _result call JS_fnc_sendResult; // send result to javascript (could also be baked into the js_fnc_nextrequest to save one callExtension call)
};
};

- SQF objects (and groups) can be represented/serialized in JavaScript with netId/objectFromNetId and should work transparently.

Afaik, netId/objectFromNetId only works in multiplayer. In single player I think you would need to use something like bis_fnc_objectVar, although I'm not sure how well this method scales (object mapping is stored in the mission namespace and broadcasted across the network).

I believe I need to enable the debug agent from my side and implement some V8 callbacks. Will see what I can do about this in the future.

I took the debug support from SilkJS, see v8.cpp in my dev branch.

To use it, you also need to install node.js and the node-inspector module (with npm). However, since node-inspector requires a working fs module to show the source files during debugging, you also need to modify the PageAgent.js to get it working.

Replace contents of the getResourceTree function with a call to done();

 // C:\Users\<username>\AppData\Roaming\npm\node_modules\node-inspector\lib\PageAgent.js
 getResourceTree: function(params, done) {
done();
   //if (this._debuggerClient.isRunning) {
   //  this._doGetResourceTree(params, done);
   //} else {
   //  this._debuggerClient.once(
   //    'connect',
   //    this._doGetResourceTree.bind(this, params, done)
   //  );
   //}
 },

Then it's just a matter of opening the node.js console and run node-inspector. Browse to the url specified and bring up the console.

Edited September 7, 2013 by mrflay
wording

[xendance 3]

Very good job. I was learning javascript before I started scripting SQF so this is extremely useful. I can see this being used for a database in the future, especially since javascript would be really easy to throw in a webpage and make it public. one limitation I was thinking of tho is, apparently

_x = 1 + 2

_x == 3 will return true

where as

_x = 0.1 + 0.2

_x == 0.3 will return false. in javascript.

now im not sure if that is fixed in googles v8 enginge. but if not would it be possible to get a function that checks for biggest decimal(lets say 0.01 is the largest decimal passed) then times every number passed by 100 and divide anything passed back by 100 to solve that problem. because what happens if I pass a units xyz with 2 decimal places and I do a check and it would always return false or something. it also might just be better to do that on our own but its still useful knowledge.

0.1 and 0.2 are floating point numbers, not exact representations of those particular decimal numbers.

http://stackoverflow.com/questions/588004/is-javascripts-floating-point-math-broken

[mrflay 4]

I'll just answer my own silly question :)

Wouldn't changing the SQF background thread to call a blocking js function (instead of using waitUntil) have the same effect?

From http://community.bistudio.com/wiki/Extensions:

Currently, the extension code doesn't need to be reentrant - sqf evaluation proceeds single threaded and halts until you return. It is unknown how that may change wrt java, though, so you may want to write as-if you can be called by multiple threads.

Hence, it's not possible to block a script (even if it is running in the background)... I did not know that.

[Simas 12]

Yep, can't really "connect" to the extension and block (with a sleep) a background SQF script. The SQF is single threaded. However, I did design the extension as if it can receive calls from multiple threads (there is mutex synchronization code). At least it's future proof..

The method I proposed earlier with recursive callExtension calls will work great with JS_fnc_exec, but still looking for an elegant solution/workaround for JS_fnc_spawn.

[mrflay 4]

Just stumbled upon this thread: http://forums.bistudio.com/showthread.php?134053-Carma-A-C-to-SQF-quot-native-quot-Interface

Carma: A C++ to SQF "native" Interface

...

As for technical details, the entire system is blocking, which means there is no annoying asynchronous message queues exposed to the programmer or anything that creates superfluous execution time. In both SQF and C++ they appear as native functions to the programmer who does not need to worry about the behind the scenes operations for getting data from the SQF RealVirtuality (RV) engine.

It works on a simple messaging system based on two threads, the main entrant thread and an execution thread. A call originating from SQF first enters the DLL (which if it hasn't been called before initializes the execution thread which waits for a command) and the type of call is determined. If it is a call to execute a C++ defined function then it messages the execution thread that there is a new task to execute. While that task executes the entrant thread waits for a message from the execution thread. The execution thread will run the defined task, and if it contains a call to SQF then it tells the entrant thread to stop waiting and return with a message to the original SQF call telling SQF to execute the needed command. The execution thread now enters a waiting period while the SQF command is executed.

The SQF command is parsed and executed in SQF space and the result is then messaged back to the DLL which determines that the type of call is a return value from a previous SQF command generated in the execution thread. This value is messaged to the execution thread, which then continues executing the SQF function call and handles the return data, eventually returning it to the original user defined function called originally. This can happen multiple times for each SQF command executed. It simply messages back and forth between threads and SQF with the required data. When the user defined function is finished all that is required is that the user call a specific return function, which is essentially the same as the previous call back to SQF messages except that it is final and no longer will come back to the DLL. The execution thread goes into a waiting state, waiting for the next user defined command to execute, and the original entrant thread also idles till the next command from callExtension (which would be a call to execute a C++ defined function).

:)

[silentkiller* 11]

Sorry for this "recovery", i have a little problem/question:

since "XMLHttpRequest" object is not included here, how to load a file (i need to load a file from an URL) using JS?

Thank You.