Remote Execution Enhancements

[zozo 104]

Remote Script Execution enhancements

As you can see, the Remote Execution has been significantly changed in the recent days (on Dev-Branch only for now). BIS_fnc_MP is being replaced by new commands:

• remoteExec
  
• remoteExecCall
  

You don't need to be worried - any changes to your scripts are not needed to be done as the BIS_fnc_MP function is being kept for the backward compatibility.

There are going to be more changes for better handling the security. White listing and possibility to disable calling the function

either on server or client or both will help to get more control over the remote execution.

Check following pages:

• https://community.bistudio.com/wiki/Arma_3_Remote_Execution
  
• https://community.bistudio.com/wiki/BIS_fnc_MP
  
• https://community.bistudio.com/wiki/remoteExec
  
• https://community.bistudio.com/wiki/remoteExecCall
  

Sometime in the following three weeks you can await another OPREP related to the Remote Execution changes and our plans with it.

We'd be pleased, if you can tell us how are you satisfied with this solution or what more would you expect from it.

Edited June 30, 2015 by RoyaltyinExile

[Dwarden 1125]

adding myself for overwatch :) for obvious reasons

[gossamersolid 155]

Any chance for a native supported callback functionality?

[killzone_kid 1331]

Any chance for a native supported callback functionality?

What do you mean?

[gossamersolid 155]

What do you mean?

I know you know what callbacks are in general, but I'm going to post this so that anybody else reading who isn't familiar can understand a bit -> https://en.wikipedia.org/wiki/Callback_(computer_programming)

The natively supported callback functionality I speak of (in my mind) would be something similar to jQuery's Ajax methods -> http://api.jquery.com/jquery.ajax/

Basically in remoteExec, you'd have an optional code type parameter which would allow the caller of said method to get data back from the machine called. We can do stuff like this right now using our own methods, but I figured that since they are putting in the effort for proper network calls that we could get callback functionality built in at the same time.

Is that a good enough summary or shall I go in a bit deeper of what I envision?

[killzone_kid 1331]

What about if target is multiple computers? Also when you say native, you mean the return from command is your callback result or you need to indicate another function to call? because if this is latter, then it could be done easily with current implementation and I would say give a lot of flexibilty in implementation too.

Edited June 16, 2015 by Killzone_Kid

[mreichi 10]

Hey,

this is looking good! The new "target" parameter is really making our life easier. For example if you want to "switchMove" yourself we had to do fancy things so far.

My suggestions are:

----

params: Anything - function arguments. Variables can be local.

I suggest extending the CfgRemoteExec so it contains the allowed number of parameters and their data types. If this is missing, abusers could just send weird stuff over the pipe. This will add another layer of security, since sending unexpected parameters will/could cause remote script errors. Like you could just do "deleteVehicle 'NOPE'" to crash a peer if there is no parameter validation.

class CfgRemoteExec
{
class Server
{
	class Functions
              {
                      class bis_fnc_Something 
                      {
                             parameters = {"SCALAR", "STRING", "OBJECT", "GROUP"};
                      };
              };
};
};

----

new Parameters: isReliable / isOrdered / orderSlotId

I have seen that there is a maximum package size that is used to determine if a UDP packet chain should be delivered reliable or not. Currently I am completely unaware if messages are delivered in order and reliable or not. But it would really really but very nice if one could control that behaviour. Most of the things will be reliable and ordered, but tiny non-important things can be send unreliable and unordered to lower the traffic/system load. For example: Broadcasting the server FPS to all clients every five seconds. This is a case where we really dont care if a packet is lost or delivered out of order.

--

Changes for mode 1 / 1-turned on, taking whitelist into account

When whitelisting is active, the authors have to add the functions/calls/messages to the config. Thats perfect. However, the parameters to steer the behaviour of where to send the packets to is still in the code. Thus abusers could send a message to everyone, even though its into intended to be delivered to the server by design.

I suggest ignoring the parameters "target", "JIP"/"isPersistent" and "call" when in whitelisting mode and use the config values instead. Thus only the mod makers have full control to where to send the messages to, thus making it harder to abuse.

For example "switchMove" would look like this in mode 0

Good version:

// Switch the move on all clients, since switchMove only has local effect
[player, "Acts_treatingWounded03"] remoteExec ["switchMove", -2, false]; 

Bad version:

// Here I abuse this and send it to the server, so it will crash, since server doesnt like switchMove's
[player, "Acts_treatingWounded03"] remoteExec ["switchMove"]; 

Switch move with whitelisting turned on

Good version:
[player, "Acts_treatingWounded03"] remoteExec ["switchMove"]; // there is no need to define more parameters, they are in the config

Bad version:
[player, "Acts_treatingWounded03"] remoteExec ["switchMove", -2, false]; // errors out, parameters cannot be set

-----

Another suggestions is to define the allowed "directions" in the config file. If you whitelist "systemchat" you may want to allow this "SERVER->CLIENT" only, whilst "CLIENT->CLIENT" is not allowed. Maybe like this?

#define REMOTE_EXEC_S2C 0      // Server to Client
#define REMOTE_EXEC_C2S 1      // Client to Server
#define REMOTE_EXEC_C2C 2      // Client (to Server) to Client

-----

Last but not least I suggest changing the order of the parameters for "remoteExec" a bit. Currently it is like that:

"WITH" remoteExec "WHAT" "WHERE" "WHEN"

I suggest

"WHERE" remoteExec "WHAT" "WITH" "WHEN"

Example

#define NON_JIP false

_myGroup remoteExec ["systemChat", ["Kappa!"], NON_JIP]

-----

Also, it would very very useful if you could add the "sender owner ID" as a parameter to remote calls. Currently there is no way to know which client send a message to the server. You have to do that manually right now.

-----

Looking forward to your response,

Bastian

Edited June 17, 2015 by MrEichi

[killzone_kid 1331]

Interesting telemetry. Looks like there is no difference in how remoteExec or remoteExecCall behave:

[color="#FF8040"]
t [color="#8B3E2F"][b]=[/b][/color]  [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#7A7A7A"]"0"[/color][color="#8B3E2F"][b];[/b][/color]
fn [color="#8B3E2F"][b]=[/b][/color] [color="#8B3E2F"][b]{[/b][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"2"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] 
[color="#7A7A7A"]"delay_ms"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#FF0000"]5000[/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]//<-- blocking nonscheduled delay 5sec[/i][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"3"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#8B3E2F"][b]}[/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"1"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#191970"][b]remoteExec[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#7A7A7A"]"fn"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] 
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"4"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]

[color="#006400"][i]// [4784.56,17571.7,"0.000033","1"][/i][/color]
[color="#006400"][i]// [4784.56,17571.7,"0.000518","4"][/i][/color]
[color="#006400"][i]// [4784.57,17571.7,"0.015001","2"][/i][/color]
[color="#006400"][i]// [4784.67,17576.8,"5.047093","3"][/i][/color]


t [color="#8B3E2F"][b]=[/b][/color]  [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#7A7A7A"]"0"[/color][color="#8B3E2F"][b];[/b][/color]
fn [color="#8B3E2F"][b]=[/b][/color] [color="#8B3E2F"][b]{[/b][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"2"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] 
[color="#7A7A7A"]"delay_ms"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#FF0000"]5000[/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]//<-- blocking nonscheduled delay 5sec[/i][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"3"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#8B3E2F"][b]}[/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"1"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]
[color="#191970"][b]remoteExecCall[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#7A7A7A"]"fn"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] 
[color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"4"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color]

[color="#006400"][i]// [5053.99,17868.7,"0.000033","1"][/i][/color]
[color="#006400"][i]// [5053.99,17868.7,"0.000078","4"][/i][/color]
[color="#006400"][i]// [5054,17868.7,"0.014022","2"][/i][/color]
[color="#006400"][i]// [5054.1,17873.7,"5.044320","3"][/i][/color][/color]

Made with KK's SQF to BBCode Converter

[Richard.biely 35]

params: Anything - function arguments. Variables can be local.

I suggest extending the CfgRemoteExec so it contains the allowed number of parameters and their data types. If this is missing, abusers could just send weird stuff over the pipe. This will add another layer of security, since sending unexpected parameters will/could cause remote script errors. Like you could just do "deleteVehicle 'NOPE'" to crash a peer if there is no parameter validation.

While this might sound like a good idea (and it is :)) I'm afraid it would increase complexity of configuration a lot without adding much value in general. All our scripted commands have their parameters validated (fingers crossed) and if there are any which don't we'll fix that.

new Parameters: isReliable / isOrdered / orderSlotId

I have seen that there is a maximum package size that is used to determine if a UDP packet chain should be delivered reliable or not. Currently I am completely unaware if messages are delivered in order and reliable or not. But it would really really but very nice if one could control that behaviour. Most of the things will be reliable and ordered, but tiny non-important things can be send unreliable and unordered to lower the traffic/system load. For example: Broadcasting the server FPS to all clients every five seconds. This is a case where we really dont care if a packet is lost or delivered out of order.

Order is fixed. Imagine you'd want to create a soldier and equip him with a weapon. Packets arriving in an oposite order wouldn't do much good in this case.

IsReliable will be considered because it might help save some traffic.

Changes for mode 1 / 1-turned on, taking whitelist into account

When whitelisting is active, the authors have to add the functions/calls/messages to the config. Thats perfect. However, the parameters to steer the behaviour of where to send the packets to is still in the code. Thus abusers could send a message to everyone, even though its into intended to be delivered to the server by design.

I suggest ignoring the parameters "target", "JIP"/"isPersistent" and "call" when in whitelisting mode and use the config values instead. Thus only the mod makers have full control to where to send the messages to, thus making it harder to abuse.

I think this could be solved by something like:

Server/Client {
 Commands/Functions {
   class switchMove{ allowedTargets=-2 }; // clients only
   class switchMove{ allowedTargets=0 }; // anyone
   class switchMove{ allowedTargets=2 }; // server only
 }
};

with allowedTargets=0 being the default value.

Mode parameter is about whether or not the set of commands/functions is enabled. Let's not change it's meaning completely. I think it does its job just fine for now.

Another suggestions is to define the allowed "directions" in the config file. If you whitelist "systemchat" you may want to allow this "SERVER->CLIENT" only, whilst "CLIENT->CLIENT" is not allowed. Maybe like this?

I think that the previous paragraph covers this question as well.

Last but not least I suggest changing the order of the parameters for "remoteExec" a bit. Currently it is like that:

"WITH" remoteExec "WHAT" "WHERE" "WHEN"

I suggest

"WHERE" remoteExec "WHAT" "WITH" "WHEN"

Multiple orders of arguments were considered along with the one you suggested. In the end, however, we decided to stich with the WITH WHAT WHERE WHEN approach. One of the main reason being this

way we can have a relatively fixed code on the right side and variable code on the left side (called scripted command/function arguments).

Also, it would very very useful if you could add the "sender owner ID" as a parameter to remote calls. Currently there is no way to know which client send a message to the server. You have to do that manually right now.

Good suggestion. This is already considered. We have yet to decide in what way to deliver this datum to receiver's side, though. It's most likely going to be some sort of an event handler which you'll be able to register to.

Interesting telemetry. Looks like there is no difference in how remoteExec or remoteExecCall behave

I think executing this in a more demanding scenario should result in a mesurable difference.

RemoteExec runs in scheduled environment whereas RemoteExecCall is executed in one go. Therefore, RemoteExec can (but does not have to) take more time to execute.

Edited June 22, 2015 by ribi

[das attorney 858]

Good suggestion. This is already considered. We have yet to decide in what way to deliver this datum to receiver's side, though. It's most likely going to be some sort of an event handler which you'll be able to register to.

Hi, if you decide to go EH route for recording who is sending what, can you make it non-removable/secure?

Sender of malicious code can potentially remove EH on server and then send naughty code (and then maybe re-add EH after to cover their tracks).

Maybe the remoteExec whitelist would cover that (not sure though if they send it through MPKilledEH or other injection method).

[killzone_kid 1331]

I think executing this in a more demanding scenario should result in a mesurable difference.

RemoteExec runs in scheduled environment whereas RemoteExecCall is executed in one go. Therefore, RemoteExec can (but does not have to) take more time to execute.

Yes, Richard, the scheduled function, would differ, but the command itself seem to return to main thread identically. Wasn't the case in early versions I think.

[Richard.biely 35]

Well, bis_fnc_mp actually spawns the function and hence I don't think it behaved any different than it does now.

Running something like this might prove me right (or not :) ):

fn = {
 t2 = "(2) " + str diag_tickTime; hint t2;
 DBG_ignoreLocks [player];
 t3 = "(3) " + str diag_tickTime; hint t3;
};
t1 = "(1) " + str diag_tickTime; hint t1;
{call fn;} remoteExec/remoteExecCall ["bis_fnc_call"]; 
t4 = "(4) " + str diag_tickTime; hint t4;

// (1) 575.971
// (2) 576.006
// (3) 576.006
// (4) 575.971

[killzone_kid 1331]

Well, bis_fnc_mp actually spawns the function and hence I don't think it behaved any different than it does now.

Running something like this might prove me right (or not :) ):

fn = {
 t2 = "(2) " + str diag_tickTime; hint t2;
 DBG_ignoreLocks [player];
 t3 = "(3) " + str diag_tickTime; hint t3;
};
t1 = "(1) " + str diag_tickTime; hint t1;
{call fn;} remoteExec/remoteExecCall ["bis_fnc_call"]; 
t4 = "(4) " + str diag_tickTime; hint t4;

// (1) 575.971
// (2) 576.006
// (3) 576.006
// (4) 575.971

what DBG_ignoreLocks [player]; does?

seems the difference is because of {} param, why is that?

remoteExecCall ["fn"]

1

4

2

3

{} remoteExecCall ["fn"]

1

2

3

4

Edited June 19, 2015 by Killzone_Kid

[Richard.biely 35]

what DBG_ignoreLocks [player]; does?

My bad. This function does not exist in dev. Imagine anything else in its stead.

seems the difference is because of {} param, why is that?

I can see no reason for that (does no difference for me and I don't see why it should).

[killzone_kid 1331]

I can see no reason for that (does no difference for me and I don't see why it should).

Hmm you want me to make a video? Latest dev and in editor as well no mods, this is the behaviour.

[mreichi 10]

I am not sure if this is already possible, but it would be nice to have the option to broadcast from the server to all client except a certain client.

Example: Client A sends switchMove to server. By using normal broadcast the server would send to Client A, B and C. With my suggestion we could have a multicast sending it to Client B and C (since it doesnt make sense to send that message back to the sender).

[killzone_kid 1331]

I am not sure if this is already possible, but it would be nice to have the option to broadcast from the server to all client except a certain client.

Example: Client A sends switchMove to server. By using normal broadcast the server would send to Client A, B and C. With my suggestion we could have a multicast sending it to Client B and C (since it doesnt make sense to send that message back to the sender).

targets (Optional): [default: 0 = everyone]

Number - the function will be executed only on client with the given owner ID. When 0, the function will be executed on each client including the one where remoteExecCall was called from. When 2, it will be executed by server only. When negative ids are supplied, PCs with matching ids will be excluded from the list.

Object - function will be executed only where unit is local.

Side - the function will be executed only on clients where the player is on the specified side.

Group - the function will be executed only on clients where the player is in the specified group.

Array - array of any of types listed above.

https://community.bistudio.com/wiki/remoteExecCall

[kadinx 12]

It has been a few weeks now that the remoteExec has been effectively back on line on the Dev Branch and I am happy to say it is working much more efficiently in MP than Bis_fnc_MP that I have it replace. Usage includes mostly animations and sound effects executed on server side for all connected clients.

Thank you!

[AgentRev 22]

Are those remote exec commands filtered by BattlEye's remoteExec.txt? If yes, then this whole thing is utterly useless for client-to-server stuff.

[kadinx 12]

I am not sure what you are referring to. I have made a small whitelist of functions and commands in description.ext and they are working consistently across clients.

[Richard.biely 35]

Are those remote exec commands filtered by BattlEye's remoteExec.txt? If yes, then this whole thing is utterly useless for client-to-server stuff.

This has nothing to do with remoteExec.txt

[AgentRev 22]

This has nothing to do with remoteExec.txt

Alright, thanks. I haven't tested the RE commands yet, I just wanted to verify that the arguments aren't also filtered via remoteexec.txt, because that would be a huge hassle to deal with.

I am not sure what you are referring to. I have made a small whitelist of functions and commands in description.ext and they are working consistently across clients.

It was just a question about the RE commands in general, not related to you.

[das attorney 858]

// Ability to jip: 0-disabled, 1-enabled (default)

So does that mean it allows only JIP'ped players to send?

Or non-JIP can send this function but JIP can/can't?

Or only JIP players can receive?

Or something else?

[Richard.biely 35]

Description updated: https://community.bistudio.com/wiki/CfgRemoteExec

It says that the function/command sent via remoteExec is allowed to be jipped (0=it is not, 1=it is).

If it's not and you try to call remoteExec ["func",0,true/"myjipid"], remoteExec won't execute (this specific behaviour will work properly starting tommorow :) ).

Edited July 1, 2015 by ribi

[das attorney 858]

Excellent, thanks :)

BTW: Does this new config structure here:

https://community.bistudio.com/wiki/CfgRemoteExec

Superceed this one here?

https://community.bistudio.com/wiki/Description.ext#cfgRemoteExec

Edited July 1, 2015 by Das Attorney