zozo 104 Posted June 9, 2015 (edited) Remote Script Execution enhancements As you can see, the Remote Execution has been significantly changed in the recent days (on Dev-Branch only for now). BIS_fnc_MP is being replaced by new commands: remoteExec remoteExecCall You don't need to be worried - any changes to your scripts are not needed to be done as the BIS_fnc_MP function is being kept for the backward compatibility. There are going to be more changes for better handling the security. White listing and possibility to disable calling the function either on server or client or both will help to get more control over the remote execution. Check following pages: https://community.bistudio.com/wiki/Arma_3_Remote_Execution https://community.bistudio.com/wiki/BIS_fnc_MP https://community.bistudio.com/wiki/remoteExec https://community.bistudio.com/wiki/remoteExecCall Sometime in the following three weeks you can await another OPREP related to the Remote Execution changes and our plans with it. We'd be pleased, if you can tell us how are you satisfied with this solution or what more would you expect from it. Edited June 30, 2015 by RoyaltyinExile Share this post Link to post Share on other sites
Dwarden 1125 Posted June 15, 2015 adding myself for overwatch :) for obvious reasons Share this post Link to post Share on other sites
gossamersolid 155 Posted June 15, 2015 Any chance for a native supported callback functionality? Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 15, 2015 GossamerSolid said: Any chance for a native supported callback functionality? What do you mean? Share this post Link to post Share on other sites
gossamersolid 155 Posted June 15, 2015 Killzone_Kid said: What do you mean? I know you know what callbacks are in general, but I'm going to post this so that anybody else reading who isn't familiar can understand a bit -> https://en.wikipedia.org/wiki/Callback_(computer_programming) The natively supported callback functionality I speak of (in my mind) would be something similar to jQuery's Ajax methods -> http://api.jquery.com/jquery.ajax/ Basically in remoteExec, you'd have an optional code type parameter which would allow the caller of said method to get data back from the machine called. We can do stuff like this right now using our own methods, but I figured that since they are putting in the effort for proper network calls that we could get callback functionality built in at the same time. Is that a good enough summary or shall I go in a bit deeper of what I envision? Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 16, 2015 (edited) What about if target is multiple computers? Also when you say native, you mean the return from command is your callback result or you need to indicate another function to call? because if this is latter, then it could be done easily with current implementation and I would say give a lot of flexibilty in implementation too. Edited June 16, 2015 by Killzone_Kid Share this post Link to post Share on other sites
mreichi 10 Posted June 17, 2015 (edited) Hey, this is looking good! The new "target" parameter is really making our life easier. For example if you want to "switchMove" yourself we had to do fancy things so far. My suggestions are: ---- params: Anything - function arguments. Variables can be local. I suggest extending the CfgRemoteExec so it contains the allowed number of parameters and their data types. If this is missing, abusers could just send weird stuff over the pipe. This will add another layer of security, since sending unexpected parameters will/could cause remote script errors. Like you could just do "deleteVehicle 'NOPE'" to crash a peer if there is no parameter validation. class CfgRemoteExec { class Server { class Functions { class bis_fnc_Something { parameters = {"SCALAR", "STRING", "OBJECT", "GROUP"}; }; }; }; }; ---- new Parameters: isReliable / isOrdered / orderSlotId I have seen that there is a maximum package size that is used to determine if a UDP packet chain should be delivered reliable or not. Currently I am completely unaware if messages are delivered in order and reliable or not. But it would really really but very nice if one could control that behaviour. Most of the things will be reliable and ordered, but tiny non-important things can be send unreliable and unordered to lower the traffic/system load. For example: Broadcasting the server FPS to all clients every five seconds. This is a case where we really dont care if a packet is lost or delivered out of order. -- Changes for mode 1 / 1-turned on, taking whitelist into account When whitelisting is active, the authors have to add the functions/calls/messages to the config. Thats perfect. However, the parameters to steer the behaviour of where to send the packets to is still in the code. Thus abusers could send a message to everyone, even though its into intended to be delivered to the server by design. I suggest ignoring the parameters "target", "JIP"/"isPersistent" and "call" when in whitelisting mode and use the config values instead. Thus only the mod makers have full control to where to send the messages to, thus making it harder to abuse. For example "switchMove" would look like this in mode 0 Good version: // Switch the move on all clients, since switchMove only has local effect [player, "Acts_treatingWounded03"] remoteExec ["switchMove", -2, false]; Bad version: // Here I abuse this and send it to the server, so it will crash, since server doesnt like switchMove's [player, "Acts_treatingWounded03"] remoteExec ["switchMove"]; Switch move with whitelisting turned on Good version: [player, "Acts_treatingWounded03"] remoteExec ["switchMove"]; // there is no need to define more parameters, they are in the config Bad version: [player, "Acts_treatingWounded03"] remoteExec ["switchMove", -2, false]; // errors out, parameters cannot be set ----- Another suggestions is to define the allowed "directions" in the config file. If you whitelist "systemchat" you may want to allow this "SERVER->CLIENT" only, whilst "CLIENT->CLIENT" is not allowed. Maybe like this? #define REMOTE_EXEC_S2C 0 // Server to Client #define REMOTE_EXEC_C2S 1 // Client to Server #define REMOTE_EXEC_C2C 2 // Client (to Server) to Client ----- Last but not least I suggest changing the order of the parameters for "remoteExec" a bit. Currently it is like that: "WITH" remoteExec "WHAT" "WHERE" "WHEN" I suggest "WHERE" remoteExec "WHAT" "WITH" "WHEN" Example #define NON_JIP false _myGroup remoteExec ["systemChat", ["Kappa!"], NON_JIP] ----- Also, it would very very useful if you could add the "sender owner ID" as a parameter to remote calls. Currently there is no way to know which client send a message to the server. You have to do that manually right now. ----- Looking forward to your response, Bastian Edited June 17, 2015 by MrEichi Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 17, 2015 Interesting telemetry. Looks like there is no difference in how remoteExec or remoteExecCall behave: [color="#FF8040"] t [color="#8B3E2F"][b]=[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#7A7A7A"]"0"[/color][color="#8B3E2F"][b];[/b][/color] fn [color="#8B3E2F"][b]=[/b][/color] [color="#8B3E2F"][b]{[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"2"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"delay_ms"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#FF0000"]5000[/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]//<-- blocking nonscheduled delay 5sec[/i][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"3"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#8B3E2F"][b]}[/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"1"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#191970"][b]remoteExec[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#7A7A7A"]"fn"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"4"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]// [4784.56,17571.7,"0.000033","1"][/i][/color] [color="#006400"][i]// [4784.56,17571.7,"0.000518","4"][/i][/color] [color="#006400"][i]// [4784.57,17571.7,"0.015001","2"][/i][/color] [color="#006400"][i]// [4784.67,17576.8,"5.047093","3"][/i][/color] t [color="#8B3E2F"][b]=[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#7A7A7A"]"0"[/color][color="#8B3E2F"][b];[/b][/color] fn [color="#8B3E2F"][b]=[/b][/color] [color="#8B3E2F"][b]{[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"2"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"delay_ms"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#FF0000"]5000[/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]//<-- blocking nonscheduled delay 5sec[/i][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"3"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#8B3E2F"][b]}[/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"1"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#191970"][b]remoteExecCall[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#7A7A7A"]"fn"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#7A7A7A"]"debug_console"[/color] [color="#191970"][b]callExtension[/b][/color] [color="#191970"][b]str[/b][/color] [color="#8B3E2F"][b][[/b][/color][color="#000000"]time[/color][color="#8B3E2F"][b],[/b][/color] [color="#191970"][b]diag_tickTime[/b][/color][color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"micro_time"[/color] [color="#191970"][b]callExtension[/b][/color] t[color="#8B3E2F"][b],[/b][/color] [color="#7A7A7A"]"4"[/color][color="#8B3E2F"][b]][/b][/color][color="#8B3E2F"][b];[/b][/color] [color="#006400"][i]// [5053.99,17868.7,"0.000033","1"][/i][/color] [color="#006400"][i]// [5053.99,17868.7,"0.000078","4"][/i][/color] [color="#006400"][i]// [5054,17868.7,"0.014022","2"][/i][/color] [color="#006400"][i]// [5054.1,17873.7,"5.044320","3"][/i][/color][/color] Made with KK's SQF to BBCode Converter Share this post Link to post Share on other sites
Richard.biely 35 Posted June 19, 2015 (edited) Quote params: Anything - function arguments. Variables can be local. I suggest extending the CfgRemoteExec so it contains the allowed number of parameters and their data types. If this is missing, abusers could just send weird stuff over the pipe. This will add another layer of security, since sending unexpected parameters will/could cause remote script errors. Like you could just do "deleteVehicle 'NOPE'" to crash a peer if there is no parameter validation. While this might sound like a good idea (and it is :)) I'm afraid it would increase complexity of configuration a lot without adding much value in general. All our scripted commands have their parameters validated (fingers crossed) and if there are any which don't we'll fix that. Quote new Parameters: isReliable / isOrdered / orderSlotId I have seen that there is a maximum package size that is used to determine if a UDP packet chain should be delivered reliable or not. Currently I am completely unaware if messages are delivered in order and reliable or not. But it would really really but very nice if one could control that behaviour. Most of the things will be reliable and ordered, but tiny non-important things can be send unreliable and unordered to lower the traffic/system load. For example: Broadcasting the server FPS to all clients every five seconds. This is a case where we really dont care if a packet is lost or delivered out of order. Order is fixed. Imagine you'd want to create a soldier and equip him with a weapon. Packets arriving in an oposite order wouldn't do much good in this case. IsReliable will be considered because it might help save some traffic. Quote Changes for mode 1 / 1-turned on, taking whitelist into account When whitelisting is active, the authors have to add the functions/calls/messages to the config. Thats perfect. However, the parameters to steer the behaviour of where to send the packets to is still in the code. Thus abusers could send a message to everyone, even though its into intended to be delivered to the server by design. I suggest ignoring the parameters "target", "JIP"/"isPersistent" and "call" when in whitelisting mode and use the config values instead. Thus only the mod makers have full control to where to send the messages to, thus making it harder to abuse. I think this could be solved by something like: Server/Client { Commands/Functions { class switchMove{ allowedTargets=-2 }; // clients only class switchMove{ allowedTargets=0 }; // anyone class switchMove{ allowedTargets=2 }; // server only } }; with allowedTargets=0 being the default value. Mode parameter is about whether or not the set of commands/functions is enabled. Let's not change it's meaning completely. I think it does its job just fine for now. Quote Another suggestions is to define the allowed "directions" in the config file. If you whitelist "systemchat" you may want to allow this "SERVER->CLIENT" only, whilst "CLIENT->CLIENT" is not allowed. Maybe like this? I think that the previous paragraph covers this question as well. Quote Last but not least I suggest changing the order of the parameters for "remoteExec" a bit. Currently it is like that: "WITH" remoteExec "WHAT" "WHERE" "WHEN" I suggest "WHERE" remoteExec "WHAT" "WITH" "WHEN" Multiple orders of arguments were considered along with the one you suggested. In the end, however, we decided to stich with the WITH WHAT WHERE WHEN approach. One of the main reason being this way we can have a relatively fixed code on the right side and variable code on the left side (called scripted command/function arguments). Quote Also, it would very very useful if you could add the "sender owner ID" as a parameter to remote calls. Currently there is no way to know which client send a message to the server. You have to do that manually right now. Good suggestion. This is already considered. We have yet to decide in what way to deliver this datum to receiver's side, though. It's most likely going to be some sort of an event handler which you'll be able to register to. Quote Interesting telemetry. Looks like there is no difference in how remoteExec or remoteExecCall behave I think executing this in a more demanding scenario should result in a mesurable difference. RemoteExec runs in scheduled environment whereas RemoteExecCall is executed in one go. Therefore, RemoteExec can (but does not have to) take more time to execute. Edited June 22, 2015 by ribi Share this post Link to post Share on other sites
das attorney 858 Posted June 19, 2015 ribi said: Good suggestion. This is already considered. We have yet to decide in what way to deliver this datum to receiver's side, though. It's most likely going to be some sort of an event handler which you'll be able to register to. Hi, if you decide to go EH route for recording who is sending what, can you make it non-removable/secure? Sender of malicious code can potentially remove EH on server and then send naughty code (and then maybe re-add EH after to cover their tracks). Maybe the remoteExec whitelist would cover that (not sure though if they send it through MPKilledEH or other injection method). Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 19, 2015 ribi said: I think executing this in a more demanding scenario should result in a mesurable difference. RemoteExec runs in scheduled environment whereas RemoteExecCall is executed in one go. Therefore, RemoteExec can (but does not have to) take more time to execute. Yes, Richard, the scheduled function, would differ, but the command itself seem to return to main thread identically. Wasn't the case in early versions I think. Share this post Link to post Share on other sites
Richard.biely 35 Posted June 19, 2015 Well, bis_fnc_mp actually spawns the function and hence I don't think it behaved any different than it does now. Running something like this might prove me right (or not :) ): fn = { t2 = "(2) " + str diag_tickTime; hint t2; DBG_ignoreLocks [player]; t3 = "(3) " + str diag_tickTime; hint t3; }; t1 = "(1) " + str diag_tickTime; hint t1; {call fn;} remoteExec/remoteExecCall ["bis_fnc_call"]; t4 = "(4) " + str diag_tickTime; hint t4; // (1) 575.971 // (2) 576.006 // (3) 576.006 // (4) 575.971 Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 19, 2015 (edited) ribi said: Well, bis_fnc_mp actually spawns the function and hence I don't think it behaved any different than it does now.Running something like this might prove me right (or not :) ): fn = { t2 = "(2) " + str diag_tickTime; hint t2; DBG_ignoreLocks [player]; t3 = "(3) " + str diag_tickTime; hint t3; }; t1 = "(1) " + str diag_tickTime; hint t1; {call fn;} remoteExec/remoteExecCall ["bis_fnc_call"]; t4 = "(4) " + str diag_tickTime; hint t4; // (1) 575.971 // (2) 576.006 // (3) 576.006 // (4) 575.971 what DBG_ignoreLocks [player]; does? seems the difference is because of {} param, why is that? remoteExecCall ["fn"] 1 4 2 3 {} remoteExecCall ["fn"] 1 2 3 4 Edited June 19, 2015 by Killzone_Kid Share this post Link to post Share on other sites
Richard.biely 35 Posted June 19, 2015 Quote what DBG_ignoreLocks [player]; does? My bad. This function does not exist in dev. Imagine anything else in its stead. Quote seems the difference is because of {} param, why is that? I can see no reason for that (does no difference for me and I don't see why it should). Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 19, 2015 ribi said: I can see no reason for that (does no difference for me and I don't see why it should). Hmm you want me to make a video? Latest dev and in editor as well no mods, this is the behaviour. Share this post Link to post Share on other sites
mreichi 10 Posted June 22, 2015 I am not sure if this is already possible, but it would be nice to have the option to broadcast from the server to all client except a certain client. Example: Client A sends switchMove to server. By using normal broadcast the server would send to Client A, B and C. With my suggestion we could have a multicast sending it to Client B and C (since it doesnt make sense to send that message back to the sender). Share this post Link to post Share on other sites
killzone_kid 1331 Posted June 22, 2015 MrEichi said: I am not sure if this is already possible, but it would be nice to have the option to broadcast from the server to all client except a certain client. Example: Client A sends switchMove to server. By using normal broadcast the server would send to Client A, B and C. With my suggestion we could have a multicast sending it to Client B and C (since it doesnt make sense to send that message back to the sender). Quote targets (Optional): [default: 0 = everyone]Number - the function will be executed only on client with the given owner ID. When 0, the function will be executed on each client including the one where remoteExecCall was called from. When 2, it will be executed by server only. When negative ids are supplied, PCs with matching ids will be excluded from the list. Object - function will be executed only where unit is local. Side - the function will be executed only on clients where the player is on the specified side. Group - the function will be executed only on clients where the player is in the specified group. Array - array of any of types listed above. https://community.bistudio.com/wiki/remoteExecCall Share this post Link to post Share on other sites
kadinx 12 Posted June 29, 2015 It has been a few weeks now that the remoteExec has been effectively back on line on the Dev Branch and I am happy to say it is working much more efficiently in MP than Bis_fnc_MP that I have it replace. Usage includes mostly animations and sound effects executed on server side for all connected clients. Thank you! Share this post Link to post Share on other sites
AgentRev 22 Posted July 1, 2015 Are those remote exec commands filtered by BattlEye's remoteExec.txt? If yes, then this whole thing is utterly useless for client-to-server stuff. Share this post Link to post Share on other sites
kadinx 12 Posted July 1, 2015 I am not sure what you are referring to. I have made a small whitelist of functions and commands in description.ext and they are working consistently across clients. Share this post Link to post Share on other sites
Richard.biely 35 Posted July 1, 2015 AgentRev said: Are those remote exec commands filtered by BattlEye's remoteExec.txt? If yes, then this whole thing is utterly useless for client-to-server stuff. This has nothing to do with remoteExec.txt Share this post Link to post Share on other sites
AgentRev 22 Posted July 1, 2015 ribi said: This has nothing to do with remoteExec.txt Alright, thanks. I haven't tested the RE commands yet, I just wanted to verify that the arguments aren't also filtered via remoteexec.txt, because that would be a huge hassle to deal with. KadinX said: I am not sure what you are referring to. I have made a small whitelist of functions and commands in description.ext and they are working consistently across clients. It was just a question about the RE commands in general, not related to you. Share this post Link to post Share on other sites
das attorney 858 Posted July 1, 2015 Quote // Ability to jip: 0-disabled, 1-enabled (default) So does that mean it allows only JIP'ped players to send? Or non-JIP can send this function but JIP can/can't? Or only JIP players can receive? Or something else? Share this post Link to post Share on other sites
Richard.biely 35 Posted July 1, 2015 (edited) Description updated: https://community.bistudio.com/wiki/CfgRemoteExec It says that the function/command sent via remoteExec is allowed to be jipped (0=it is not, 1=it is). If it's not and you try to call remoteExec ["func",0,true/"myjipid"], remoteExec won't execute (this specific behaviour will work properly starting tommorow :) ). Edited July 1, 2015 by ribi Share this post Link to post Share on other sites
das attorney 858 Posted July 1, 2015 (edited) Excellent, thanks :) BTW: Does this new config structure here: https://community.bistudio.com/wiki/CfgRemoteExec Superceed this one here? https://community.bistudio.com/wiki/Description.ext#cfgRemoteExec Edited July 1, 2015 by Das Attorney Share this post Link to post Share on other sites