Dwarden 1124 Posted February 12, 2009 (edited) I'm creating this thread with poll to obtain some feedback on one 'possible' feature for ArmA 2 (with very rare chance this to appear also for ArmA 1) server / client. Original thread with POLL was here : http://www.flashpoint1985.com/cgi-bin/ikonboard311/ikonboard.cgi?act=ST;f=57;t=77288 The feature is option for server to define file source forward for MP missions content (HTTP) and client's ability to download that content then from defined source. Description: The Arma 1 (and most likely 2 too) engine could easily support this feature , as MISSION related content is possible to load after engine start (contrary to map or other type of game changeable content like models). So technically this feature would require that instead of downloading MISSION file from ArmA *1/2 game server directly there will be inserted of small&simple portion of code telling game to download file from server defined URL location (if enabled server side). The routine about file placement and check if present/loadable etc. is then same/similar to already existing part of code. i split the poll into these sections tho i would like users to post with comments why they for or against this useful feature: 1. server admin/owner/donator + support this feature 2. just user + supporting this feature 3. anyone against this feature thanks everyone for taking part p.s. once again just in other words for these who not understand above actual situation : 1. client w/o mission establish connection with server 2. mission file is downloaded from server using server-client connection 3. when mission file is downloaded and not corrupt then client loads it into memory and enters missions positivies: + already done negatives: - will use bandwidth block of game server for mission distribution instead of game traffic - try imagine popular public server with 1k Unique Users / week download several 5MB missions ...ouch proposed situation: 1. client w/o mission establish connection with server 2. server tells client there is possible download location at http://www.somewebsite.net/folder/arma2mission.pbo (this part must be coded in) 3. clients starts wget on background and waits for error code or timeout on error code (this part must be coded in) 4. if error code is success then file is placed into mission directory like usual and loaded as usual (code already present ingame) 5. if file not match (already code present ingame) or if error code is fail / timeout then client may fall-back to try download from server using already existing method (this part must be coded in with reusing already present code) 6. when mission file is downloaded and not corrupt then client loads it into memory and enters mission *not need to use external opensource wget and use own portion of code, it was just example negatives: - some code must be added but not much (as most of code needed is already in) positives: + offloading the mission download to different source than gameserver ... Edited May 30, 2010 by Dwarden Share this post Link to post Share on other sites
.kju 3239 Posted February 12, 2009 What is/are the benefit(s)? I do imagine at least one, yet please go on. Share this post Link to post Share on other sites
Dwarden 1124 Posted February 12, 2009 erm i though the benefits are clear ... actual situation w/o this feature mean that any player w/o the mission in question is downloading it from the server: which means : - higher bw demand on server resulting to e.g. higher cost (bw block limits) - less bw for players already on server (possible issues with quality of gameplay) today most of modern MP oriented games and engines already offer this (e.g. engines from trio Epic, Valve, ID) for map/mission content the problem turns worse in moment You use 'big/huge' mission and run popular server (100s to 1000s unique players) with the feature you would then: + define mission source URL independently on the server location + save your server bw block + increase the quality of gameplay on server in short instead of 'wasting' precious bandwidth on your  server (not everyone have dedicated server line beyond T2) You can utilize some ultra fast public mirror hosting mission file(s) Share this post Link to post Share on other sites
.kju 3239 Posted February 12, 2009 while i agree that BI is engine is lacking so many aspects most other mp engines have, i doubt whether the effort for this would be well spent. the engine needs to handle several things for his or you make it very crude (unique identification, error handling, fallback, etc). instead the should focus on proper configuration, addon download (game restart is ok), more stable / resistant to errors, optimization, linux server etc etc Share this post Link to post Share on other sites
Dwarden 1124 Posted February 12, 2009 there is no need to complicate things and that's so neat on this feature game server: add query string with Mission_downloadable == "true"/"false" add query string with Mission_download_URL == "*" game client: if client sees true for Mission_downloadable then client uses Mission_download_URL and auto adds "http://" to URL defined then client executes e.g. wget to download URL+mission filename (+ needed switches) then client waits till error returned or file downloaded or other error returns or else timeout then client tries load mission like now (same part of code used for actual download from server) yet if any of above fails client can easily fallback to already existing direct download from game server code also client can use file operations code already present that's all ... simpler it can't be p.s. don't mismatch this with any addon content ... that's completely different and unrelevant unrelated to this feature Share this post Link to post Share on other sites
walker 0 Posted February 12, 2009 Hi all I think Q has completely misunderstood and indeed obfuscating the simplicity and the importance of this proposal for Join In Progress (JIP) And how many ways this will improve ArmA. JIP Lag and Mission load speeds Basically this addresses Mission load speeds and JIP Lag. It does so in three aspects. 1) When you first play a mission or when a mission is updated, you all have to download the mission. If it is one of the big servers say 60 plus players it causes loads of problems: It can take up to 10 minutes, people get bored, there are dropped connections adding to the load if they half way through a download. Downloading the mission in advance removes this problem. Torrents might be a useful additional location for missions then just put up a server index page for the torrents. That way a clan can trickle their servers mission and their addons, no need for emails to tell people to update addons either, it all becomes automated. 2) JIP lag for existing clients/players on a server when some one joins a server. The server has to slow to send the whole mission file which with sound and even video files to every new client. We have all experienced the dreaded Lag in a helo coming into the Hot LZ where we all crash and do not even know why, as a new player from a different continent joins with a mission they have never downloaded before (it is always polite ask on Team Speak if you can JIP in) JIP lag for a pilot is a killer. Downloading the mission in advance removes this problem because all downloads are done separate from the server, mission load JIP Lag is removed, synchro Lag still exists but more on that later. 3) JIP lag for a new client/player is removed, meaning you join the game faster. Synchro Lag still exists but more on that later. Synchro Lag This is outside what Dwarden is proposing. This mostly affects big missions and Persistent Server missions like Warfare and Evolution and only when Joining In Progress When you JIP into a mission that has been going for some time the mission environment has changed. Units have moved/changed position from the mission download positions, bushes trees lampposts have been knocked down by bad drivers (mostly me that, sorry) buildings blown up ammo load outs are all different people shoot bullets, lots of dead bodies where the bullets hit people, smoke, explosions etc. What is needed for Synchro Lag is a server farm architecture essentially the ability to use client as a server. Nothing special just an additional server that acts as a dumb client to the main server and can be used by JIP players to synchronize to. Another solution may be to use peer-to-peer updating from players in game, but there are a whole bunch of technical issues as pert of that route. Anyway I think we should give Dwarden our full support on this. Kind Regards walker Share this post Link to post Share on other sites
.kju 3239 Posted February 12, 2009 1.15 fixed the mission download problem. IC with 100 players had no longer any problem during JIP, mission start, long mission download. at least to our experience this is no longer true. bandwith is the least issue of the A1 DS imo. the other stated ones are way more important imo. so i see this as a very minor issue in these days. again you need to have the mission still on your server, make a compare if the URL is offering the same file. Share this post Link to post Share on other sites
Dwarden 1124 Posted February 12, 2009 @Q the client already compares the file (if you have the mission already) and if not it redownloads it ... so again that part of code is already included in client ... and please don't think in scope of 'rare server' with 'uber fast' line running 'small' well spread mission for 'limited' number of clients ... this is aimed at: - public server - 64-128 or more player server - mission size is 1 MB or bigger for starter let say You have 2000 unique players / month and you rotate 10 missions with total size 25MB (avg 2.5MB/mission) and let say each mission gets 2 builds per month if 25% of players already got these missions other way then bw block needed is 'only' 0.75*2000*25*2 = 75000 MB (just for mission itself downloads) now let's not even try imagine very popular server with way more and lot bigger missions and even more often updated ones ... in short something what ArmA 2 try aims to be right ? @ walker You not lightened my argument either as some of the issues (or most) were resolved in public 1.15 beta already anyway You still right that 'cluttering' the server bandwidth on server where is some 'limit' (be it max bw block or actual throughput) still exist and that's exactly what above suggested feature remove Share this post Link to post Share on other sites
Lt.Kettch 0 Posted February 13, 2009 I am with Dwarden on this one Share this post Link to post Share on other sites
SWAT_BigBear 0 Posted February 13, 2009 (edited) Dwarden, You know I am for this, since I use this form of "redirect" for RvS and CoD servers. Edited February 17, 2012 by SWAT_BigBear I'm to I am (new forums) Share this post Link to post Share on other sites
ombas 0 Posted February 13, 2009 i'm all for an automatic http redirect for the user. Â whatever makes it easier to get on and improves the play experience is a good thing for everybody. Â i emphasize AUTO http redirect. keep up the good work! Share this post Link to post Share on other sites
Hud Dorph 22 Posted February 13, 2009 I was a counterstrike-source server operater when this was introduced to css, and it took a massive bandwidth-load off the servers, this sounds like a good idear. Share this post Link to post Share on other sites
Maddmatt 1 Posted February 13, 2009 Agreed. Downloading the mission straight from the server is: 1. Slow, can be a problem with large missions. Having to sit and wait for the mission to download at speeds much slower than your connection could handle is frustrating. 2. Uses bandwidth that should be used to run the game. Might cause desync in some cases. Maybe it could be done for addons, although I don't know if I would be happy with loads of low-quality addons being forced on me. And that would be more complicated to implement. Share this post Link to post Share on other sites
max power 21 Posted February 13, 2009 Agreed. Downloading the mission straight from the server is:1. Slow, can be a problem with large missions. Having to sit and wait for the mission to download at speeds much slower than your connection could handle is frustrating. 2. Uses bandwidth that should be used to run the game. Might cause desync in some cases. Maybe it could be done for addons, although I don't know if I would be happy with loads of low-quality addons being forced on me. And that would be more complicated to implement. It would be too hard to unify everyone's addon structure and it would hog up your hard drive. you think your arma dir is big now? Imagine if you had every popular addon multiple times, depending on the tastes of the server admins! Share this post Link to post Share on other sites
Dwarden 1124 Posted February 14, 2009 i repeat it again please don't mismatch this feature with addons / maps etc. downloading which may seems similar but is absolutely different ... that would mean radical changes into engine which is exactly opposite where feature above aims ... so keep it simple as it is ... Share this post Link to post Share on other sites
Brendon 0 Posted February 14, 2009 Thank you walker for explaining it. I don't think English is the Topic creators main language, so didn't understand him too much. Anyways, I support the idea as a user, anything to stop JIP lag as it can really kill a fire-fight and it also makes it quicker to join since it won't use the servers resources but its own. Share this post Link to post Share on other sites
Spooner 0 Posted February 25, 2009 Although I'm in favour of this, have you considered using ArmAlib to do this as a community side (tiny missions that are downloaded normally from the server, but that downloads the mission-specific files from an external source, be that HTTP or database)? Certainly not happening for ArmA before A2 at the very least, so this might be a better option, though getting the average user to use something like Armalib is harder than getting them to predownload the mission, in all likelyhood. Share this post Link to post Share on other sites
bluevein 0 Posted February 26, 2009 This Idea never even crossed my mind but what a good one. I'm with you 100%, server owner/admin & user + I hope BI concider this. Share this post Link to post Share on other sites
Dwarden 1124 Posted March 18, 2009 Although I'm in favour of this, have you considered using ArmAlib to do this as a community side (tiny missions that are downloaded normally from the server, but that downloads the mission-specific files from an external source, be that HTTP or database)? Certainly not happening for ArmA before A2 at the very least, so this might be a better option, though getting the average user to use something like Armalib is harder than getting them to predownload the mission, in all likelyhood. the point is simple feature out of box as part of the game client/server operations ... anything what needs user interaction with 3rd party compoenents just for mission download forwarding downplays such feature into oblivion ... Share this post Link to post Share on other sites
bravo 6 0 Posted March 18, 2009 Dwarden, to be honest i also had some difficulties to understand the main idea of the pool, i feel its not quite clear yet. I believe there are also more people that would like to participate on this but they just don't understand these kinda techniques. So from what i understand to have better performance it would need 2 servers, one for the game it self and another server(s) for hosting missions (optional/resource). These way people would download missions from different sources and prevent lag. Is this the idea? If so, i still don't know what option to choose in the pool Share this post Link to post Share on other sites
Dwarden 1124 Posted March 18, 2009 what you not understand ? ok i updated the first post and it's under the P.S. so read and tell me if You now able understand what i got in mind and about the 2 servers need ... no need for two servers ... if there is UBLIC and FAST http/ftp mirror for missions then You 'lighten' the load off your server to that download mirror ... yet the option to download mission from Your server still exist in case the mirror goes down or client fails to connect to it or correctly download mission file just note nearly every MP game now supports this ... Share this post Link to post Share on other sites
Alex72 1 Posted March 18, 2009 Wasn't too clear for me either to understand fully lol. But if its about taking stress away from the server running the missions and improving JIP im all for it. Sounded scary with one of the negatives though. If server rotating heeps of maps me as a user will have to sit back and just take all of them before joining. Dont know if i got that right? But if this was a feature out of the box server hosts would know not to use too big rotations. So that wouldn't be a problem. Small note in the manual (or extra sheet if manual is done) could explain this easy. The more solid the core game is - the better. Good job Dwarden coming up with the idea. PS. Hope i got your point somewhat correct. Regards Alex Share this post Link to post Share on other sites
Dwarden 1124 Posted March 18, 2009 fail again you don't need download all the heap of maps ... just the one you joining in ... now in ArmA if you download modified Warfare with 25MB size it's downloaded from the server it's hosted on ... with this server admin may decide that you may download it from somewhere else (usually faster) which helps game server too ... ... where is that smilie with smiley running around crazy screaming Share this post Link to post Share on other sites
TeRp 1 Posted March 20, 2009 Well, I understood you perfectly and I'm supporting your idea. //type fixed Share this post Link to post Share on other sites
Alex72 1 Posted March 21, 2009 @Dwarden LOL! Sorry mate for making you nuts. Misunderstood the mission rotation bit. Thaught you wrote it was one of the downsides to have to download it all. No harm done. Still supporting. Even more now. Alex Share this post Link to post Share on other sites
