Serious Bug: JIP-LAGs (+ possible easy-fix idea)

[Hellop 0]

Ok, we have have followed the instructions and done some testing.  The best that we can determine is:

Summary:

1. A player with very low bandwidth values: 17, 27, 25, causes everyone to lag when that person joins or disconnects, no matter if local server has focus or is alt-tabbed out.  This user got 861kilobits/sec down, 307kilobits/sec up, connection speed results from speedtest.net.

2. When the local server host was alt-tabbed out of the game, join/disconnect lag seemed less for everyone.

Tests performed:

1. 10 players. Server host playing game. 11th player joins.

# Tests: 10

Result: Everyone gets desync.  Some yellow/red chain icons.

2. 11 players. Server host playing game. 1-2 player(s) disconnect/reconnect.

# Tests: 10

Result: Everyone gets desync.  Many yellow/red Chains.

3. 8-10 players. Server host alt-tabbed to windows. 1-2 player(s) disconnect/reconnect.

#Tests: 10

Result: There was very little desync for everyone.  Everyone got some light desync when the person joined, but no lag/chain icons.  Joiner got the most desync.  All desync disappeared within 60 seconds.  2/10 times the person reconnecting only had a yellow chain for a short time.

Notes: You need to have over 10 players to reproduce lag consistently.

Conclusion: We tried to reproduce our tests as exact as we could by changing only whether the server host had focus or not.  However, we lost a couple players when we did the alt-tabbed-out-to-windows test, so that may have contributed to the reduction of lag we saw.  At least one of our players had custom skins/sounds, which may have affected the results.  Something we think is important, is that when any 1 player is connected that has low bandwidth, connects/disconnects by any player, causes everyone to lag.  That is a definite bug irrespective of the focus of the local server rendering engine.

-To reproduce that error, have someone connect to a server on dial-up.  Then have any player connect/disconnect.  You should then see desync and chain icons.

Thanks to the following people from the Sahrani-Life Arma Mission Community who helped with this test:

x3r0, St3v3-O, ACS|Pat|LTC, hellop, Giorgio, Die Chronik, Datscher, Blunt, =BBUK=MagicShot, |ACS|Gamster|SGT|, |ACS|DaKa|WO2|

[Hellop 0]

When using custom files, there is still a period when the custom files are being broadcasted to all other players, which essentially blocks any other game traffic.

Can rate-limiting on the server's firewall provide a possible work-around for the custom file issue?

We hope we will bring some solution for this as well, but this is definitely not something you could test with 1.14.

Would testing with/without custom files help?  Would setting MaxCustomFileSize in basic.cfg prevent custom file lag?

-To Whom it may concern, Thanks for your help.

[ModaFlanker 0]

If you need more testers, post a timeperiod ahead of time and I'll do my best to be there.

A fix for this would dramatically increase the quality of ArmA.

[mr.g-c 6]

Hellop, how fast was the Line of the guy who run the server?

I bet it was to low and therefor caused desync.

[Hellop 0]

Hellop, how fast was the Line of the guy who run the server?

I bet it was to low and therefor caused desync.

mr.g-c, the guy's DSL has 2MBit upload.

That's pretty good for ~10 players. Unfortunately, it seems that you cannot "login" to a locally hosted game, so you cannot run the #monitor command to see what the bandwidth use it.

If anyone knows how to run #monitor from a locally hosted server, please share.

Of course you are right. Who knows what effect his 2Mbit cap had on our tests. Especially since it was my understanding that alt-tabbing to windows should produce more lag... but we saw the opposite.

From our tests, it seemed that running a locally hosted server that is alt-tabbed out worked better then a dedicated server... But, maybe the host has a slow computer, so rendering the game lagged everyone else out.

It is curious that kicking the guy with the 1mbit/384kbit DSL improved everyone's game. But, maybe he had some crappy addons. It's difficult to test in an environment which you cannot control.

I think the test could be improved by: Having 12+ players. Ensuring no one has addons or custom files. Host on a fast PC with a 5mbit upload connection.

[NightShade 0]

When using custom files, there is still a period when the custom files are being broadcasted to all other players, which essentially blocks any other game traffic.

Can rate-limiting on the server's firewall provide a possible work-around for the custom file issue?

We hope we will bring some solution for this as well, but this is definitely not something you could test with 1.14.

Would testing with/without custom files help? Would setting MaxCustomFileSize in basic.cfg prevent custom file lag?

-To Whom it may concern, Thanks for your help.

Rate limiting on the server's firewall would only cause the lag to last longer as it would mean it takes longer to send that mission file to the player (as it freezes while it's sending it) so you'd do the opposite to what you want to do here.

[thirdrm 0]

If you can wait about a month, I will have a 10/10 connection.... and we can re hoast instead of my slower 6/2 connection.....

Btw.. im ACS Pat

Sid note: I will be at my parents house for the rest of the week, where I believe i have 15/15 .... We could test it again

[Hellop 0]

Rate limiting on the server's firewall would only cause the lag to last longer as it would mean it takes longer to send that mission file to the player (as it freezes while it's sending it) so you'd do the opposite to what you want to do here.

Nightshade, how do you know this to be true?

The problem is that the server is not calculating available bandwidth correctly, so it is sending information too fast. So, the fix would be to send the information slower. Which means that it would take longer to send the data.

Just because it would "take longer" is not sufficient information to determine that it would not work. Is there some other reason you think it would not work?

Really, I think only the programmers of Arma can answer this question since only they know what kind of verification system there is in the application to determine the success of a file sent via UDP. It could be that it sends parts of whatever data is being sent, then waits for some type of UDP acknowledgment packet, like a checksum. It could very well be that while it is waiting, the server could process other network I/O, and hence, rate limiting would help. On the other hand, it could be that if the server doesn't receive an "acknowledgment" in a certain time period, it re-sends the data, in which case, rate limiting would only compound the problem.

-hellop

[cross 1]

What i understand from Suma's explanation, probem lies in the game itself rather than server bandwith settings. The "Arma Bandwith controller" need to be tweaked so that it reads the FPS correct from another source than the rendering subsystem and realizes that there is enough bandwith it can utilize for syncronizing purposes without limiting outgoing data/packages to the existing players or existing players bandwith. Limiting/allowing bandwith should not have any effect as the data, that game reads to base its calculations, is wrong regardless of the current bandwith. Game itself is diverting traffic from existing players to the joining player.

@Suma...if you are able to provide an updated release of "ArmaServer.exe" without any client update requirements, I'm sure IC-Arma tournament will be happy to help you stress test it with 80+ players before any beta release.