Sibolo

Hi all, I'm Sibolo from the BDR and we are having lots of trouble running our DS for our ATC Campaign. ATC is a multiclan PvP campaign with 80 players played once a week. During tests we have done, before the start of the campaign, our old server was showing great lag problems with very low server FPS and all strange of behaviours in game (scripts failing and so on) and it was really unplayable. We tried different missions (both light non scripted missions and the well known Valhalla) just to understand if it had to do with the mission we were using and the server kept this lagging behaviour, so our attention was focused on the server performance. We discovered to have a lot of lost packets on our server connection, and the hosting company told us that they had recently setup a filter on UDP traffic to avoid hacker attacks. We tried playing on another clan's server and found out that although the FPS were quite low, nobody had lags or any other problem. So we decided to get a new server on this clan's hosting company and we bought a more costly and powerful option to try and be sure we would get good performance. This is the new server specs: Once our server was up we started testing again to find out we had the same lag problems. I investigated on the server and found out that it was only using one of the 8 cores (I mean really using, some threads were on the other cores but they were not working intensely) and infact, although we had very low server FPS around 1-3, the maximum CPU usage was only 15%. So reading on some very interesting threads on this forum I found out that the server should have autodetected the number of cores, but that sometimes this fails so I put manually the Exthreads=7 and cpucount=4. This showed a very different behaviour, now the server was actually using up to three cores and the overall cpu usage went up to around 30%. So reading further I understood that since Hyperthraeding splits cpu's in two, in this case it would have been better to switch it off so that those 3 cores could have more power to them. So now HT is off and the server can use during games up to 75% of the cpu power. On our game this means that the FPS has greatly improved, infact during the same mission with almost the same number of players we have higher FPS which actually stays in the 30-40 zone....but suddenly it goes down to around zero, than after a second it comes back up to high values. This gives smooth game for about 30 secs and than a big lag to everybody: unplayable. Further investigating showed that maybe those lags can be due to sudden big I/O operations so I thought of a disk bottleneck and started looking at the possibility of using a ramdisk and tried the Fancycache program, but we are still having these problems. During the tests we have tried fiddling around with the server performance settings and we used the default values, the same values as our friend clan (which runs smooth but with low FPS) and also values from the Kelly's Heroes site who have a nice guide on these settings. So now we are really lost, we are still paying both servers and any of your help would be greatly appreciated.

I often come across this line in scripts: if (true) exitWith {}; especially at the end of the script as the last line of code. Wouldn't the script end anyway? Why is it added? Other times this command is used in the following way: if (!isServer) exitWith {}; to limit the rest of the execution of the script only on non server machines. Reading from the Biki about the Exitwith command it is clearly stated that: exitWith exits the execution of a loop defined by one of commands do, for, count or forEach. When you use exitWith not inside a loops, the behaviour is undefined - sometimes it may exit the script, sometimes some other scope, but this is not intended and designed behaviour of this command, and it is not guaranteed to work reliably. It exits the loop only, not the script. So why is this so widely used, and what would be the "correct" way to end script execution? Thank you.

Eliteness did the job very well, Thanks.

There seems to be some problem with your pbo, can't extract files with cPbo or PboView.

Could you also share your mission so we can see what you mean. Thanks.

@Party3AH: Yes, the config could be very useful.

Looking at these I believe that you do have a CPU bottleneck (although it might not be the only problem). Basically my understanding is this: Your overall CPU is divided in 8 cores (4 real and 4 logical due to Hyperthreading). To use your entire CPU power the programs you are running should be able to spread the workload on all cores generating and using enough threads. This does not appear to be the case of the Arma server. From what I have seen on our server and on our mission, only 3 threads (out of about 15 created by the server) where actually using CPU power. One was maxing out it's assigned core, the other two where only using a fraction (around 10-20%) of their assigned cores. This basically sums up to what you point out to be 145% CPU load (on a 800% maximum). But I believe that the mistake in it's interpretation is that it's not using more CPU power, not because it doesn't need it, but because it is not capable of doing it. So you end up using only a small amount of your overall CPU but experience low FPS. Infact, in general, one thread can only be executed on one core, so if an application would generate only one thread it would never be able to use the remaining 7 cores, even if it was struggling for CPU cycles. My advice is to disable HT as this has shown to give better performance (almost a 100% increment in FPS) by not splitting the cores. The main thread will run on a complete core and so will have more CPU power assigned to it raising your FPS. But in my experience this has not been enough, we are still having quite low FPS on large player count, and have disabled BE because when the server occasionally lags (freezes) many players are kicked out by BE and have a hard time rejoining. Also having 20-30 people disconnecting at once and trying to reconnect is very hard on the server and makes things even worse. Good luck.

I'm trying to improve my mission optimization knowledge by studying other well known missions like the Valhalla (from Dao.nu) or the AAS series especially for large player count games. I am focusing on how the code is structured through the mission when you need to launch several scripts that will be running for the entire game. The way this is done in those missions is that all the scripts are sqf files that are precompiled at game start and stored into functions. Then you have a main thread (big loop) that calls (or spawns) those functions in sequence throughout the whole game. Something like: _a =precompile a.sqf; _b =precompile b.sqf; _c =precompile c.sqf; ... //Main thread while {true} do { call _a; call _b; call _c; ... }; Opposed to this style of mission creation, often you find simply calling the scripts in sequence, and each script is an infinite loop. Something like: _a= execvm "a.sqf"; _b= execvm "b.sqf"; _c= execvm "c.sqf"; ... where each script contains its loop: while {true} do { blah; blah; blah; ... }; I would like to discuss the pros and cons of these two styles and how important it is in terms of difference in mission performance. Thank you.

Good!!! (Although it is in fact a 100% improvement :cool:) As far as mission optimization that some of you mentioned, basically I inherited the mission file from the previous campaigns where we didn't have so many problems even with a higher player count (50vs50). I did change many things, but always trying to do my best to keep it optimized. Most of the scripts are running only client side, variables are sent through the network only if really needed and so on. Mission dimension is not always the most important fact (although big missions are often overbloated) as you can write only one line of code and if it's done wrong it will mess everything up. But there is always a lot to learn on this topic.

I can recognize the core usage pattern (at least for this mission) involving mainly 3 threads in three different cores. I'm also quite sure that disabling HT is going to give it a boost, but I don't know how much that will be in terms of FPS. Anyway I think the server managed the game quite well even with those 2 FPS, didn't see anything too strange and at the very end both vehicles and infantry units were moving smoothly. Did any of your guys complain about lags? There was only one point in time that I disconnected and couldn't see the server on the list and someone was asking me if the server was stuck, but it was only for a few seconds. I then reconnected succesfully and didn't have any problems for the rest of the game.

I am analyzing some script's performance and have come across this situation a few times. Basically in some scripts you need to wait for a specific condition to be true then execute the code and get back to the start waiting for the condition to be true again. Many times I see it solved in this way (only an example): Wait for condition; execute code; call script again; //Script.sqf waitUntil {somecondition}; code; code; code; execVM "script.sqf"; if (true) exitWith {}; I would like to know the pros and cons, and if this could be improved or done in a completely different manner that will give better performance as this script is going to be ran several times. Thank you.

I understand that it does work in many cases to exit fom scripts and so it is widely used, but are there any alternatives? Maybe ones that will work inside EH?

Is there any circumstance a [left][color=#007700][font=monospace]while {[/font][/color][color=#0000BB][font=monospace]true[/font][/color][color=#007700][font=monospace]} do { [/font][/color][color=#007700][font=monospace]... ... }; [/font][/color][/left] loop will break out...some times I've seen it used like this: //Script.sqf while {true} do { waitUntil {somecondition}; code; code; code; }; execVM "script.sqf"; if (true) exitWith {}; and I interpret this that to be sure the script will always be active you put that execvm at the end just in case the loop breaks.

Yes, we run CO so that setting may need to be adjusted.

Ok so based on informations from this thread I've written a new server config file: This is because it is intended for an 80 people PvP, with no AI, on a 100mbit connection. The mission usually runs at 10-15 FPS (even lower) so the math I applied was this: CLIENTS x MAXMESS x PACKETSIZE x FPS = BANDIWTH REQUESTED 80 x 64 x 1024 x 15 = 78643200 I still have to test this config but please comment it.

How about client side? Do we need to do this math too? Usually home connections don't have big upload bandwidth, so is there the need to set these parameters too?

How does the number of people connected to the server come into this calculations about bandwidth? The server generates packets and sends them to people connected. Everyone receives different packets regarding his situation update from the server. We have said that if I put MaxMsgSend to 100 then the server can generate up to 100 packets for each "simulation cycle". If there's one person on the server than he will be receiving up to 100x50=5000 packets per second at 50 server FPS. Two people will receive 50x50=2500 packets per second at 50 server FPS. ... 50 people will receive 2x50=100 packets per second at 50 server FPS. 100 people will receive 1x50=50 packets per second at 50 server FPS. Is this right?

Thank you Dwarden for clearing this out, although I was hoping it could be a solution. I have disabled this checks and I must say that I haven't had those lag spikes anymore. On the most serious test we made the spikes were gone, but for some strange reason we had other problems like scripts not initialized, and about 20 people suddenly moved to open sea after a few minutes of play. So the test was aborted after about 10 minutes from start. When we moved to the other server, we managed to play for about 1 hour and then we had an enourmous freeze for about 3 minutes. When the server recovered, a lot of peolpe were kicked out by BE, and could not even see the server in the list to rejoin. The remaining people could instead continue playing. Another strange thing that is happening (since 1.60) is at mission start, we have a syncronization process in which the clients wait for public variables to be initialized by the server. This used to go almost unseen previously, but now it always takes about 1 minute, (in fact the first time we believed the server had frozen or crashed), but then it recovers and seems to work fine. All these problems are really making it hard to continue and keep up our ATC campaign, so I ask you all to continue suggesting solutions to these problems. Thank you.

I've read all your posts paying a lot of attention to all the matters. One thing that really interested me, following your suggestion about the regularCheck command is this: From the BIS WIKI: If you do not include the regularCheck option or set regularCheck=""; it will be activated. This means the server checks files from time to time by hashing them and comparing the hash to the hash values of the clients. Since newer server versions this has lead to some lag spikes on certain systems, because the whole file is hashed in one burst. (The heavy I/O operation essentially blocking the whole server application for 1-5 secs, depending on the file size) regularCheck is also known to cause sporadic (10 mins - 2 hrs) disconnects, terminating the client with "You were kicked off the game." on the client side and "Player Test disconnected." in the console log. To turn this function off, write regularCheck="{}";. But beware, this will also make the server more prone to cheating (even though most cheats are averted when connecting) The description of the effects of the regularCheck on "some" systems is amazingly similar to what I've come up with in fact my own description of the problem in the first post was: "we have higher FPS which actually stays in the 30-40 zone....but suddenly it goes down to around zero, than after a second it comes back up to high values. This gives smooth game for about 30 secs and than a big lag to everybody: unplayable. Further investigating showed that maybe those lags can be due to sudden big I/O operations so I thought of a disk bottleneck and started looking at the possibility of using a ramdisk and tried the Fancycache program, but we are still having these problems." How can I disable regularCheck? Is it the regularCheck="{}"; that Das Attorney suggested? Thank you.

Thank you for your reply: a) Server.cfg Basic.cfg: We have tried different configs including default values without solving our problems. The config that I retain the best choice is the one I copied off Kelly's Heroes site but I am not sure it is: b) All our tests have been done on the 1.59 stable release. We now have moved to 1.60 stable only a few tests were conducted without much improvement but further testing is needed (not easy to keep 80 people up for testing...). c) Surely I've done what I could to optimize the mission scripts. These were the first beleived cause of the problem, but then we tried different missions which had already been played in the past and had the same behaviour. Also tried some light missions, and the well known DAO Valhalla missions, still having problems. So this made us think it's mainly (but not only) a server issue.

I'm using a script for vehicle respawn, that makes the vehicle respawn in the same spot where it was destroyed. If the vehicle goes on fire and starts burning before exploding, it still burns even after it has respawned. Any help in stopping this fire?

I'm using this script (Vehicle.sqf): ---------- Post added at 01:39 PM ---------- Previous post was at 01:02 PM ---------- Tried it on a dedicated server and the problem was gone...