dedicated server memory usage constantly increasing

[dlegion 98]

hello, after 2 days of tests, i sadly found a lethal problem for dedicated server:
memory usage is constantly increasing in presence of AI....when it goes over 2000mb server crash with "out of memory" error.
at first i thinked it was my mission, and to exclude that i made a fresh new mission, just placed 3 playable units, and 6 rifle squads (8 men each) and 4 tank platoons (4 tanks each), all with a "move" waypoint and a "cycle" waypoint.
sadly....even there, the memory usage slowly increase. seems to be directly related to how many AI is there, for example, here are the results of test of different missions:

Spoiler

TEST MISSION WITH FEW AI PLACED FROM EDITOR (48 AI + 16 tanks):
908 MB at 08:16
919 MB at 08:50
921 MB at 09:00

------------------------------------------------------
DLEGION DWAR mission (MORE UNITS AND THEY RESPAWN):
1080 MB at 12:03 (in game time)   
1250 MB at 12:13
TEST-------2----------------
1100 MB at 12:03
1256 MB at 12:13
1264 MB at 12:15
1490 MB at 13:12
1536 MB at 13:26
1611 MB at 13:41
TEST-------3----------------
1240 MB at 00:24
1501 MB at 00:58
1546 MB at 01:03
1621 MB at 01:24
1662 MB at 01:35
TEST-------4----------------
1150 MB at 00:33
1208 MB at 00:38
1283 MB at 00:47
1349 MB at 01:00
... 6 hours later (or less), server crashed with "out of memory" error

 

----------------------------------------------------

AN OLD XENO DOMINATION mission (FEW AI COMPARED TO THE OTHERS, BUT STILL PRESENT):
1684 MB at 09:54
1759 MB at 10:03
1786 MB at 10.13
1800 MB at 10.40
1853 MB at 14:45

in an empty map with just that AI placed by editor...there should be no possibility of "external problems" (i use no mods, just pure vanilla for tests).
to see the memory increasing i used " #monitor 1 " at start,, but noticed that windows task manager is even better to monitor it!
hope that gets fixed soon, as it is game-breaking!
image of error found in the morning after leaving server running in the night:
https://ibb.co/k30HBa
 

[claws01 22]

Just increase virtual memory

Sent from my SM-G925T using Tapatalk

[dlegion 98]

hem...sorry for my ignorance...how should that solve the problem?
i suspect i didnt explained correctly....the problem is not just that server run out of memory...is the fact it keeps increasing memory usage for no apparent reason....so even if i double the memory, i just delay the crash!

[dlegion 98]

for a better troubleshooting, here is a list of all certain info up to now:

PROBLEM DESCRIPTION:
dedicated server memory usage keeps increasing at a variable rate apparently influenced by number of AI in mission.
it keeps increasing more or less constantly and endlessly. on 32bit "arma3server.exe" when over 2000MB it start to choke, decreasing down to 2 FPS,
and ultimately crashing with htis error:   https://ibb.co/k30HBa

3D EDEN CREATED TEST MISSION from NEW (empty):
(lets ignore my mission for now, and focus on standard test): Altis, 6 WEST rifle quads (8 men each) 4 WEST combined tank platoon (4 tanks each), 3 playable units,
 all with a move waypoint and a cycle waypoint. (can double the numbers to accelerate the process).

WHAT I DID EXACTLY STEP BY STEP TO REPRODUCE THE PROBLEM:
launch TADST v3.0.0, leave default settings (for testing purposes) checked box "persistent battlefield" and "auto-init",
selected "arma3server.exe" and tried also with "arma3server_x64.exe". (different starting memory usage, but both were increasing).
selected previously created TEST mission. Launched the server. Opened the windows task manager.
keeps monitoring the "arma3server.exe" process memory usage. Notice slow, but pretty constant increase in memory usage.

WHAT OTHER TEST I DID:
i tested my mission with all my scripts (and they are many) except the ones spawning AI. in my mission all AI is spawned by script, so there were 0 AI.
server did not had increasing memory usage.
i tested an old mission version (more than 6 months ago), that i'm sure it was running fine for at least 24 hours
(i remember playing on it with friends one night, then forgetting server up, then the following evening playing again on it with friends).
the test now showed a memory usage increasing.
i tested other missions made in the past, i dont have any documented previous check on them about memory usage, but now its increasing at various rate,
apparently related to numbers of AI.

MY HARDWARE / SOFTWARE / CONFIG ecc...
i5 4440 3.10ghz, 12Gb ram, 64bit, NO-SSD, connection: ADSL 7Mb upload 0.4Mb download.
(its the same configuration i had at the time of the 24 hours server running with no problem)
i always used 32bit "arma3server.exe" except for the very latest test.
LINK to latest version of my mission (here problem is accentuated due to spawn-despawn units based on player distance):
https://mega.nz/#!KtZ2GLiQ!-PxDcaPxXqjeuAaSeUEk2YT1e-UDZf22TXr0dhrVS5E
here the test results:

Spoiler

-------------- T E S T S -----------------------------------------
DWAR 20/07/2017 NO SCRIPTS                   = 848 MB seems stable
DWAR 20/07/2017 with R3F_LOG & MAG_REPACK    = 849 MB seems stable
DWAR 20/07/2017  +   minedetector            = 849 MB seems stable
DWAR 20/07/2017  +   Dfps                    = 849 MB seems stable
DWAR 20/07/2017  + Dclean & mineMarker       = 850 MB seems stable    // another test : 888.260 at 12:58 -- 895.288 at 13:11 -- 897.788 at 13:58 -- 897.884 at 14:06 -- 897.760 at 14:14 -- 897.836 at 14:25

DWAR 20/07/2017  +   GMOhostages             = 875 MB slowly increased to 912 in 15 minutes (spawned 9 units + 1 tank)....when deleted (by script after 30 minutes) memory stays at 910Mb
DWAR 20/07/2017  +   HQBtroops+HQRtroops     = 928 MB increasing 3MB every 1 minute (spawned 4 APC with crew)
DWAR 20/07/2017  +   GMOreset                = 1000 MB increasing 1Mb every 10 seconds (spawned 51 soldiers + 1 tank + 1 helicopter)
DWAR 20/07/2017  +   Dvehspawner.sqf         = 1110 MB constantly increasing 1 Mb every 1-3 seconds (spawned 2 helicopters, 2 airplanes, 8 civs on foot, 8 civs in cars, 2 apc, 2 soldiers, 2 boats)

 

 

TEST MISSION WITH FEW AI PLACED FROM EDITOR (48 AI + 16 tanks):
908 MB at 08:16
919 MB at 08:50
921 MB at 09:00
924 MB at 09:36

------------------------------------------------------
DLEGION DWAR mission (MORE UNITS AND THEY RESPAWN):
1080 MB at 12:03 (in game time)   
1250 MB at 12:13
---2----------
1100 MB at 12:03
1256 MB at 12:13
1264 MB at 12:15
1490 MB at 13:12
1536 MB at 13:26
1611 MB at 13:41
-------3-----------
1240 MB at 00:24
1501 MB at 00:58
1546 MB at 01:03
1621 MB at 01:24
1662 MB at 01:35
-------4------------
1150 MB at 00:33
1208 MB at 00:38
1283 MB at 00:47
1349 MB at 01:00
... 6 hours later (or less), server crashed with "out of memory" error

----------------------------------------------------

AN OLD XENO DOMINATION mission (FEW AI COMPARED TO THE OTHERS, BUT STILL PRESENT):
1684 MB at 09:54
1759 MB at 10:03
1786 MB at 10.13
1800 MB at 10.40
1853 MB at 14:45

hope it helps to solve it!

[dlegion 98]

just got a report from a friend, he recreated test mission on his PC, fired up dedicated, and had same problem of memory usage increasing!
he has an i7, 16gb ram and an epic 100MB download and 20MB upload connection.
 

[dlegion 98]

posted bug on tracker, but will surely be ignored by devs.
...better keep selling DLC than fix game.

[yxman 90]

i don't think that this crash "out of memory" is related with the dedicated server, only time i saw this was with missions using broken scripts or just bloated missions or broken mods.

[dlegion 98]

heh...in the past ( 6-9 months ago) i can confirm i runned a dedicated for 24 hours and was still running perfectly...but how can be a "broken script" or be a "bloated mission" ...when its just a fresh new map with some units in and no script or mods at all ?

[O.Languedoc 67]

This issue is still happening and isn’t fixed. Using HC client makes the memory rise even faster. Our server with alive running can last around 2 1/2 day. We have over 30gb or ram. 

 

 

[dlegion 98]

yep, i posted on feedbacktracker, but they dont care at all.