Linux Server CoreDumps

[apt 10]

Hello,

My Linux Server CoreDumps does anyone else have this problem or know why this is happening?

21:42:24 Mission aas100S V1.12 Bloodbath read from bank.

21:42:55 Roles assigned.

21:42:55 Reading mission ...

*** glibc detected *** /home/enclave/arma2operationarrow/server: free(): invalid pointer:

0x0e94aa28 ***

======= Backtrace: =========

/lib/libc.so.6[0x5f35a5]

/lib/libc.so.6(cfree+0x59)[0x5f39e9]

/home/enclave/arma2operationarrow/server[0x8158358]

/home/enclave/arma2operationarrow/server[0x81f84e0]

/home/enclave/arma2operationarrow/server[0x81f3cb2]

/home/enclave/arma2operationarrow/server[0x81f3d9b]

/home/enclave/arma2operationarrow/server[0x8594b82]

/home/enclave/arma2operationarrow/server[0x8594c5a]

/home/enclave/arma2operationarrow/server[0x8de9a07]

/home/enclave/arma2operationarrow/server[0x85b2ee4]

/home/enclave/arma2operationarrow/server[0x85b894f]

/home/enclave/arma2operationarrow/server[0x85b8ad5]

/home/enclave/arma2operationarrow/server[0x86a3829]

/home/enclave/arma2operationarrow/server[0x81ab505]

/home/enclave/arma2operationarrow/server[0x81ae267]

/home/enclave/arma2operationarrow/server[0x86ab6bb]

/home/enclave/arma2operationarrow/server[0x8e1bc75]

/home/enclave/arma2operationarrow/server[0x8e1bdc6]

/home/enclave/arma2operationarrow/server[0x8e2f285]

/lib/libc.so.6(__libc_start_main+0xdc)[0x59fe9c]

/home/enclave/arma2operationarrow/server[0x804ba91]

======= Memory map: ========

0056b000-00586000 r-xp 00000000 03:01 3932936 /lib/ld-2.5.so

00586000-00587000 r-xp 0001a000 03:01 3932936 /lib/ld-2.5.so

00587000-00588000 rwxp 0001b000 03:01 3932936 /lib/ld-2.5.so

0058a000-006dc000 r-xp 00000000 03:01 3932978 /lib/libc-2.5.so

006dc000-006de000 r-xp 00152000 03:01 3932978 /lib/libc-2.5.so

006de000-006df000 rwxp 00154000 03:01 3932978 /lib/libc-2.5.so

006df000-006e2000 rwxp 006df000 00:00 0

006e4000-006e7000 r-xp 00000000 03:01 3936548 /lib/libdl-2.5.so

006e7000-006e8000 r-xp 00002000 03:01 3936548 /lib/libdl-2.5.so

006e8000-006e9000 rwxp 00003000 03:01 3936548 /lib/libdl-2.5.so

006eb000-00700000 r-xp 00000000 03:01 3936550 /lib/libpthread-2.5.so

00700000-00701000 r-xp 00015000 03:01 3936550 /lib/libpthread-2.5.so

00701000-00702000 rwxp 00016000 03:01 3936550 /lib/libpthread-2.5.so

00702000-00704000 rwxp 00702000 00:00 0

00706000-0072d000 r-xp 00000000 03:01 3936547 /lib/libm-2.5.so

0072d000-0072e000 r-xp 00026000 03:01 3936547 /lib/libm-2.5.so

0072e000-0072f000 rwxp 00027000 03:01 3936547 /lib/libm-2.5.so

00731000-00743000 r-xp 00000000 03:01 5488733 /usr/lib/libz.so.1.2.3

00743000-00744000 rwxp 00011000 03:01 5488733 /usr/lib/libz.so.1.2.3

007e7000-007f2000 r-xp 00000000 03:01 3936549 /lib/libgcc_s-4.1.2-20080825.so.1

007f2000-007f3000 rwxp 0000a000 03:01 3936549 /lib/libgcc_s-4.1.2-20080825.so.1

007f2000-007f3000 rwxp 0000a000 03:01 3936549 /lib/libgcc_s-4.1.2-20080825.so.1

008ba000-0099a000 r-xp 00000000 03:01 5488735 /usr/lib/libstdc++.so.6.0.8

0099a000-0099e000 r-xp 000df000 03:01 5488735 /usr/lib/libstdc++.so.6.0.8

0099e000-0099f000 rwxp 000e3000 03:01 5488735 /usr/lib/libstdc++.so.6.0.8

0099f000-009a5000 rwxp 0099f000 00:00 0

00a9f000-00aaf000 r-xp 00000000 03:01 3936567 /lib/libresolv-2.5.so

00aaf000-00ab0000 r-xp 0000f000 03:01 3936567 /lib/libresolv-2.5.so

00ab0000-00ab1000 rwxp 00010000 03:01 3936567 /lib/libresolv-2.5.so

00ab1000-00ab3000 rwxp 00ab1000 00:00 0

08048000-0900d000 r-xp 00000000 03:01 1688537 /home/enclave/arma2operationarrow/server

0900d000-0901f000 rwxp 00fc5000 03:01 1688537 /home/enclave/arma2operationarrow/server

0901f000-0904a000 rwxp 0901f000 00:00 0

096aa000-1ed94000 rwxp 096aa000 00:00 0

b4300000-b438c000 rwxp b4300000 00:00 0

b438c000-b4400000 ---p b438c000 00:00 0

b441a000-b441b000 ---p b441a000 00:00 0

b441b000-b442b000 rwxp b441b000 00:00 0

b4470000-b4871000 rwxp b4470000 00:00 0

b5073000-b6074000 rwxp b5073000 00:00 0

b6074000-b6075000 ---p b6074000 00:00 0

b6075000-b6a75000 rwxp b6075000 00:00 0

b6a75000-b6a79000 r-xp 00000000 03:01 3932929 /lib/libnss_dns-2.5.so

b6a79000-b6a7a000 r-xp 00003000 03:01 3932929 /lib/libnss_dns-2.5.so

b6a7a000-b6a7b000 rwxp 00004000 03:01 3932929 /lib/libnss_dns-2.5.so

b6a7b000-b6a7c000 ---p b6a7b000 00:00 0

b6a7c000-b747c000 rwxp b6a7c000 00:00 0

b747c000-b747d000 ---p b747c000 00:00 0

b747d000-b7e7d000 rwxp b747d000 00:00 0

b7e7d000-b7e87000 r-xp 00000000 03:01 3932315 /lib/libnss_files-2.5.so

b7e87000-b7e88000 r-xp 00009000 03:01 3932315 /lib/libnss_files-2.5.so

b7e88000-b7e89000 rwxp 0000a000 03:01 3932315 /lib/libnss_files-2.5.so

b7e92000-b7f96000 rwxp b7e92000 00:00 0

b7f99000-b7f9b000 rwxp b7f99000 00:00 0

b7f9b000-b7f9c000 ---p b7f9b000 00:00 0

b7f9c000-b7fa0000 rwxp b7f9c000 00:00 0

b7fa0000-b7fa1000 r-xp b7fa0000 00:00 0 [vdso]

bfbfc000-bfc22000 rw-p bfbfc000 00:00 0 [stack]

WATCHDOG (27924): [Tue Jul 13 21:42:58 PDT 2010] Server died, waiting to restart...

[apt 10]

Here's some additional info.

CentOS release 5.5 (Final)

cat /proc/cpuinfo

processor : 0

vendor_id : GenuineIntel

cpu family : 6

model : 23

model name : Intel® Xeon® CPU E5410 @ 2.33GHz

stepping : 6

cpu MHz : 2327.522

cache size : 6144 KB

fdiv_bug : no

hlt_bug : no

f00f_bug : no

coma_bug : no

fpu : yes

fpu_exception : yes

cpuid level : 10

wp : yes

flags : fpu de tsc msr pae cx8 apic sep cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx lm constant_tsc up pni

bogomips : 4656.56

# free

total used free shared buffers cached

Mem: 1048720 1006604 42116 0 85348 496964

-/+ buffers/cache: 424292 624428

Swap: 0 0 0

# cat server.cfg

hostname="The Enclave 24/7 AAS";

password="";

passwordAdmin="";

reportingIP="arma2oapc.master.gamespy.com";

logFile="server_console.log";

motd[]=

{

"Welcome to The Enclave's AAS Server in OA!",

};

maxPlayers=50;

kickDuplicate=1;

disableVoN=0;

vonCodecQuality=0;

persistent=1;

regularCheck="";

onUnsignedData="kick (_this select 0)";

onHackedData="ban (_this select 0)";

onDifferentData="";

BattlEye=0;

class Missions

{

class aasgardenedentakistan // name for the mission, can be anything

{

template = "aas100_gardeneden.takistan"; // omit the .pbo suffix

difficulty = "veteran"; // recruit, regular, veteran & expert

};

class aaskingofchakchaktakistan // name for the mission, can be anything

{

template = "aas100_kingofchakchak.takistan"; // omit the .pbo suffix

difficulty = "veteran"; // recruit, regular, veteran & expert

};

class aasbloodbathzargabad // name for the mission, can be anything

{

template = "aas100_bloodbath.zargabad"; // omit the .pbo suffix

difficulty = "veteran"; // recruit, regular, veteran & expert

};

};

cat arma2oa.cfg

language="English";

adapter=-1;

3D_Performance=1;

Resolution_Bpp=32;

Windowed=0;

cat basic.cfg

MinBandwidth=320000;

MaxBandwidth=2147483647;

MaxMsgSend=256;

MaxSizeGuaranteed=1024;

MaxSizeNonguaranteed=64;

MinErrorToSend=0.0049999999;

MaxCustomFileSize=1600000;

Windowed=0;

arma2server config:

ARMA_DIR=/home/enclave/arma2operationarrow

CONFIG=server.cfg

PORT=2302

PIDFILE=${ARMA_DIR}/${PORT}.pid

RUNFILE=${ARMA_DIR}/${PORT}.run

LOGFILE=${ARMA_DIR}/log.${PORT}.txt

SERVER=${ARMA_DIR}/server

OTHERPARAMS=-mod=expansion

OTHERPARAMS=-exThreads=1

I can also send the corefile to bis if they want it.

Question though to other people running linux dedicated servers what OS are you using if you aren't having these problems? I went with CentOS thinking it would be the most stable but if everyone else is running ubuntu without problems then I could just easily switch to that.

Thanks to anyone that can help.

[killswitch 19]

I have a small CentOS 5.5 (32-bit) server here for testing, and it seems to work just fine. Do you get core dumps from that particular mission or does this happen for any mission you try to load?

What server version are you using? The latest OA server for Linux at this time is the 1.52.71900 one. (You're not using the ArmA II version, 1.07, are you? You have "-mod=expansion" as one of the two OTHERPARAMS variations in your arma2server start script which may suggest that you're force-loading that mod folder using the wrong Linux server version. If you use the OA server, the "expansion" mod folder is loaded implicitly.)

Edited July 14, 2010 by Killswitch

[apt 10]

That could be a problem, i'll try removing the expansion in the config and see if that helps. Im using the OA server, not sure on the version since it doesn't say it anywhere that I can find, it was the most recent stable as of about this time last week.

[Sekra 10]

I'm getting coredumps every now and then, specially after a mission that had gear selection in briefing screen, start a new mission after that and boom. http://dev-heaven.net/issues/11856

Edit: Just reading your post more accurately now, I dont know why you have exthreads param on, our linux server runs just fine without it. Do you get that dump every time you start the server, after a specific mission or anything else reproduceable? I'm running on archlinux 64bit myself and I had to get the 32 bit libraries for a2 and oa to work properly, check the biki for more info: http://community.bistudio.com/wiki/ArmA:_Dedicated_Server I've added the libraries I needed to install to get the server running on a 64bit linux. You should be able to tell the version of the server from the archive filename you downloaded btw.

Edited July 15, 2010 by Sekra

[apt 10]

It seems to only core dump on the aas missions, we run about 4 of them. But on the other hand we don't run alot of the other mission types, coop a few times and it hasn't crashed. So it may only crash under high server load or it could be the aas missions are giving us a problem. Just really hard to tell since we've only had the server up for a few days. We are going to try out a domination mission tomorrow so we'll see if it core dumps then or not.

The core dump is also quite random, sometime it's under heavy load, other times the server handles 30 people just fine, very intermittent so hard for me to diagnose without having the core file analyzed by BIS.

[apt 10]

32bit and the dump is random like I said.

I added on the exthreads as a troubleshooting measure, it dumps with or without it.

[apt 10]

I get core dumps while running a domination mission as well, so it clearly isn't the mission that's causing it to dump.

[killswitch 19]

I get core dumps while running a domination mission as well, so it clearly isn't the mission that's causing it to dump.

At what point does the crash occur? Is it some time into the mission or does the server crash when you try to load the mission? (If both, specify that) EDIT: You mention above that it's intermittent, so I guess it happens while a mission is played.

Can you offer the specific mission files that cause the crashes for download - maybe I or someone else can reproduce the crashes on other Linux dedis?

Edited July 16, 2010 by Killswitch

[apt 10]

It does not crash at mission load time, it crashes randomly during a mission.

Link to the aas OA mission pack.

http://dev-heaven.net/attachments/download/6487/AAS-V112-r583-OAMissionPack.7z

Then run bloodbath chakchak or garden or eden, all of those crash for me at some point.

Link to domination pack

http://www.armaholic.com/page.php?id=10993

I was running west r

So go ahead and give it a shot.

[Dwarden 1124]

please upload all crashdumps as per instructions here

http://forums.bistudio.com/showthread.php?t=102991

so Your CIT ticket can be easier tracked and resolved