1.90 update - Troubleshooting

[Dwarden 1125]

new 1.90.145454 profiling branch performance binary build with more fixes is out

 

[ZeakDK 10]

I've run into a problem after the recent 1.90 update.

 

What happens is we load into the server and we just sit there, with a blackish screen with the ocean sound in the background.

This only happens when (certain) mods are enabled. I managed to track the issue down to Unsung and ACE.

I get no errors in the logs, that I recognize could be the issue. The server is active in the server list and joinable, but once you join you just sit there.

I've tried this on two separate instances, though they are both from the same host, they are in different countries.

 

I've tried everything that has been posted in this thread and the Discord server.

I've tried:

- Disabling Verification Signature

- Re-downloading the mods (multiple times)

- Re-installed the server (multiple times)

- Removing the cache (from both the stable and perf branch)

- Removing all bikeys

- Perf branches

- Disabling UFW (in case some new port was added)

All of this in various combinations.

 

I initially rolled back to 1.88 and that fixed it, but I also discovered that everything works when I disable BattleEye on the server.

 

[TheNightstalk3r 6]

3 hours ago, kelley said:

I agree with this. My players are not going to play with anything other than what auto downloads for them. Most players won't roll back.That is just a fact. Then it's my fault they can't log on to my server. I don't understand why B.I. thinks this is okay?

You rather never have an update? Or did you maybe even read the story behind this?
The development was done.

The RC (Release Candidate) branch was tested thoroughly.

Only on the public release these issues were found. You can roll back all you want and complain about BI not testing or get a grip on yourself and assist in getting this issue out of the way once and for all.

[jgaz-uk 132]

Well lucky you Stanhope, there are numerous long time ArmA, A2 & A3 & ofp customers of Bis that cant use their dedicated servers as they could before update 1.9.

[kelley 11]

1 hour ago, TheNightstalk3r said:

You rather never have an update? Or did you maybe even read the story behind this?

These threads always devolve into this weirdness.

I've run dozens of BI servers over the years. Arma 2, Arma3,and Dayz.

1 hour ago, TheNightstalk3r said:

You can roll back all you want and complain about BI not testing

I never complained about BI not testing in all the years I've been hosting servers.

The problem I have is losing my players because there is no easy way for them to log on to my server . Why is that? Because BI hasn't offered one yet. Like I said before, it is costing me money.I can't wait another day.It is obviously time to move on.It does seem after all the money and time I have invested I would receive a little better support from BI. Not surprised but still disappointed. 

[Dwarden 1125]

another important note for recent profiling binaries,
use ``-servermod=`` for mods not available at client,
which are used only at server, for mods at both use ``-mod=``

 

(this will become mandatory for stable main branch too)

[oukej 2910]

15 hours ago, kelley said:

Why is that? Because BI hasn't offered one yet.

With the current profiling (v05) you should be able to debug what pbos are causing the issue. Chances are there's a .pbo on the server which is required from the clients but the clients don't have it.

[celticalliance 31]

While I understand all the remarks about logging and debugging there are actually people who don't know how all that works. I am one of those.

 

So if I understand correctly the server owners and hosts are now supposed to check their whole mod setup on server level? I had quite some issues with setting the servers up so they were running for everyone and now I have to go through that again? Just to fix this one issue?

 

Before moderators decide I need moderation again, for people like me (who have only a basic understanding of debugging etc., and do not run the servers themselves) this is impossible to do. It takes people like me hours to change things around, time I do not have.

 

Removing the two files mentioned did not solve anything. I also don't have the time to reinstall my servers and set up everything again. This update causes nothing but issues for a lot of server owners and hosts and as someone already said this is costing everyone players. It also makes people steer clear of the game. A little more communication about how servers are affected by the new security would have been nice.

 

I am not referring to the testing phase, but the fact the issues started after release says enough. I for one don't even know what I can do anymore to get this fixed and the material presented to me so far doesn't help me much either.

[ZN Anhor 40]

6 hours ago, Dwarden said:

another important note for recent profiling binaries,
use ``-servermod=`` for mods not available at client,
which are used only at server, for mods at both use ``-mod=``

 

(this will become mandatory for stable main branch too)

 

LOL

You kidding us Dwarden? This is how my commandline looks since 2 years .........

 

And BTW, what @celticalliance said above is absolutly correct! I run a server for me, my friends and some players who wanna join. (BTW, most player who own A3 did not know how they handle Mods)

I am not a programmer, I can not do C ++ and I do not crawl anyone here in the back in the hope that I will become an underpaid favorite of BI!

What I expect as a simple, uncaring customer, is that my purchased product works ...... and without any expertise!

 

 

[FallujahMedic -FM- 867]

Once again it seems to be necessary for a reminder.

If you have proven facts and have reproducible steps which may help address this issue, please share them.

 If, on the other hand, you are here to merely vent, please save your comments.

Any post that does not factually contribute to the solving of this issue may be removed and marked as spam. Do not make duplicate posts if nothing has changed from your previous post. 

Moderator
 

[oukej 2910]

10 hours ago, celticalliance said:

So if I understand correctly the server owners and hosts are now supposed to check their whole mod setup on server level? I had quite some issues with setting the servers up so they were running for everyone and now I have to go through that again? Just to fix this one issue?

....
A little more communication about how servers are affected by the new security would have been nice.

There's a stricter verification of server data vs. client data in order to prevent clients with malicious data from joining. So with verifySignatures every .pbo that exists (and is automatically loaded - e.g. is in the Arma3/Addons, Arma3/*DLC*/Addons folders and also in the Addons folders of all mods listed under the "-mod=" parameter) on the server also needs to exist on the client. And it needs to contain the same data as on the server and it needs to be accompanied with a correct .bisign that matches a corresponding key on the server.

If something worked in previous game updates then that does not necessarily mean it wasn't causing any harm. Clients with data mismatch could've been causing various other issues. In the past they would normally be able to join the game before getting kicked at some point later once already playing. Now they can't join at all.

 

6 hours ago, ZN Anhor said:

You kidding us Dwarden? This is how my commandline looks since 2 years .........

Good to hear! It may still be a helpful reminder for someone else 😉
 

6 hours ago, ZN Anhor said:

What I expect as a simple, uncaring customer, is that my purchased product works ...... and without any expertise! 

We've tried to improve the UX of running a modded game as much as possible during the A3 development and I do believe that with Steam Workshop integration and the Launcher we've made the players experience as smooth as switching any standard game option in any other game. 

Yet I am afraid that because the game allows you to create your own server and adjust it to your liking, because of the openness and unparalleled moddability of Arma 3 and inherent complexity of the multiplayer security in such environment, the server administration tends to be a demanding task. Anyway, here I'm already drifting slightly off-topic.

 

 

 

[Dwarden 1125]

10 hours ago, ZN Anhor said:

 

LOL

You kidding us Dwarden? This is how my commandline looks since 2 years .........

 

And BTW, what @celticalliance said above is absolutly correct! I run a server for me, my friends and some players who wanna join. (BTW, most player who own A3 did not know how they handle Mods)

I am not a programmer, I can not do C ++ and I do not crawl anyone here in the back in the hope that I will become an underpaid favorite of BI!

What I expect as a simple, uncaring customer, is that my purchased product works ...... and without any expertise!

well if you check the  https://community.bistudio.com/wiki?title=Arma_3_Startup_Parameters

(cur | prev) 14:17, 14 March 2015‎ Dwarden (talk | contribs)‎ m . . (15,652 bytes) (+297)‎ . . (→‎Server Options: Added -ServerMod= commandline parameter)

 

then you can see the feature is 4 years old (also you can guess who was the mastermind behind and pushing it to get done)

anyway the thing is, i already thanks to the logging discovered several admins who had it wrong and were using the -mod= for what was supposed to be -servermod= for sure

 

what the logging and profiling allowing us is find all those edge cases where something doesn't work or we forgot

this for you who can't or don't want to spent time with 'debugging' means we can roll out hotfix where it just works ...

 

thanks for your understanding and i'm really sorry it takes longer than some days and hinders your gameplay experience

[celticalliance 31]

19 hours ago, oukej said:

There's a stricter verification of server data vs. client data in order to prevent clients with malicious data from joining. So with verifySignatures every .pbo that exists (and is automatically loaded - e.g. is in the Arma3/Addons, Arma3/*DLC*/Addons folders and also in the Addons folders of all mods listed under the "-mod=" parameter) on the server also needs to exist on the client. And it needs to contain the same data as on the server and it needs to be accompanied with a correct .bisign that matches a corresponding key on the server.

If something worked in previous game updates then that does not necessarily mean it wasn't causing any harm. Clients with data mismatch could've been causing various other issues. In the past they would normally be able to join the game before getting kicked at some point later once already playing. Now they can't join at all.

 

Thanks for the reply. I'll have a look at that. I also know right at this point none of us have mismatching data. Also, this is the sort of explanation I was hoping to get instead of several others I got over the past week.

 

It complicates things for quite a bit though for setups like ours where we are running different modsets on our servers. We also had to set up our servers the way we have now to avoid issues for people who were joining through the Launcher and ended up with a broken mod set up because it would give them all sorts of mismatches and such. The Launcher, or Steam, would just mess up the entire set up for them. An issue that seemed to happen for a lot of people. Let's hope this is fixed by now because back in the day there were plenty of people having major issues. Anyway, just explaining why we are not using Steam Workshop for our mod set up and prefer the old school method of doing things manually.

 

UPDATE: did not fix anything, although I made sure I was running the same mods on both client and server level. Back to the drawing board 🙂

 

UPDATE 2 (18:00 GMT+1): now with all our servers set up as suggested in the Arma 3 Launcher, in the Servers tab, the public server shows all sorts of signature mismatches while we ARE running the same mods on both client and server level. This is on a server with BattlEye enabled. This is EXACTLY the issue we (and many others) had before and why we had to set up our servers the way we had them set up. On another server (without BattlEye) everything shows up fine. I start to wonder what is causing this behaviour? Anyone?

[ToxaBes 13]

I think we also have some issue with BattleEye and terrain mods.

Example: If I use

-mod=@cba_a3;@sma;@cup_core;@cup_weapons;@cup_units;@cup_vehicles;

(CBA A3, Specialist Military Arms and CUP mods without CUP Maps) all working fine

But if I try to include some map mod like CUP Maps:

-mod=@cba_a3;@sma;@cup_core;@cup_weapons;@cup_units;@cup_vehicles;@cup_maps;

nobody can join to server. I tested about 10 custom terrain mods and all they have this issue. If I disable BattleEye - all this terrains works fine.

Checked on Linux Dedicated Arma 3 server on stable Arma version and Dwarden's builds (v01,v05).

This issue is strongly related to terrain mods (maps).

[celticalliance 31]

3 hours ago, ToxaBes said:

I think we also have some issue with BattleEye and terrain mods.

Example: If I use

-mod=@cba_a3;@sma;@cup_core;@cup_weapons;@cup_units;@cup_vehicles;

(CBA A3, Specialist Military Arms and CUP mods without CUP Maps) all working fine

But if I try to include some map mod like CUP Maps:

-mod=@cba_a3;@sma;@cup_core;@cup_weapons;@cup_units;@cup_vehicles;@cup_maps;

nobody can join to server. I tested about 10 custom terrain mods and all they have this issue. If I disable BattleEye - all this terrains works fine.

Checked on Linux Dedicated Arma 3 server on stable Arma version and Dwarden's builds (v01,v05).

This issue is strongly related to terrain mods (maps).

 

I have no issues with the Terrain mods. This I think is BattlEye related. Check my previous post (update 2).

[ToxaBes 13]

36 minutes ago, celticalliance said:

 

I have no issues with the Terrain mods. This I think is BattlEye related. Check my previous post (update 2).

This mean what different mods blocked on different servers but core of issue is BattlEye related. And it seems we don't have any chance to fix it on our (server) side!

BIS devs - do you copy? psh psh

[pierremgi 4850]

Anyway, there is something to do with battleEye, kicking undue players, not filtering Warlord and Vanilla server cheaters, and now failing in checking signatures. Sorry for not being more constructive.

[ToxaBes 13]

Also, for some reasons BattlEye block standard Malden map with the mods which perfectly works with Altis and Tanoa. Malden, Carl.

This forum definitely need some facepalm emoticons...

[eggbeast 3673]

hey oukej and dwarden

 

thanks for your support guys

using the profiling branch exe we have no more crashes now and can use sig checking on our server using

DSUI, CBA, Unsung, TFAR and Prei map

 

that's a case of kozel next time I'm over there 🙂

 

[celticalliance 31]

Any word on when the hotfix is released?

[jgaz-uk 132]

News on a hot fix would be very welcome!  

[Homesick 290]

@celticalliance @jgaz-uk Your prayers have been heard. https://dev.arma3.com/post/spotrep-00086

[RH-SPECTER 3]

@BIS_HomesickStill receiving following error in console after hot-fix

 

[2019-03-20 08:41:41]: SPECTER uses modified data file

[2019-03-20 08:41:41]: BattlEye Server: Player #0 SPECTER (69.136.133.25:19058) connected

[2019-03-20 08:41:41]: Player SPECTER connecting.

[2019-03-20 08:41:41]: BattlEye Server: Connected to BE Master

[2019-03-20 08:42:42]: BEServer::finishDestroyPlayer(657327114): users.get failed

[2019-03-20 08:43:43]: NetServer::finishDestroyPlayer(703492291): DESTROY immediately after CREATE, both cancelled

[oukej 2910]

@RH-SPECTER I'm sorry to hear that. What has happened on the client? What was the disconnect message?

[RH-SPECTER 3]

None, no session lost it just has the loading map screen before the voting mission screen displayed endlessly with ocean audio