Uniting the Multiplayer community

[maddogx 13]

OK, we clearly have a disconnect at some point.  Tell me, what do you think a signature file is?  What does it represent?  How is it associated with the file that it supposedly was generated from?

For a start, I'd like to refer you to the article about digital signatures on Wikipedia. It explains the basics of digital signatures and is also valid for Armed Assault.

To clarify the issue: I'm no expert on the actual process of signing files. I don't know what algorithm BI uses to generate the signature files, nor do I know exactly how those algorithms work in general. My knowledge on the concept is based on what I have learned about assymetric encryption during my apprenticeship as a software engineer and my experience gained in ArmA from working with the signing tools provided by BI.

What I know for sure is that a signature file in ArmA must be generated for a specific PBO. It is only valid for that PBO and only as long as the data in the PBO doesn't change. Signature files contain some kind of information on the content of the PBO they are associated with - be it a hash value or whatever. As you can read in the Wiki article, the integrity of a file can be validated by an algorithm that checks the file and its signature against the corresponding public key. If the file has been modified or the signature doesn't belong to it, then the validation process will fail.

Anyone can test this using the tools provided by BIS. Specifically DSCreateKey, DSSignFile and DSCheckSignatures.

For example, try the following:

- Create a private/public key pair using DSCreateKey.

- Create a small simple addon, for example "dstest" and pack it to "dstest.pbo".

- Sign the PBO with your private key. It should generate a signature file with the file name <pboname>.<keyname>.bisign

- Check if the signature is valid using DSCheckSignatures and the public key. It should pass.

- Now make a small change to your dstest addon and pack it again to dstest.pbo (do not sign it again)

- Check it again with DSCheckSignatures. It should no longer pass.

[.kju 3244]

SgtRock

make

checkfiles[]={};

if you have verifySignatures=1;

This is very IMPORTANT!

Checking whole PBO's is NO good!

[=jps=sgtrock 4]

It sounds as if you might understand the difference between the two commands. Care to elaborate? I'd love to understand what they do.

[acadiancrusader 0]

sgtrock, i think you were referring to what i previously posted. without double checking, maybe i mis-worded my intent.

some addon makers are using the same KEY to sign multiple addons, thus creating different files. the server has one file that can potentially check multiple addons instead of equal number of files for server and client

on the surface, this is seems pratical and easier to maintain for both the addon maker and the server admin. the creator only needs one key to sign all his/her individual addons. and the server admin only needs to have one file that can check for multiple addons instead of one file for one addon.

but the problem with this is that the addon maker could develop the greatest multiplayer pvp addon in existence (thus potentially becoming a community standard) AND the same addon maker could have created a useful single player addon, but when used online, it's functions could be seen as cheating, or worse, crash the server. the problem here is that both addons could be signed with the same key. and if that happens, there would be no way to allow one addon and disallow the other. the only way to do this is for the server admin to create their own key and to sign the individual addons that they deem worthy.

and this brings us back to a dis-unified community. with so many different signitures, the client could literally have to maintain dozens upon dozens of signed files for one single addon. kinda pointless if you ask me.

so this is why we need a community standard. a sort-of governing body that dictates which addons are good and which are bad... and to police it so that the examples that i gave above don't happen.

this will be no easy task. to make it easier is the reason why we are discussing a potential third party app to do all the legwork for the client. ideally, BI would implement something like this in their game. probably best if it was in the server browser. but there could be multiple ways of making this happen.

[.kju 3244]

For the files in the checkfiles array the server and client compute

some kind of checksum. If you have large files, this takes a lot

of processing power and time. Essentially the server stops until

he completes. So this is a huge no no on large files.

Even for smaller files like config.cpp/bin its no longer needed if

you use the signature system. The checkfile system is from OFP

times. Signatures are the latest stuff.

Signatures work a bit different in general. Yet it terms of server

processing power, it takes very less and its hardly noticeable.

As we don't know the implementation, we cannot tell how the

client and server compare the files here.

The new system is way more flexible and way way faster.

[General Barron 0]

I haven't read thru the whole thread here, but I get the impression you are approaching the problem from only one angle.

That is, to make sure the player loads the game with only BIS stuff, and no extra addons outside of a specific, publicly 'approved' set. Then all servers need to only allow that set.

What if you kinda went the other way around? I'm not sure how Arma MP works and what kind of tools are available, but what if you had an external program like the old OFP Watch, which tells you what servers are available.

Each server would have its own listing of "required" and "optional/allowed" addons. The program would know what addons you have available on your computer. Therefore, the program would be able to tell you what servers you can and can not connect to, and obviously it would be nice to be able to automatically download required addons.

Anyway, instead of having to deal with mod folders, you could have all of your addons in a single location on your HDD. The program would then move only those addons required / allowed by the server you want to join, into a single mod folder, and start Arma.

[Yoma 0]

I haven't read thru the whole thread here, but I get the impression you are approaching the problem from only one angle.

That is, to make sure the player loads the game with only BIS stuff, and no extra addons outside of a specific, publicly 'approved' set. Then all servers need to only allow that set.

What if you kinda went the other way around? I'm not sure how Arma MP works and what kind of tools are available, but what if you had an external program like the old OFP Watch, which tells you what servers are available.

Each server would have its own listing of "required" and "optional/allowed" addons. The program would know what addons you have available on your computer. Therefore, the program would be able to tell you what servers you can and can not connect to, and obviously it would be nice to be able to automatically download required addons.

Anyway, instead of having to deal with mod folders, you could have all of your addons in a single location on your HDD. The program would then move only those addons required / allowed by the server you want to join, into a single mod folder, and start Arma.

Check my tool. It does what you want and more.

[maddogx 13]

....

Check my tool. It does what you want and more.

It does?

You're saying a server admin can make the list of trusted .bikeys on his server available for viewing, so your program can download them, check which installed addons on the client are compatible with them and then launch the game only with those addons?

If that is the case, we have a very good starting base.

[PuFu 4600]

....

Check my tool. It does what you want and more.

It does?

You're saying a server admin can make the list of trusted .bikeys on his server available for viewing, so your program can download them, check which installed addons on the client are compatible with them and then launch the game only with those addons?

If that is the case, we have a very good starting base.

Yes, it does

[maddogx 13]

....

Check my tool. It does what you want and more.

It does?

...

Yes, it does

Well, in that case I'm gonna have to check that out. I had a brief look at the addon synchronisation functionality a while ago, but thought it was limited to downloading addons.

Does it come with DSCheckSignatures or how does it validate the addons?

[Yoma 0]

....

Check my tool. It does what you want and more.

It does?

You're saying a server admin can make the list of trusted .bikeys on his server available for viewing, so your program can download them, check which installed addons on the client are compatible with them and then launch the game only with those addons?

If that is the case, we have a very good starting base.

the part of checking which addons are compatible with them it cannot do.

The question is: does it need to?

Another question is: is that technically possible?

Without creating any external services dependencies?

Another thing i can say is that the problems you mention are near unexistend for players that connect with my tool on a server that uses my tool.

All they need to do is use only the addons the server proposes

[maddogx 13]

....

Check my tool. It does what you want and more.

It does?

You're saying a server admin can make the list of trusted .bikeys on his server available for viewing, so your program can download them, check which installed addons on the client are compatible with them and then launch the game only with those addons?

If that is the case, we have a very good starting base.

the part of checking which addons are compatible with them it cannot do.

The question is: does it need to?

Another question is: is that technically possible?

Sure it is possible. Just bundle the DSCheckSignatures tool by BIS with it.

Using that, you can very easily check what addons the user can use. Like General Barron said, it would be cool if servers could create two seperate lists of addons: one "must have" list of addons, and one "can have" list of bikeys and addon hashes with valid signatures (addons themselves are not needed, since they are only optional).

The must have addons could work just like Yoma tools does now. For the can have list, the server admin would need to host the key files installed on his server. When the tool checks the server, the key files would be downloaded and checked against the addons that the user has installed using DSCheckSignatures.

Better yet: if the user has a PBO file that is allowed on the server, but he doesn't have the right signature file for it, the tool should be able to download only the signature from the server, so he can use it.

Here a more detailed explanation:

Must have list:

- Works just like the Yoma sync tool does now. No changes necessary.

Can have list:

- All public keys (.bikey files) installed on the server are available for download.

- A list of addons (addon name + MD5 hash) including a valid signature file for each addon is available.

- All addons on the client machine are checked against the BIKeys (DSCheckSignatures). Those addons that pass verification are allowed and on the "green list". The rest of the addons is on the "yellow list" for now.

- If there are addons on the yellow list, the tool downloads the list of "can have" addons from the server.

- If the user has a "can have" addon in his yellow list, it probably means that he does not have the correct signature for that addon. If the corresponding signature is on the server, it will be downloaded, putting the addon on the green list.

- All the rest of the yellow-listed addons are put on the red list.

The game can then be launched using all required addons and the "green list" addons.

I hope this was understandible. I'm not very good at explaining stuff.

Quote[/b] ]Another thing i can say is that the problems you mention are near unexistend for players that connect with my tool on a server that uses my tool.

All they need to do is use only the addons the server proposes

That is beside the point. Players should not be forced to, for example, use a specific sound mod. Sound mods are optional, so they should be allowed - not forced.

Server admins should not only have the possibility of making players use specific addons, they should also be able to allow stuff that is not "must-have" for the server.

[=jps=sgtrock 4]

sgtrock, i think you were referring to what i previously posted. without double checking, maybe i mis-worded my intent.

some addon makers are using the same KEY to sign multiple addons, thus creating different files. the server has one file that can potentially check multiple addons instead of equal number of files for server and client

on the surface, this is seems pratical and easier to maintain for both the addon maker and the server admin. the creator only needs one key to sign all his/her individual addons. and the server admin only needs to have one file that can check for multiple addons instead of one file for one addon.

but the problem with this is that the addon maker could develop the greatest multiplayer pvp addon in existence (thus potentially becoming a community standard) AND the same addon maker could have created a useful single player addon, but when used online, it's functions could be seen as cheating, or worse, crash the server. the problem here is that both addons could be signed with the same key. and if that happens, there would be no way to allow one addon and disallow the other. the only way to do this is for the server admin to create their own key and to sign the individual addons that they deem worthy.

emphasis added

This, in a nutshell, is what is wrong with relying solely upon signatures for determining whether to allow or disallow a file to be use. If you can only deny the use of a mod based upon its signature by disavowing all files signed by the same key, you haven't really addressed the problem at all, have you?

That is why I advocated downloading a script to run a hash check for approved mods. It provides a means by which server admins can manage their own environments without having to rely upon a single point of failure external source for authorization of players. It has the additional benefit of not requiring any pre-installed code by the players, making it much easier for them to find and play on our servers. And let's face it: We have trouble enough building a player community as it is.

Quote[/b] ]and this brings us back to a dis-unified community. with so many different signitures, the client could literally have to maintain dozens upon dozens of signed files for one single addon. kinda pointless if you ask me.

so this is why we need a community standard. a sort-of governing body that dictates which addons are good and which are bad... and to police it so that the examples that i gave above don't happen.

this will be no easy task. to make it easier is the reason why we are discussing a potential third party app to do all the legwork for the client. ideally, BI would implement something like this in their game. probably best if it was in the server browser. but there could be multiple ways of making this happen.

I see the problem, I just don't buy the solution. No centralized approach is going to be workable in the long run. IMO, the discussion surrounding Yoma's app that has been going on since your post looks like a more promising direction. Unfortunately, I still think that requiring the players to DL and install a piece of client software is a fundamentally flawed idea for several reasons:

<ul>It makes the mistake of assuming that griefers won't hack the client to let them connect to our servers anyway. That's why I think that only code deployed by the server owner has any hope at all of being trusted. It's still not a perfect solution because the code still has to be run on the client. However, at least it raises the bar by forcing a griefer to figure out how to manipulate the code while it's resident in memory /and/ he's already gotten ArmA running.

<ul>Unless and until you have a large fraction of players routinely installing Yoma's app, you're stuck with a tough situation. A server admin will only be able deploy Yoma's code on his server after he's pretty sure he can meet the following conditions

<ul>Already built up a community

<ul>Spent a great deal of time patiently explaining what he's doing ahead of time so people have a chance to get ready

<ul>Dealt with the inevitable storm of protest by some players who hate having to do anything extra to play. (I know, a lot of them would be whiners who we'd be better off seeing gone, anway. However, a lot of them would still be valuable members of a community.)

<ul>He's willing to take the inevitable hit in the loss of players after he gets Yoma's code up. (People who won't install, people never got the word and don't bother to dig too far, etc.)

<ul>It fails to recognize that people may want to use other external browsers, not just Yoma's. I'm thinking of apps like kali, AllSeeingEye, xfire, Steam, etc.

I think Yoma's on to something with the basic approach. The one issue that we need to deal with is; can his client app (or one very like it) be adapted to deploy from a gameserver as part of each player connection? If it can, it solves all of the shortcomings that I've pointed out. If not, the approach won't fly unless the vast majority of server admins insist upon it. Right now, I don't see that things are bad enough for them to take that drastic a step on all servers.

[maddogx 13]

Quote[/b] ]It has the additional benefit of not requiring any pre-installed code by the players, making it much easier for them to find and play on our servers (....) I still think that requiring the players to DL and install a piece of client software is a fundamentally flawed idea for several reasons

The player would not necessarily need to install anything. Players who are not using any 3rd party addons would have no problems connecting to a server which only maintains a list of "can-have" addons. Also, players who are only using compatible addons would be able to join without problems. Only players with non-MP-compatible addons would be blocked by the ArmA signature checking system, which is the whole point.

Quote[/b] ]It makes the mistake of assuming that griefers won't hack the client to let them connect to our servers anyway.

Providing they can also bypass the key and signature checking system, on which the whole idea is built.

[=jps=sgtrock 4]

Then I misunderstood what Yoma was proposing. It sure sounded to me as if the only way to make his approach work was with his client side code pre-installed. That's not the case?

Yoma, any thoughts?

[Yoma 0]

Well i'm currently toying with the DSChecksignatures command.

What i'm starting off with for now is:

-Making something that can run over the entire arma folder + subfolders.

-Have it store this info in a sqlite

-have you select a keys folder

-Then run DSChecksignatures over all addon folders and store the output in the same db.

This way people could at least check what addons are signed and what are not against a given server key directory.

The good thing about this (and my first simple goal) is that this procedure can be used to clean up the addonsfolder of users that put addons in their arma/addons folder instead of subfolders.

The bad thing is that on my system it cannot load some of the hashes, allthough most probably this won't happen with properly signed mods??

Allso bad is that if i whish to extend my addonsync with further functionallity i tend to think about a full rewrite which would take a lot of time. I may not have this time as i may very soon need to find a new real life job.

If i get some time this weekend i may however push out a smallish tool to clear out messed up default addons folders.

I'm thinking about putting this in my addonsync tool.

Also there are some catches to downloading keys and checking modfolders using those keys:

Say we have server A which uses some keys, then we have the default keys. I'd rather not write ANYTHING to the default keys directory (to avoid exploits where people could get default bis key overwritten etc) The catch here is that DSChecksignatures only takes 1 keys directory. That means every directory should at least take 2 scans and that i'd have to look on the second run if the addons have good keys on the second run. Another catch is that some files rest unsigned (config files e.g.) i think.

One other worry is: would BIS mind if i included DSCheckSignatures in my tool?

[maddogx 13]

Sounds good, Yoma.

About the DSCheckSignatures tool, I'm not sure about the rights to provide it alongside different software.

Here is a quote from the license of the BI tools:

Quote[/b] ]You are entitled to use the Program for your own use, but you are not entitled to: [...] (ii) Publish and/or distribute the computer Program or any of its parts

That seems to imply that it is not allowed. I'll write a PM to Maruk about it. He should be able to settle the matter.

[Yoma 0]

Sounds good, Yoma.

About the DSCheckSignatures tool, I'm not sure about the rights to provide it alongside different software.

Here is a quote from the license of the BI tools:

Quote[/b] ]You are entitled to use the Program for your own use, but you are not entitled to: [...] (ii) Publish and/or distribute the computer Program or any of its parts

That seems to imply that it is not allowed. I'll write a PM to Maruk about it. He should be able to settle the matter.

Well i could allways not bundle it and have the end user install it. But again that makes stuff more complicated for the end user... I most certainly do not intend to get sued over a program that i distribute freely to enhance the game

[maddogx 13]

Sounds good, Yoma.

About the DSCheckSignatures tool, I'm not sure about the rights to provide it alongside different software.

Here is a quote from the license of the BI tools:

Quote[/b] ]You are entitled to use the Program for your own use, but you are not entitled to: [...] (ii) Publish and/or distribute the computer Program or any of its parts

That seems to imply that it is not allowed. I'll write a PM to Maruk about it. He should be able to settle the matter.

Well i could allways not bundle it and have the end user install it. But again that makes stuff more complicated for the end user... I most certainly do not intend to get sued over a program that i distribute freely to enhance the game  

Indeed. I wrote to Maruk about it and hope he will be able to clear the issue up. I also asked whether he would grant special permission to include the DSCheckSignatures in the tool. With any luck he'll answer sometime soon.

[Yoma 0]

1) If you can only deny the use of a mod based upon its signature by disavowing all files signed by the same key, you haven't really addressed the problem at all, have you?  

2) That is why I advocated downloading a script to run a hash check for approved mods.  It provides a means by which server admins can manage their own environments without having to rely upon a single point of failure external source for authorization of players.  It has the additional benefit of not requiring any pre-installed code by the players, making it much easier for them to find and play on our servers.  And let's face it:  We have trouble enough building a player community as it is.  

1)Again i think there is absolutely nothing wrong with server admins signing/resigning the mods they allow

2)Euhm isn't that really what signing/sign checking of mods actually IS internally. If someone has a really really big server i could of course create some kind of repositry server where people could download files on a hash basis. The problem with that is that it would be an utter bitch to maintain. And we all know where unmaintainebility leads (think thrashcan).

3)Of course BIS could implement stuff clientside, but i'd rather they implement a bugfree Arma2 then a bugridden Arma2 that can download mods. Secondly a lot of people are nagging about ingame lag, on jip, how would that be if the server also started distribuating mods... It would be A LOT worse.

They could ofcourse implement some kind of redirect, but maybe that would be a bit more work on their side?

4)It would be nice to not need a client side application, but then again, for me it's quite simple: I like launching arma better from my tool then launching arma straight away.

It takes away less resources, i can sync my mods and if i want to play with a given mod i just need to check a box. And i can look who's playing on a server without needing to connect ingame (causing less needless JIP lag). And i can have it autolaunch arma if the given server is full.

5)those players that don't want to connect with my app DO NOT NEED TO! It's perfectly possible to download the addons/install them manually. The problem with that is that it imposes MORE WORK on them. This is exactly why i made my tool. Again i advice all people that haven't tried it yet to just give it a go, there are multiple default servers in it allready and those work very nice. I'm considering making the "default servers" part a bit more dynamic, so people can submit their server to me and the app has some kind of "default server browser" where you can pick a server and add it to the ones you use. This would make "adding a server" a lot more userfriendly.

7) I'm still waiting if i can include the DSCheckSignature.exe. This way i could offer people the opportunity to clean up their default addons folder.

Of course all this isn't IDEAL, but that's a problem our entire world has, no?

[maruk 80]

1)Again i think there is absolutely nothing wrong with server admins signing/resigning the mods they allow

Indeed, there is nothing wrong with it, see see advanced signature usage scenario "server keys" description on the biki

The entire signature concept is based on one key thing: you trust the authority that signed the addons that these addons are not used for cheating etc. and that their private key is really kept private and unauthorized people can't use it to sign anything. See also another part of the biki article.

[BLSmith2112 0]

1)Again i think there is absolutely nothing wrong with server admins signing/resigning the mods they allow

Indeed, there is nothing wrong with it, see see advanced signature usage scenario "server keys" description on the biki

The entire signature concept is based on one key thing: you trust the authority that signed the addons that these addons are not used for cheating etc. and that their private key is really kept private and unauthorized people can't use it to sign anything. See also another part of the biki article.

Amen to that brother - there is absolutely nothing wrong with signatures. However...

I do understand where some of these guys are coming from. Since there's no way to use hyper links and/or autodownload to a servers FTP client in real time through the game, it's not only annoying having to do all of the research and having to invest a lot of time trying to get everything in order just to play (especially if your not used to it), buts it potentially makes people just give up all together and go to another game. I mean I understand that if people do a little digging, they can get into a game but if this is the way addons should be carried out from this point forward, I think this should be in some way incorporated into the game itself so people can just enjoy the game not dread over being forced to do something. Thats just my opinion...

[=jps=sgtrock 4]

1)Again i think there is absolutely nothing wrong with server admins signing/resigning the mods they allow

Indeed, there is nothing wrong with it, see see advanced signature usage scenario "server keys" description on the biki

The entire signature concept is based on one key thing: you trust the authority that signed the addons that these addons are not used for cheating etc. and that their private key is really kept private and unauthorized people can't use it to sign anything. See also another part of the biki article.

Except that you're creating more work for players to connect. Quoting from the article that you link to:

Quote[/b] ]Advanced usage scenario: Server keys

One addon can be accompanied by multiple signatures, each signed with a different key. One application for this can be server admin can create his own key for his server, and can use it to sign all addons on the server he considers safe.

Note: For this usage style, signature files for the addons need to be distributed to all users connecting to that server.

Let me restate the pertinent part:

Quote[/b] ]Note: For this usage style, signature files for the addons need to be distributed to all users connecting to that server.

So how am I supposed to get these new signatures deployed to people who have just found my server? How do I build up a player community when my basic security requirements force them to get all the appropriate bits in place first?

Worse, others in this thread pointed out that modders were misusing the concept of signature creation anyhow. So how do you guarantee that both ends of the player-server connection are using a current, valid key?

Let me tell you what a typical new player will do the first time he runs into either of these scenarios on a server: He'll go somewhere else. He won't bother to find out why the connection failed, he'll just leave and probably never re-try connecting.

If enough server admins begin creating and using their own keys so that the player has a hard time finding a server to connect to, then he'll quit playing the game. From BI's point of view, what's worse is that he'll tell all his buddies not to buy the game because it's too hard to find servers that you can connect to. End result, empty servers and no money for BI.

Earlier in this thread, I proposed a model by which when a client initiates connection, the first step would be for the server to download code to the client that verify that the client was running only approved mods. I'm slowly coming to the conclusion that's not going to work in the long run. Any such scenario would probably take so long that clients would give up and go looking for another game. End result is therefore the same; empty servers and no money for BI. No wonder most server admins turn off signature checking.

At this point, the only way forward that I can see that makes any sense at all is something similar to id software's sv_pure variable. If there is any extra or modified code on the client, then the server needs to prevent the connection /and/ the player needs to be informed of that fact. I imagine that would bring on howls of protest from people running their favorite sound mods, but I honestly can't see any other way to get to an easy to use security model that can be trusted.

The only other proposal that I've seen that I think may work is something along the lines of Yoma's. However, I don't think it can work unless it's deployed by BI to all as an official, required patch.

[Yoma 0]

All i can say is this:

If we promote my tool or a decent alternative throughout the community, we can have (and allready are having) a nice addon-rich, cheatfree, changing on a allmost daily basis, multiplay experience.

Sure there will be people that don't get it. But a lot of these people we don't want to play with in the first place. Because they are exactly the BF2 "run,gun,MY CHOPPER MY A10"-crap filled nogoods spoiling decent games anyway.

I look at it just the way i looked at OFP: I'd rather have quality players then quantity of players.

[=jps=sgtrock 4]

All i can say is this:

I look at it just the way i looked at OFP: I'd rather have quality players then quantity of players.

I'm with you on the quality over quantity. However, as Stalin sagely observed, "Quantity has a quality all its own."

In this context, that means that the more good players we can draw in, the better the overall gaming experience will be for all of us. IOW, we need to be inclusive, not exclusive if we want our gaming community to ever reach its full potential. Any security model which doesn't take this fact into account is simply an impediment to play and will either be disabled, ignored, or continue to drastically limit ArmA's popularity. None of these options are desirable.

Look, I like the basic concept of what you suggested. My issue is that I simply cannot afford to implement it as a community run option when my server sits empty so much of the time as it is. In order for it to be truly effective without damaging my chances of filling my server, it has to be on all servers and accepted by all players. How are we going to do that without a BI required patch?