Arma Addon Sync 2009

[Yoma 0]

Updated version BETA 1.0.3 is released on dev-heaven

http://dev-heaven.net/projects/show/yoma-addonsync2009

[mattxr 9]

Sounds Really Good.

[.kju 3241]

Let's see minor god Yoma  

A few thoughts on this one.

Let's start at the end. Right now all downloads are FTP/HTTP based.

The way to go for a thorough solution should be a p2p or torrent approach.

While p2p can be completely serverless, the torrent (still?) needs a master server.

That would have to be located, setup and managed by some admins. This is a big drawback.

So my dream is that all addon sites can participate in a file

grid easily and people can download from various servers.

A server admin only needs to configure which addons are

needed or even allowed on his server. Both is optional.

Instead of a your own signature, your tool could use the BI

sign tool to create the unique identifier by default.

So your tool signs all addons on a client and server with your public key.

Modified addons should result in a different signature.

If thats not possible, you may want to consider different levels

of checking pbos.

First check could be only by naming.

Next by file size.

Lastly by a checksum.

GUI and interface separated from the logic via clean interfaces

to allow templates with different layout and "work flow" or

more simplistic version.

Open source project to let people participate and help improve

the project.

Will post more thoughts in the coming days.

We all want to dream.  

[Yoma 0]

Let's see minor god Yoma  

A few thoughts on this one.

Let's start at the end. Right now all downloads are FTP/HTTP based.

The way to go for a thorough solution should be a p2p or torrent approach.

While p2p can be completely serverless, the torrent (still?) needs a master server.

That would have to be located, setup and managed by some admins. This is a big drawback.

So my dream is that all addon sites can participate in a file

grid easily and people can download from various servers.

A server admin only needs to configure which addons are

needed or even allowed on his server. Both is optional.

Instead of a your own signature, your tool could use the BI

sign tool to create the unique identifier by default.

So your tool signs all addons on a client and server with your public key.

Modified addons should result in a different signature.

If thats not possible, you may want to consider different levels

of checking pbos.

First check could be only by naming.

Next by file size.

Lastly by a checksum.

GUI and interface separated from the logic via clean interfaces

to allow templates with different layout and "work flow" or

more simplistic version.

Open source project to let people participate and help improve

the project.

Will post more thoughts in the coming days.

We all want to dream.  

The bis sign tools i'd like to avoid.

If some day i want to use this stuff for a different game i don't want to be stuck with BIS tools. MD5 does a good job and is even used in forensics computer crime investigation.

Also they don't even allow me to package them with the installer so that would mean configuration hell.

I like the idea of a torrent approach, but then again, some American providers are currently actively blocking torrents. (too much illegal stuff on torrents). Do a search on google for "torrent blocking" and you'll see what i mean.

On the file checking by name, size etc that's generally a bad idea. If you use the cache the application goes fast enough atm. Also i'm pondering on using the cache by default in the new version (after fixing some bugs of course). This just means that initially the md5's have to be calculated but in time this takes up less and less time. I don't consider the MD5 speed a real pitfall for the application. It has proven it's use. And is very reusable for other applications.

However, the "file grid" thing may be realised in a somewhat different manner, i have some ideas on this.

I don't think i have the patience/time to run it as a true open source project atm.

[Sneaker-78- 0]

Quote[/b] ]Get some very sexy girl for the audio messages.

so we now hear you're voice

But it sounds good , i am happy with you`re current version it makes it lot easier for my friends to joint my server

[Yoma 0]

I have a very important question for the developers:

What's the limit on characters arma.exe will take from commandline, if any.

Other people can answer this too, but i don't want any guesses, i want exact numbers.

*Edit*

Ok i just found out that even if there is no limit to the parameters Arma takes, .net limits me to 2003 characters.

[Yoma 0]

Quote[/b] ]Get some very sexy girl for the audio messages.

so we now hear you're voice  

But it sounds good , i am happy with you`re current version it makes it lot easier for my friends to joint my server  

The current version works, but a lot could be better.

Now you hear my voice indeed, with a bit of an Aussi accent added :-)

[Yoma 0]

**Minor progress report**

-my intention is for all "timeconsuming" tasks to be delegated to worker classes (eg. downloading,extracting,calculating md5)

This way i'll be able to provide a much better gui.

-today i experimented with having a download progress bar inside the datagrid showing the addons. This seems to work nicely so far! (Status bar is blue while downloading and turns green after download is finished)

[.kju 3241]

good news about separate thread for the other tasks!

In terms of checking of update and downloads.

It would be very useful to have a tree view over all packages.

Right now you have to check every package individually.

While it would be very nice to have everyone in one view.

So similar to windows explorer view for example.

[Yoma 0]

Actually if you look at explorer, you'll find you can only "see" the content of one folder at a time.

I'm thinking more about a "combined" view.

something like one datagrid that shows all files in the packages combined with maybe a tree view you can use to filter stuff.

This way you could show everything at once and still be able to filter out a single folder/folder with its subfolders.

This might prove hard to implement, but i won't die trying.

[gonk 0]

Now if you want a REAL challenge...

Maybe look at sharing the addons among clients aswell as the server.  So you can download the addons and spread the load across a few machines/net connections.... a tad like torrents... well you got me..a lot like torrents..

good luck...

[Yoma 0]

Now if you want a REAL challenge...

Maybe look at sharing the addons among clients aswell as the server.  So you can download the addons and spread the load across a few machines/net connections.... a tad like torrents... well you got me..a lot like torrents..

good luck...

See the comment about torrents in this very thread. As some providers block them i won't be using them as a main download system.

However if i get the time/find the right tools i might just implement something along the lines. (no promises at all)

Also i don't know if there are people out there interested in putting up torrent-tracker servers and if it's even possible at all to avoid people linking up illegal files on such servers. The people that run the trackers will be very interested in this as i don't think they'd like to have cops take away their servers...

I have a lot more issues to think about without torrents atm.

[Yoma 0]

**Minor progress report**

-The gridview looks quite nice when downloading (it gives exact status in process + extract status)

-I'm experimenting with treeview component to have on top of gridview (for filtering purposes), which is working quite well (i'm just testing filling up the treeview, but it works)

-Next things to tackle are some database stuff.

[.kju 3241]

sounds good. what about a screen for the treeview ?

[Yoma 0]

sounds good. what about a screen for the treeview ?

Euhm not much to show there yet, it's just a standard tree component...

Of course you'll be one of the first to receive preliminary beta versions ;-)

Also i intend not to "break" compatibility with the older version.

(It won't support the modfolders.txt and xml's but when you deploy it on a server you'll be able to deploy both with the old version and the new version on the same repositry)

The new version will have the possibility to spread your addons accross different servers as well. (in the db you'll have a url for each addon file)

[Sneaker-78- 0]

maby an idea to let the client download the missions also and ofcourse in the right map

[.kju 3241]

download missions is possible already now.

however to check if the required islands is available on the client

is doable to a large degree. might be a good idea to add this check

[Yoma 0]

I won't dabble into truely checking addon dependencies as this would require massive maintenance for which we do not have any reliable organisation nor infrastructure. (that's the downside of having a truely open game)

Also there is no way of doing this without extracting addons and even then it's a total bitch.

Of course for missions you could use the name of the file to "obtain" the name of the island it needs to run.

However this does not include a version number of the island at all.

For now i focus on the new tool having about the same functionality as the previous version, with some changes and improvements.

[.kju 3241]

yoma you misunderstood him (or me )

its only about the island.

you can read from the mission the required islands from the mission naming:

MISSIONNAME.ISLAND

ie testmission.intro

intro = Rahmadi

So you would only need one list for this mapping.

This could be delivered by your tool and possible to update

by user.

We are NOT talking about addons.

[Yoma 0]

Yes, but even that list is variable.

People make new islands every day.

And as i said, mission x may require island y v1.0 where mission z may require island yv1.2

As the name of the island stays the same there is no way of telling which version of island the mission needs.

[.kju 3241]

well any mission is always compatible to any version of the islands

in a technical and practical sense.

to avoid that the config would need a different cfgPatches class

or people had to rename their island (in pbo name and config class).

so again in a practical sense you only have new islands popping

up like your problem with new yomatools server  

[Yoma 0]

**Minor progress report**

-started work on "importing" data from remote server db

It still has lots of issues, but the basics seems to work fine.

-next in line will be downloading this file, verify the md5 of the file and import it (the nice thing about this is that the tool will be able to tell if you even need to think about updating your addons)

-I'm still a long way from making a first workable beta version, but the good news is that i now have "proof of concept".

I think a lot of admins will like the fact that you'll be able to alter for example game ip without the need to warn all players that it has changed. Same goes for TS settings.

-I'm still in doubt about importing addon passwords and ts passwords, i think i'll make that optional when creating the db to put online.

One problem i have has to do with "security": importing the server settings is a good thing (eases configuration a lot), but i wouldn't want people to put db's online ruining the settings of other servers. If anyone has a good idea on this please let me know.

At the moment i think of it like this:

the url one uses to download the server settings is most likely the only thing to be unique.

What i could implement is something like: if the url stored in the remote db differs from the url you entered at server config level, do not import the settings.

This way it should be impossible to ruin settings of an other server unless you actually spoof the remote server url.

[Yoma 0]

**Minor progress report**

-I've been looking into providing a "game server browser" in my tool as well (compare it too the ingame server browser).

It works pretty well atm, but uses an external tool to query the servers. The great thing about this external tool is that it's actively under development and could provide an easier interface when switching to Arma2.

My goal here is to provide some kind of interface so you can add servers to a "default serverlist", that default server list will be the same as your "addon server serverlist".

This way users will be able to easily add servers to their default server list, and use various launch settings/download the addons needed for the server.

It even may be possible to capture some "data" from the servername string so we could further automate stuff.

I dream about something like

-select a server from the gamespy serverlist

-have the url to the file describing the "modfolders.txt" in the servername itself

-parse the string and automagically "know" where to get the correct addons.

[Yoma 0]

**Minor progress report**

-I think i'll leave the "repositry" serverside in xml after all.

(it has the benefit of being easy to "generate" with external tools / edit etc)

-I'll keep the local database idea as it eases up a lot of stuff.

-I also think about reading the xml directly from remote source as this could greatly ease up integration with websites (untill now the idea would have been to download the entire xml in a compressed format, extract it and then use it, but if the amount of files/data in the xml is not overly big then actually a direct read could ease up a lot of stuff for me)

-Bought a halfdecent FTP component this means that given a lot of development the tool could actually "sync" the files to an ftp server/ delete files from an ftp server that are no longer needed. This could make deployment/maintenance a breeze for admins.

[Yoma 0]

**minor progress report**

-working on a lot of gui stuff.

-As more and more full mods are being released we are starting to see more and more "server mod x events". I've been thinking about this and maybe it would be nice to have some kind of "event" support inside the tool, where users could be notified of upcoming events and serveradmins could prepare a special "event" sync on the server, with server password optionally included as well as TS settings. If you have any good thoughts on this, don't hesitate to tell them.