SOW Multiplayer Saving and Compression Functions

[Rune 0]

We at the Sinews of War Mod have decided to release some tool-packages based on previous work on the main SoW code. This is one such tool.

What is this then?

This is a collection of function files, that mission makers can include in their multiplayer missions to allow the mission to save and load data during a mission.

It is based on work done several years ago by Crashdome, the leader of the Sinews of War Mod. This mod has been working hard to bring about the tools that are needed to create true campaigns in multiplayer.

This is in essence a re-release of some code that was in the last release from the Sinews of War Mod. This time it has been separated from the main SoW code to be offered as a stand alone tool for all mission makers to use if they want. It was always possible for everyone to use this, at least in theory. But we have recently been made aware that the big complicated package of our previous releases has been keeping some people from using our work.

So if you are making multiplayer missions and want to have persistent data in your missions before Armed Assault is out- this is the tool for you.

What will it do exactly?

It will help mission makers save, load, compress and decompress data while saving in multiplayer. This will be an invaluable tool for anyone wanting to save a lot of data(say for CTI) and for smaller amounts of data it will offer rock solid saving and loading - any value saved is guaranteed to load exactly as it was saved, unlike all other implementations I've seen. It will also allow you to save some variable types such as boolean or integers of 3 digits or less with much better efficency(using less space and less time for same data).

This release is part of my personal work on the core saving functionality for SoW v2.1 it should make life a lot easier for those who wish to save and load data reliably in multiplayer. The 'standard' way of doing it has been found to be highly flaky...not surprising really as it is not an official feature of OFP. Which is also why I cannot guarantee that this will work, or be needed, when Armed Assault is out.

If you don't wan't anything fancy though you can simply not use the features beyond the fairly straight forward save and load functions included.

What is in the download?

The download is a .zip file. In it is a readme file to describe what the package and the individual functions do, how they do it and how they are used. The .zip also includes the function files themselves and a demo mission. The demo mission was really a test mission to test all aspects of the functions, it deliberately uses the functions in the wrong way in places to provoke error messages. This is because I wanted the error messages to be a reliable indicator as to what was wrong in the code using these functions, the error messages are to be considered features of this release, helping you to find bugs in your code...so don't think they are broken because there are error messages in the demo - if you look closely at what is going on when they pop up you may be able to learn why they happen from the demo. Basically the demo should allow you to see how reliable these functions really are so you can turn your attention to making missions rather than worrying about whether or not the saving code has a bug in it...

How do I get it?

Go to the Sinews of War download page and download the small zip file under the name "MP Saving and Compression Functions".

Future releases

We have a range of useful things we would like to release in this way, so if you know of anything in particular from SoW you would like to see as a stand alone tool, feel free to ask, then we might do that next - no promises though as some things will simply take too long to 'dig out'.

EDIT(copied up from later posts):

What is Sinews of War about?

It is basically many things at the same time, because we are different people with tons of ideas each of us.

The whole thing started out being just Crashdome wanting to make his ultimate mission, a dynamic campaign inspired by games like Jagged Alliance and Falcon 4. For a long time we were all mostly working on continuing and improving the work Crashdome had already done at that time. He had been making the tools that he would need for his campaign.

Recently though SoW has been evolving into sort of an umbrella organization or network for lone developers working with dynamic campaigning, realtime strategy or roleplaying concepts. The SoW site is being changed into a professional developer's invironment, that is it will have project managing, version tracking and such things. It is going to be much much more streamlined than trying to work together through just a forum, forums are great for coming up with the ideas and such, but they utterly suck at coordinating efforts for bigger projects...

Each developer at SoW has their own projects, and the others will give their assistance with whatever they can and feel like. We are not a rigid timetable-oriented network, developers are responsible only to themselves for what they do - unless they decide to join someone else's project of course. We are, if you will, a force multiplier - we make the individual developer 'stronger' in the struggle to reach his own goals.

This is going to be the way we work in the future also only we are going to optimize for it with the site-change. We are looking to be a base for 'lone-wolf' developers, anyone with a good idea and the will to develop it. They will be able to come to SoW to make it a reality, and the end result is going to be much better than if that person had been simply working on their own all along.

Sinews of War - Previous Work

If you are new to Sinews of War missions the place to start is at the Sinews of War download page I would suggest first downloading Crashdome's excellent "SOW v2.0 Demo Mission User Manual" to read before you play a mission. Then download and play "SOW v2.0 Stand Alone Demo (PBO Only)" with a friend or two. This will be a good way to learn how things work. And finally, once you know how to work the menus you can proceed to play the so far unfinished campaign by Nubbin77(which is really cool I can assure you) named "The New Resistance v0.98 (Source)" in the download page.

[.kju 3245]

big thanks Rune and rest of SoW team

hopefully tacrod can get it working in his cce mission again with this version

just one suggestion:

rework that post - someone who doesnt know SoW wont really understand this or at least lets say the presentation could be better

what about some good structured summary with the use of bold, colors etc

[Rune 0]

Thanks Q, will rework it - guess I was too tired  

[Drongo69 117]

Excellent. I'll take a look at this when I get a chance.

[stegman 3]

...allow the mission to save and load data during a mission.

Very interesting.

I guess that if you exit the mission, the data is lost?

Also, could it be used to save DAC spawned units?

(I'm currently very interested in DAC and am using it some missions for myself and a few friends who like the complete unpredictability of if )

[Shashman 0]

just one suggestion:

rework that post - someone who doesnt know SoW wont really understand this or at least lets say the presentation could be better

This man speaks the truth.

I have to admit I was a bit lost when first reading it. I visited the SoW website too but there's no brief description of what you guys do...

[Rune 0]

...allow the mission to save and load data during a mission.

Very interesting.

I guess that if you exit the mission, the data is lost?

No, it will be there forever and loadable from any other mission played after that, even on different islands. Only if you delete the objects.sav file the data is kept in or if you delete or overwrite the data by use of a scripting command in OFP will it ever go away .

This is why it is at the core of Sinews of War, because it is how we store the status of a multi-mission campaign for later.

Quote[/b] ]

Also, could it be used to save DAC spawned units?

(I'm currently very interested in DAC and am using it some missions for myself and a few friends who like the complete unpredictability of if )

I am not aware of the details of how DAC works, but this can be used to store any value of a variable that is or can be represented by boolean values, integers or real numbers.

So in other words it depends on what you would like to save, if it can be transformed into numbers without being overly complicated then yes it should be quite easy to save say a total number of enemy MG gunners so that no more are created once you killed all of them or something like that.

And Shashman, yeah I know we just got a new site and it is not quite set up yet. And besides we are going to be changing the way we do things in the future exactly because people have tended not to understand what is it we do, so hopefully we will have a good explanation up soon

What is Sinews of War about?

It is basically many things at the same time, because we are different people with tons of ideas each of us.

The whole thing started out being just Crashdome wanting to make his ultimate mission, a dynamic campaign inspired by games like Jagged Alliance and Falcon 4. For a long time we were all mostly working on continuing and improving the work Crashdome had already done at that time. He had been making the tools that he would need for his campaign.

Recently though SoW has been evolving into sort of an umbrella organization or network for lone developers working with dynamic campaigning, realtime strategy or roleplaying concepts. The SoW site is being changed into a professional developer's invironment, that is it will have project managing, version tracking and such things. It is going to be much much more streamlined than trying to work together through just a forum, forums are great for coming up with the ideas and such, but they utterly suck at coordinating efforts for bigger projects...

Each developer at SoW has their own projects, and the others will give their assistance with whatever they can and feel like. We are not a rigid timetable-oriented network, developers are responsible only to themselves for what they do - unless they decide to join someone else's project of course. We are, if you will, a force multiplier - we make the individual developer 'stronger' in the struggle to reach his own goals.

This is going to be the way we work in the future also only we are going to optimize for it with the site-change. We are looking to be a base for 'lone-wolf' developers, anyone with a good idea and the will to develop it. They will be able to come to SoW to make it a reality, and the end result is going to be much better than if that person had been simply working on their own all along.

[Drongo69 117]

Quote[/b] ]Also, could it be used to save DAC spawned units?

Given the complexity of DAC (units, waypoints, reduced units, etc), and the way that DAC must initialize at that start of every mission, I doubt it is practical. There is just so much information to collect. However, it should be possible to detect which DAC zones are active (eg. have troops in them) and save a simple boolean variable which could be used to activate and deactivate zones upon loading a mission.

This is all theroretical, I haven't had the chance to implement SoW in CCE yet. I really hope I'm wrong, as CoIn2 is heavily based on DAC and I would love to be able to save all that information.

[stegman 3]

Quote[/b] ]Also, could it be used to save DAC spawned units?

Given the complexity of DAC...I doubt it is practical. There is just so much information to collect. ...

This is all theroretical, I haven't had the chance to implement SoW in CCE yet. I really hope I'm wrong, as CoIn2 is heavily based on DAC and I would love to be able to save all that information.

Maybe it could be used to simply store the ammount of respawns left, or if the respawn camp has been destroyed.

I agree, you more than likely cant stor the location of each unit, bu values such as material etc should be a doddle.

This is exiting news for cce

[Rune 0]

I agree, you more than likely cant stor the location of each unit, bu values such as material etc should be a doddle.

True, when you are thinking of general data that can be added up like the number of enemy troops, there is almost no limit to how much you can save, but if you start to save data for individual units and especially positions of them then you will quickly get to a limit of sorts.

This is why I feel I need to add a little to what I've written so far:

SUGGESTED GUIDELINES FOR PUBLIC MISSIONS

If you are just playing around with this a little to make a mission or two to play with a few friends on LAN you really don't need to read the stuff below, but if you intend to release missions that save in multiplayer to the public there are some important guidelines you should really follow, at least if you have respect for your own work and that of others, which I think most mission makers fortunately do

Don't clog up the objects.sav file

The biggest problem if there are different missions using saving in multiplayer is the limited space or rather size of the objects.sav file - there is no hard limit that I know of to it, but saving and loading becomes progressively slower the more data is stored in it. An overfilled objects.sav file can often only be fixed by deleting the whole file, this means that data is lost for all games using saving in multiplayer, also ones made by and played by someone other than you. On a dedicated server there might be no one around to actually delete the file regularly and also there could well be other missions there such as SOW missions which would have the status of their campaigns deleted if the objects.sav file has to be deleted. So it would be by far the best way to not overfill the file in the first place.

Reducing the space your mission takes up

If you where to save a big CTI-game with every unit's position and ammo/damage status and facing and all that you would be looking at 3 to 10 minute minimums for saving and loading depending on how powerfull the PC and how big the CTI mission and such(IIRC from my research in early 2005). And that is based on only having one CTI-game saved at a time. That would all be fine if the mission maker that made the mission was only affecting his own mission's saving and loading times, but it affects everyone saving and loading data in multiplayer because all saving happens to the same file. For this reason I would suggest that anyone planning on releasing missions to the public while using more than about 100 to 200 individual entries should think carefully about reducing the space they take up to a minimum. This can be done by simply not saving too much obviously, for instance by not trying to save a 11 vs. 11 CTI, but limiting yourself to 6 vs 6 or something. You can also get a long way by thinking creatively about the way the data is stored, for instance a damage value really does not need the full precision of a real number, if you can settle for integers 0,1,2,...,99% you can tripple the amount stored in the same space, or if you are ok with 0,10,20,..,90% then a factor 6 is possible - this is the  kind of compression my functions are built to do for SOW.

Clean up after yourself

This one is easy to forget when making a mission, but remember that unless you actively delete the data you have saved it stay in the file. Imagine your mission is finished and the saved data no longer needed, then the easiest thing to do is to just leave all that data there to be overwritten next time the same mission is played. But I would advice you to always delete whenever you can to keep the size down because if many missions do that and especially if they also save a lot of data each, then the file will become overfilled over time - and there is no way to clean it up short of deleting the objects.sav file or making each mission able to clean up after itself. So make it a rule for yourself to make a cleanup script and activate it whenever possible, especially if you save more than just a few entries of data.

Protect your own data from accidental overwrites

The "public space" issue also introduces a risk of walking over the data of some other mission. Say two missions both save the time the mission has been underway in the entry "time", then you will mix the data up if you play one and then the other. The solution is to use tags in front of all your entries just like different variable names when mixing code from different sources. SoW uses a set of tags like "SOW_" at the start of all entries and another tag like "NubbinTNR_" right after the start tag of entries from a particular 'campaign space' - just to give an example.

[coma73 0]

hey guys i really enjoyed your ideas but ive been gone a while so lemme ask you ..did you get the bugs outta your mission? i had a problem playing it with my clan at the time. im back into ofp now and would love to see this up and running lemme know whats up if ya can

[nubbin77 0]

Which bugs are you referring to? The bugs in the missions themselves or in the SOW code? I haven't updated the missions in quite a while. I busy trying to creat a new type of CCE / SOW mixed campaign mission. I beat myself up about not going back to the TNR campaign that we created a while back, but I just lost interest in it for now. I left quite a bit of inefficient coding in there and I have learned quite a bit about mission design since then. So I am trying to incorporate what I have learned into a new mission.

Crashdome and R seem to have fixed most of the bugs related to the saving / loading and menu issues of SOW as of its latest version.

[General Barron 0]

AWESOME!!!

I've been wanting this for a LONG time (I think I never got around to asking though)--that is, the MP saving/loading functions being separate from the other SoW code. Packaging it this way makes your great work much more generic and useable by the community. I see many, many great things ahead using your work. Thanks a million!

Unfortunately, I still have yet to get heavily into MP editing. So I don't think I'll be playing with these for a little while... but I'm glad I noticed this thread! When OFPEC comes back up, be sure to make a news post about it there, and I'd love for us to host it.

Just taking a quick look at the download, I've got a couple suggestions/complaints:

1) The functions are all in the root mission directory!

Please put them in a separate directory so they don't clutter the mission folder. I think all large script projects should do this. I'm a bit of a neat freak in this regard, but there are practical reasons for this as well. For example, if you update the scripts, all I would have to do to update my mission is drag and drop the new folder into my mission.

2) No tags on your global variables!

*Puts on "standards police" hat*

Stick a tag in front of your functions (eg "SoW_SaveRaw", or maybe "MPS_SaveRaw" for "MP save"). This does 2 things: prevents other scripts from conflicting (unlikely but possible), and it also makes it easier to understand what each function in a script is doing. For example, if I look thru a script and see a "SoW_SaveRaw" function call, I'll know it is a SoW MP saving function, and I'll know what documentation to look in for more info.

3) Script initialization clutters up init.sqs.

Just something I've decided on thru my own projects: it is usually better to have a 1 line initialization in init.sqs for large script systems, instead of many lines.

So basically, all the lines you currently have in init.sqs would go into a "SoW_init.sqf" function/script which is inside of the MP saving functions folder. All the editor has to do is start this init script in HIS init.sqs via one line.

The reason? It makes updating a breeze. What if you add functions in your next update? What if you add variables, or other stuff that needs initializing? What if you remove functions?

If this is all initialized in a separate script, all the editor has to do is drop in the new script folder. Currently, the editor will have to make sure to change everything in his init.sqs properly. And I speak from experience when I say that can be easy to overlook.

----------

Anyway, don't take my suggestions as criticisms. I only take the time to think about and give suggestions on stuff I really like. SoW has always been pushing the boundries of OFP, and I'd love to see that continue. Keep up the great work!

[Rune 0]

Coma73 I am not entirely sure either which bug you are talking about. But there were two major ones we were working on at the time, we had people contact us about both. Crashdome believes he has fixed the network issue where more people would cause problems in the menus. The other issue was the so called 'super AI bug' where the saving function of a SOW mission was interfered with by having 'super AI' turned on. This bug I fixed by transferring the SOW saving to these functions and it is therefore fixed in the present SOW version 2.2. So please note that the operation of these functions is in no way affected by the use of 'super AI'.

AWESOME!!!

I've been wanting this for a LONG time (I think I never got around to asking though)--that is, the MP saving/loading functions being separate from the other SoW code. Packaging it this way makes your great work much more generic and useable by the community. I see many, many great things ahead using your work. Thanks a million!

Good to hear you approve

Quote[/b] ]Just taking a quick look at the download, I've got a couple suggestions/complaints:

1) The functions are all in the root mission directory!

(...)

2) No tags on your global variables!

(...)

3) Script initialization clutters up init.sqs.

>>FIXED<<...>>FIXED<<...>>FIXED<<

Download has been changed to reflect the changes(v1.1)

Honestly I have not been editing much for the last few months since my new graphics card won't ALT-TAB to windows:( So consider two in three of those to be oversights by a rusty programmer...the function name thing I actually never thought of, I mean I have always used tags for variables, but for some strange reason I never transferred the namespace logic to preprocessed functions lol. Must have had my mind elsewhere when I programmed the functions originally...anyway, good catch - thanks.

Quote[/b] ]Anyway, don't take my suggestions as criticisms.

Constructive criticism is way better than no criticism...no offence taken, you were spot on with everything

[.kju 3245]

i think adding your existing work to this thread might be a good idea or at least point the people where to download and especailly what you have made so far.

(in terms of playable stuff)

[crashdome 3]

Sorry to arrive so late....

This pack is Rune's hard work in trying to squeeze out probably the most important bits of SOW before ArmA is released and we have to recreate everything.

For the newcomers, we apologize our website isn't very meaningful. The truth is we just switched over to a new site and we are still working on the layout and haven't produced much content. You will see this change over the next few weeks.

In fact, you can expect SOW to take on a much LARGER role with the release of ArmA, but I won't give away any details too soon!

@Gen Barron:

In our own defense: The *original* SOW init file was always a bit over produced but it really comes down to the fact that most of the SOW code IS in it's own sqf file, but some external functions needed to be run at specific moments (i.e. before briefing, during, briefing, and then directly after briefing) that calling a seperate sqs file (or sqf) would have broken it. Some parts could have been removed and placed in other files.. yes... but in all honesty, I decided to keep the commented init.sqs file for ease of understanding for the mission makers instead of breaking it up into several files and confusing them further.

[Rune 0]

i think adding your existing work to this thread might be a good idea or at least point the people where to download and especailly what you have made so far.

(in terms of playable stuff)

Well, the link in my first post goes to the download section at SoW, so the link is there already

But just to try and be organized I will add a little help to any people new to the whole thing:

Sinews of War - Previous Work

If you are new to Sinews of War missions the place to start is at the Sinews of War download page I would suggest first downloading Crashdome's excellent "SOW v2.0 Demo Mission User Manual" to read before you play a mission. Then download and play "SOW v2.0 Stand Alone Demo (PBO Only)" with a friend or two. This will be a good way to learn how things work. And finally, once you know how to work the menus you can proceed to play the so far unfinished campaign by Nubbin77(which is really cool I can assure you) named "The New Resistance v0.98 (Source)" in the download page.

[Drongo69 117]

I have just been trying to implement this into CCE. I get no error messages when it runs, but after loading, all of the values I saved are zero. I have the SOW_Saving folder copied into my mission directory.

I'm hoping you guys can tell me what I'm doing wrong. Here are my scripts:

init.sqs

Quote[/b] ]; prep SOW saving

call loadfile "SOW_Saving\SOW_Saving_init.sqf"

.

.

.

save.sqs

Quote[/b] ]; called from SERVER\SERVER_UPDATE.SQS

?(!local server) : exit

? (lock_save) : exit

lock_save = true

~1

; create integers for saving

_com = commitment + 0

~0.5

_mor = enemy_morale + 0

~0.5

_exp = enemy_experience * 1000

~0.5

_mat = enemy_material + 0

~0.5

_pop = enemy_popularity + 0

~1

[_com, "CCE_com"] call SOW_SaveZC

~0.5

[_mor, "CCE_mor"] call SOW_SaveZC

~0.5

[_exp, "CCE_exp"] call SOW_SaveZC

~0.5

[_mat, "CCE_mat"] call SOW_SaveZC

~0.5

[_pop, "CCE_pop"] call SOW_SaveZC

~0.5

lock_save = false

~1

server_code = 96

~1

publicvariable "server_code"

exit

load.sqs

Quote[/b] ]; called from SERVER\SERVER_UPDATE.SQS

?(!local server) : exit

? (lock_save) : exit

lock_save = true

~1

_com = [CCE_com] call SOW_LoadZC

~0.5

_mor = [CCE_mor] call SOW_LoadZC

~0.5

_exp = [CCE_exp] call SOW_LoadZC

~0.5

_mat = [CCE_mat] call SOW_LoadZC

~0.5

_pop = [CCE_pop] call SOW_LoadZC

~1

commitment = _com + 0

~0.5

enemy_morale = _mor + 0

~0.5

enemy_experience = _exp / 1000

~0.5

enemy_material = _mat + 0

~0.5

enemy_popularity = _pop + 0

~1

lock_save = false

~1

server_code = 97

~1

publicvariable "server_code"

exit

Cheers.

[Rune 0]

I am very glad to see someone using this

Is "CCE_com" the string name you are saving under, or is CCE_com the variable name for the string name you are saving under? - your loading code uses one and the saving the other

By the way you are doing a bit of encoding into the _exp variable that you might want to use SOW_RealToInt to encode and then SOW_IntToReal, that will save you the trouble of trying to manually check the ranges for all possible values you might need to save - in this case I gather that you are only trying to save with 3-4 digit accuracy anyway.

The code for this might go something like this

<table border="0" align="center" width="95%" cellpadding="0" cellspacing="0"><tr><td>Code Sample </td></tr><tr><td id="CODE">_exp = [enemy_experience] call SOW_RealToInt

To replace the line in the saving code that says

<table border="0" align="center" width="95%" cellpadding="0" cellspacing="0"><tr><td>Code Sample </td></tr><tr><td id="CODE">_exp = enemy_experience * 1000And

<table border="0" align="center" width="95%" cellpadding="0" cellspacing="0"><tr><td>Code Sample </td></tr><tr><td id="CODE">enemy_experience = [_exp] call SOW_IntToReal

To replace the line in the loading code that says

<table border="0" align="center" width="95%" cellpadding="0" cellspacing="0"><tr><td>Code Sample </td></tr><tr><td id="CODE">enemy_experience = _exp / 1000

I also noticed that you have pauses in between the SOW_SaveZC and SOW_LoadCZ calls, there are none in the test-scipts for these functions and I have yet to see them create a problem...Since they are functions and entirely local they will not be overtaken by other events if you don't have a pause between them.

[Drongo69 117]

Is "CCE_com" the string name you are saving under, or is CCE_com the variable name for the string name you are saving under? - your loading code uses one and the saving the other

I... don't... know...

By "string name", do you mean the name of the entry in the save file? That's what the CCE_xxx entries are intended as.

I tried to copy the code in the saveloadtestZC.sqs script from the demo. I assumed that I just had to create a name for the functions to save under. Basically, I want to save and load commitment, enemy_morale, enemy_experience, enemy_material and enemy_popularity. The CCE_xxx variables are not used by CCE outside the save and load scripts. What should the code look like?

As for the pauses, it's just a habit I developed long ago as a result of failing to use pauses often enough

[Rune 0]

...you just made a very common and sometimes hard to spot mistake of forgetting the quotes - that is why I wrote ""...I thought what I wrote in the first post would be enough for you to see it, sorry dude - I know how frustrating that kind of bug can be when you can't find it.

As for the code you need to change:

Put quotes around your CCE_xxx'es in your loading script and all will be fine - I think

[Drongo69 117]

It's right there in your demo code too, can't believe I made such a noob mistake... the shame

Thanks for the quick responses, I'll let you know how I go with it

[Rune 0]

Lol, I make that mistake all the time myself and I bet most other coders do too...I am happy if the problems you get yourself into are that easy to fix

[Drongo69 117]

It seems to work fine now. Thanks very much for your help

[stegman 3]

Sorry, I have to post something or I'll lose the thread from my mailing list!

I'm trying out the new CCE by Tacrod that incorporates the SOW saving script. Very nice work.