Clutter, Buldozer, and You

[Pennyworth 180]

Preface
I want to start by saying that my intention isn't to make a quick-and-dirty guide to getting clutter working in Buldozer. I think it's important to try and understand how the technology works, so I will explain in probably too much detail how I figured this out.

While trying to determine if it was possible to update Buldozer lighting to look similar to the lighting in game, I stumbled upon something rather interesting. I noticed that Buldozer was reading "Arma 3 Steam Installation\dta\bin.pbo", or "P:\bin\config.cpp", if "Arma 3 Steam Installation\dta\bin.pbo" wasn't present. I had known that Buldozer loaded a config from somewhere, but never really looked into it in detail. It came as no surprise that there was an entry for "CfgWorlds >> DefaultWorld" in there, but I couldn't figure out if "DefaultWorld" was the terrain being used by Buldozer, so I ignored that for the time being. What was more interesting was that there were entries for CfgSurfaces and CfgSurfaceCharacters. Well, why is that useful?

 

How CfgSurfaces, CfgSurfaceCharacters and Clutter work
If you understand how surfaces and clutter work, then you can skip to the next section. To the best of my knowledge, the full process of how clutter appears on your terrain goes something like this:
 

1. You define a material for a surface in your layers.cfg (e.g. material = "yourTag_yourMap\data\yourTag_someGrassSurface.rvmat";)

   Your RVMAT contains two "stages",
   //yourTag_yourMap\data\yourTag_someGrassSurface.rvmat
   /* ... stuff ... */
   class Stage1
   {
   	texture = "yourTag_yourMap\data\yourTag_someGrassSurface_nopx.paa";
   	/* ... stuff ... */
   };
   class Stage2
   {
   	texture = "yourTag_yourMap\data\yourTag_someGrassSurface_co.paa";
   	/* .. .stuff ... */
   };

    

2. When you generate layers in TB (Terrain Builder), it takes your original satellite, mask, and normal (if you are using normal map) images and splits them into "tiles". At this point it reads the RVMATs from your layers.cfg (e.g. yourTag_someGrassSurface.rvmat) and creates one, or more RVMATs for that specific tile (e.g. p_000-000_l00.rvmat). The generated tiles (e.g. p_000-000_l00.rvmat) use your yourTag_someGrassSurface_nopx and yourTag_someGrassSurface_co textures, instead of the original yourTag_someGrassSurface.rvmat.
    

3. You define properties for that surface in "CfgSurfaces", for example:

   class CfgSurfaces
   {
       class Default;
       class yourTag_someGrassSurface: Default
       {
           files = "yourtag_somegrasssurface_*";
           character = "yourTag_someGrassSurface_character";
           soundEnviron = "grass";
           soundHit = "soft_ground";
           impact = "hitGroundSoft";
       };
       //more surfaces
   };

    

   The important things to note here are "files" and "character". 
       a) The "files" corresponds to the name of the texture used in your surfaces RVMAT (yourTag_someGrassSurface.rvmat), in this case yourTag_someGrassSurface_co, or yourTag_someGrassSurface_nopx. I couldn't tell you definitively which is used. This is why it's important you tag the name of your textures, if you don't you could easily mess with the vanilla or third party terrains if you are using the same texture or surface names.
       b) The "character" corresponds to an entry in your CfgSurfaceCharacters.
    

4. You define properties for that surface in "CfgSurfaceCharacters"

   class CfgSurfaceCharacters
   {
       class yourTag_someGrassSurface_character
       {
           probability[] = {0.4, 0.4};
           names[] = {"yourTag_GrassGreen", "yourTag_ThistleThornGreen"};
       };
       //more surface characters
   };

   So now your character is defined, but now the game needs to know which models to actually place on your terrain. The entries from "names[]" point to things defined in "CfgWorlds >> yourTag_yourMap >> clutter". 
    

5. You define the actual clutter models that will be used in "CfgWorlds >> yourTag_yourMap >> clutter".

   class CfgWorlds
   {
       class someBaseTerrain;
       class yourTag_yourMap: someBaseTerrain
       {
           //lots of stuff before this
           class DefaultClutter;
           class clutter
           {
               class yourTag_GrassGreen: DefaultClutter {/* config entries here for your clutter */};
               class yourTag_ThistleThornGreen: DefaultClutter {/* config entries here for your clutter */};
           };
       };
   };

    

6. When your terrain is loaded in game, and you go to a specific location, it looks at which of the generated tiles RVMATs (e.g. p_000-000_l00.rvmat) corresponds to the location you're at. Using that, it applies the nopx and co textures for all the surfaces on that tile, and possibly surrounding tiles.
    

   Now you might ask, where does the clutter come into this?
    

7. For each of those surface textures it loads, it looks at the "files" config property for ALL of "CfgSurfaces" for something that matches the pattern of the texture it's searching for. In the case of yourTag_someGrassSurface_co, it matches the surface with the "files" entry of "files = yourtag_somegrasssurface_*", which has a "CfgSurfaces" entry of "yourTag_someGrassSurface". Again, this is why it's important to tag your surfaces or surface texture names. At any given moment, every single terrain you have loaded has all of its CfgSurfaces loaded at the same time, so it's completely plausible surfaces and textures that aren't tagged can conflict.
    
8. Now that the game knows what surface is being used, it can assign all of that surface's properties to it. For example, if you were to walk on that surface (yourTag_someGrassSurface), you'd get the "grass" sound, if you were to shoot that surface, you'd here the "soft_ground" hit sound and you'd see the "hitGroundSoft" particle effect, and most important for us, the "character", which is "yourTag_someGrassSurface_character". If you get the position of that command with scripting and pass it to the surfaceType command, it should return "#yourTag_someGrassSurface". If it returns "#Default", that likely means it didn't match the texture for that surface with any "files" from the various "CfgSurfaces" entries.
    
9. Sweet, it found a match for the surface (yourTag_someGrassSurface), and now it knows the character (yourTag_someGrassSurface_character). As you might guess, it then looks at "CfgSurfaceCharacters >> yourTag_someGrassSurface_character" to see what character needs to be used. The "names[]" of the clutter used by "yourTag_someGrassSurface_character" are "yourTag_GrassGreen" and "yourTag_ThistleThornGreen". Similar to "CfgSurfaces", the "CfgSurfaceCharacters" for all terrains are loaded at the same time, so tagging is important.
    
10. Finally, now that it knows the names of the clutter it's looking for (yourTag_GrassGreen and yourTag_ThistleThornGreen), it looks in "CfgWorlds >> yourTag_yourMap >> clutter" for matches. Now it can display the corresponding clutter models on that surface. From what I can tell, it only looks for the "clutter" entry on the terrain you're currently loaded on, but I'm not 100% sure on that.
     

     

There you have it, from start to finish how clutter is added to a terrain (I think ).
 

Testing and Experimentation
From previous tests, I concluded that the surfaceType command returns "#Default" at a certain position if it didn't match the texture for a given surface with any "files" from the various "CfgSurfaces" entries. With some scripting, I was able to show what surface was at the middle of the screen at that time in Buldozer by adding the following to the initBuldozer.sqf

[] spawn {
    while {true} do {
        sleep 1;
        private _pos = screenToWorld [0.5,0.5];
        _surface = surfaceType _pos;
        systemChat format ["Surface: %1, at :%2", _surface, _pos];
    };
};

As expected, it returned "#Default" for everywhere I went. Well, that makes sense, it's not finding any "CfgSurfaces" with a matching "files". So I wondered, what if I added my surfaces to the "CfgSurfaces" in "bin\config.cpp"? We already know that it's reading "bin\config.cpp", and "CfgSurfaces" is accessible across all terrains, so that means even if I don't know what "CfgWorlds" Buldozer is loading, I should still see what surfaces are being used. Lo and behold, it worked! With a test terrain, I was able to get it showing the corresponding "CfgSurfaces" entry for the surface that was below the cursor. As shown in the gif below, you can see it prints the name of the surface below the cursor as it moves.

 

 

Alright, well that's fine and dandy, but we're still missing clutter. We know "CfgSurfaceCharacters" is also accessible no matter what terrain you're using, but that's useless without the clutter from "CfgWorlds" for that terrain. Here is where I got a bit lucky, I just guessed that it's loading "DefaultWorld", and I'll be damned, it is. So after adding some basic clutter to "CfgWorlds >> DefaultWorld >> clutter", I saw this.

 

 

To confirm that it wasn't just a flook, I tried the same thing with Takistan and this was the result. Success!

 

 

 

How-to
As a warning, in its current state, this is something I'd only recommend for advanced users. To get it working initially I'd recommend experimenting on a smaller terrain so that you can quickly open and close Buldozer to test. For those not aware a "//" comments out everything on that line, and a "/*...*/" comments out everything between the asterisks, so those were used to provide more context. For the sake of length, I also snipped out parts of configs to allow things to be more readable.

 

1. Open "P:\bin\config. cpp "
2. Find "class CfgSurfaces", and add the "class CfgSurfaces" for your terrain. In my case, I split the "class CfgSurfaces" from my terrain's original config. So after modifying it, this how it looked:

   //P:\bin\config.cpp
   CfgSurfaces
   {
       class Default {/* default surface that I didn't touch */};
       class Water {/* default surface that I didn't touch */};
       #include "\path\to\your\CfgSurfaces.hpp" //could be a different file extension than hpp
   };

   For example, my "CfgSurfaces.hpp" looked like this:

   //P:\path\to\your\CfgSurfaces.hpp
   class myTagSurface1: Default {/* various surface properties */};
   class myTagSurface2: Default {/* various surface properties */};
   //more surfaces, etc

    

3. Find "class CfgSurfaceCharacters", and add the "CfgSurfaceCharacters" for your terrain. In my case I split the "class CfgSurfacesCharacters" from my terrain's original config. So after modifying it, this how it looked:

   class CfgSurfaceCharacters
   {
       class Empty    {/* default empty character that I didn't touch */};
       #include "\path\to\your\CfgSurfaceCharacters.hpp"
   };

   For example, my "CfgSurfaceCharacters. hpp " looked like this:

   //P:\path\to\your\CfgSurfaceCharacters.hpp
   class myTagCharacter1
   {
       names[] = {"myTag_someClutterName1", "myTag_someClutterName2", "myTag_someClutterName3"};
       probability[] = {0.2, 0.2, 0.2};
   };
   //more surface characters, etc

    

4. Find "DefaultWorld >> Clutter", and add the clutter from your terrain. In my case I split the "clutter" from my terrain's original config. So after modifying it, this how it looked:

   class CfgWorlds
   {
       /* didn't touch the config entries before this */
       class DefaultWorld
       {
           /* didn't touch the config entries before this */
           class clutter
           {
               #include "\path\to\your\clutter.hpp"
           };
           /* didn't touch the config entries after this */
       };
   };

   For example, my "clutter.hpp" looked like this:

   //P:\path\to\your\clutter.hpp
   class myTag_someClutterName1: DefaultClutter {/* various surface properties */};
   class myTag_someClutterName2: DefaultClutter {/* various surface properties */};
   //more clutter definitions, etc

    

5. Check your Buldozer RPT to determine if "Arma 3 Steam Installation\dta\bin.pbo" is being loaded. You would only see it in the RPT if you removed "-noLogs" from your Buldozer parameters in TB. You'll see it about 40 lines down in the RPT in the "==== Loaded addons ====" section.
        If you don't see it loading "Arma 3 Steam Installation\dta\bin.pbo"
             1) Launch Buldozer and see if your clutter is present.
             2) If you do see clutter, celebrate! If you don't, then briefly mourn, check everything is done correctly, and try again.
       
        If you see it loading "Arma 3 Steam Installation\dta\bin.pbo", proceed to make a backup of that PBO. It's not the end of the world if you lose it, as you can always verify integrity of the game cache on Steam. Just don't forget to restore the original PBO if you actually want to play A3.
             Option 1, deleting the PBO:
                  a) Delete "Arma 3 Steam Installation\dta\bin.pbo". If Buldozer doesn't see that PBO then it *should* default to using "P:\bin\config.cpp".
                  b) Launch Buldozer and see if your clutter is present.
                  c) If you do see clutter, cheer! If you don't, take a moment to shed a tear, verify you didn't make a mistake somewhere, and try again.
             Option 2, replacing the PBO (more tedious):
                  a) Using a PBO packer of some kind (I'd recommend pboProject), pack "P:\bin\config.cpp".
                  b) Copy your newly packed PBO and replace the existing one in "Arma 3 Steam Installation\dta\bin.pbo"
                  c) Launch Buldozer and see if your clutter is present.
                  d) If you do see clutter, scream with joy! If you don't, temporarily weep, make the necessary sacrifice, and try again.

 

Conclusion
Thanks for taking the time to read this. Hopefully it wasn't too dry or bloated, and people found it helpful. My goal with explaining some aspects in detail was in hopes that others might be inspired or have ideas on similar aspects. If you believe any of the information is incorrect, or have a better explanation for things, feel free to let me know. And thanks to Lappihuan for letting me bounce random things off him.
 

I haven't had time to try and achieve my initial goal of changing the lighting in "DefaultWorld", but I think this was a good diversion. It seems like there is potential here for meeting that goal, and allowing for more as well.

[Richie 330]

Clever, very clever

[mikero 79]

what's surprised me here is buldozer isn't reading THE config.cpp/bin for THE wrp being displayed. Immediately followed by "it can't" because: 1) it won't have the class it inherits from (stratis, caworld, whatever) and 2) the config won't necessarily be adjacent to the wrp, so won't find it.

 

So, since clutter, per se, is not baked into the wrp, there's nothing to display, hence, the need for pasting config data to something it must read.

 

Although not related in any way, it is a similar cause to the minForestSquares issue that binarise (in this case) needs to know about.

 

Nice investigative work, nice proof of concept, and a hell of a bonus for people needing to twiddle their clutter 'on the fly'.

 

The incidence of bin config being altered by updates is currently quite low, so anyone taking the trouble to apply this patch should be confident of being able to 'set and forget'.

 

[mondkalb 1087]

You can change the default world config that is being loaded in two ways:

1. Using the -worldCfg=yourCfgWorldsClassName startup parameter in the buldozer link
2. Changing the initWorld=; parameter in those core configs to be your world config

[mikero 79]

changing initworld=  in your map's config also has the same (and less damaging) effect. From then on, if you're map is available in mod\somewhere\addons it becomes the default, by default