Paras return to Helmand (Map editing questions)

[stormridersp 11]

Hi guys, I'm trying to create the maps for Helmand Province, but I have a few questions and need a little guidance.

Ok, lets start with the basic:

Im running Win7 64bit via Bootcamp on my new Macbook pro 2.53Ghz. For those who doesnt know what bootcamp is, I can say its not an emulation. It creates a new partition in my disk, using windows file system and fires up a regular installation like any pc user.

I have installed here:

-Global Mapper 12

-Microdem (really that difficult to use)

(Tried installing 3dm, but it crashes at startup)

-BIS Tools 2

-Google Earth Pro timed trial

-Arma2 OA, @Ace...

I found some really great resource maps fro USGS site that includes:

-SRTM 90m DEMs

-ASTER Level 1b

-Soviet 1:200k Topos

-Landsat ETM + Raw data.

http://afghanistan.cr.usgs.gov/flash.php

(Choose your data from the dropdown menu, click the map quad you want if available, then download immediatly icon from the left index below the drop down menu.)

But now I find myself in trial & error. Cant get the work done and need some guidance if anyone is willing to help.

[shezan74 11]

Well, from your actual position the way is simple. You have some clear ideas so all it's easy. There is a nice tutorial sticky on this forum. It's a good start.

To create your first island get a .pbl file from the tutorial file. The pbl contains some simple data, like elevation (min/max) of your black/white elevation DEM (you need a black/white image for getting elevations) and the scale of the map (1 pixel of the elevation map is x pixels on your map).

When you have this .pbl related to your heightmap you must place it into a folder, as explained on the tutorial and as you can see in the example map project. Beware PATH are important. At this point you can load your PBL into Visitor3 and you will have the terrain ready for work on Visitor3

Some hints: try L3DT software (freeware version is enough to start) and Wilbur. Wilbur will also produce a correct png export image with correct bit depth.

To better help you we need to know where you've found troubles :)

[stormridersp 11]

Thanks for your quick answer Shezan74, my troubles resides on the map programs themselves, at least for now given Im still in this phase of develpment.

Can I use ASTER maps or should I stick with the lower res SRTM 90m for the elevation?

Regarding sizes, given the fact that the maps I currently have, which are pretty big, 60x60km per quad, what should I do? Correct me if I'm wrong. I was thinking of merging multiple maps together and then crop in smaller quads.

1- How can I crop them and edit? Should I convert it to PNG, photoshop it up, and then convert back to DEM/XYZ for use in Visitor3? Any special settings, hints or tips on that?

2- One other idea, but I dont know whether its possible or not, would be to resize the entire map area for game balancing purposes. What I mean is, reduce the map lenght, for example, from 60km to 20km, thus convoy mission wont get too boring either for driving huge distances, and I could merge the entire map into one single island instead. The thing is, I'm not sure that this would fuck up coordinates or references or anything inside excluding the possibility of importing back or converting forward to XYZ or any visitor3 supported data. Any thoughts?

You'd mentioned a pbl file, but can you link me to what tutorial file youre refering to?

thank again,

stormrider

Edited October 12, 2010 by stormridersp

[shezan74 11]

You need a DEM image (not datafile). A DEM example in the correct format is:

Obviously more resolution you have, better is the final result. You can merge different DEM's but the final size is very important. For the first project i suggest no more than 10x10 km, so a crop of a good DEM is enough. 2048x2048, 5120x5120 pixels or 10240x10240 pixels dem could be used. (too big DEM anyway could create lag and troubles). 2048x2048 is absolutely ok for a 10km map.

Obviously you can also reduce a DEM scale, that is really used. In that case you must reduce the MAXIMUM HEIGHT of the same ratio.

For example if you load a DEM of 60x60 with min altitude = 0, max altitude = 3000 if you reduce the scale to 20x20 the max altitude must be set to 1000 (3000/3)

And 1000 meters is still a very high mountain on ARMA2 maps (consider climbing them...)

Finally you're not necessarily forced to keep the altitudes from real DEM, you can reduce them at the level you need (no more than the "original ratio" height, to avoid unrealistic peaks).

Hope this helps, i've had some complaints about my bad english :D

[mir_o 85]

Can I use ASTER maps or should I stick with the lower res SRTM 90m for the elevation?

The best way always is to get a data into the game and see how it works for proper area(comparing resolution, size, details...). For the Takistan map we made "a trick" where original Afghanistan area of 25.6 x 25.6 km was scaled down a half to 12.8 x 12.8 to get more mountain like areas in the map(of course height informations had to be scaled as well). Propably it`snt answer to your question but I just wanna say there are many ways how to play with input height data.

1- How can I crop them and edit? Should I convert it to PNG, photoshop it up, and then convert back to DEM/XYZ for use in Visitor3? Any special settings, hints or tips on that?

I usually use Global mapper to do this. I prepare vector rectangle, which coordinates are somewhere around your location(I prefer OpenJump), save it as a *.shp, load in Global Mapper(dont forget to set proper Projection and Datum). You can tweak rectangle position by right clicking on a layer and choose "Shift selected layer a fixed distance". If you are satisfied with position, select rectangle and go to export to XYZ or png. In "export bounds" tab, check "Crop to selected area feature". You should get only the area which is covered by vector rectangle. You will have to play a bit with the "Sample spacing" to get proper terrain grid size (1024x1024 or 2048x2048).

[bushlurker 45]

Quote:

Originally Posted by stormridersp

Can I use ASTER maps or should I stick with the lower res SRTM 90m for the elevation?

Depends on the ASTER data for that area...

Briefly - SRTM was derived from Radar scans - radar is good, clouds don't matter - the resolution wasn't brilliant though, at 1pix/90m... this kind of resolution can be slightly disappointing around coastlines in particular, where you tend to get abrupt dropoff which usually needs tweaking.

ASTER is optically derived - it's light-based sensor arrays - not just visible light right enough - that means there were multiple "layers" sampled at each light frequency range for each "granule"/tile... If you're cunning - and you have no alternative - these "L1B Registered Radiance At the Sensor" layers can be combined in various ways to produce "False Colour Composites" - since different ground covers reflect light/heat primarily at distinctive frequencies you can derive ground cover info (ie: Vegetation Masks) from that sort of data... Concrete, trees, even crops - all stand out like brightly coloured beacons if you can find the right frequency/colour layering combination...

Posh programs like ENVI will do this sort of stuff automatically, but it can also be done in any art package where you can layer RGB and select via "colour range"...

Getting ahead of myself there tho... the ASTER DEM's however, are also derived from two of these optical layers... that's tricky to do, but fortunately NIMA/NASA have caught up now, so it's all ready done when you download...

The good thing about ASTER is its resolution - 30m/pixel is much better than SRTM - the downside is the "optical" scanning - clouds DO matter - unless you're lucky there will be "holes" - NIMA may have "fixed" these (usually shows as a suspiciously flat bit)... Coastlines are better - no such an abrupt dropoff, but oceans often end up being very shallow (easily tweaked)...

So thats the basic differences between the datasets - which should you use?

Depends on what area you want to make... how big a terrain you want your final result to be, how many years you want to spend on your project ;), etc...

Mir_o's pretty much on the money with his suggestion - don't over-theorise - get it in game and take a look! - it's the easiest way to get a feel for the effects of scaling...

My Global Mapper memories are a little rusty - theres various other programs you can use too, but if I remember correctly, again, Mir_o is right - get the DEM in there (I believe the new GM has a direct hook to ASTER downloads within the program - SRTM direct download is in there too) - just make an exact square with the vector shapes tool - scale it up or down a bit - move it around like a "frame" over the map/dem of the area you have loaded - "frame up" the area you think you might want - then just export that section directly - using the vector shape like a "cookiecutter" - the "spacing" can be used to control output resolution - but for the purposes of testing - once you get your output raw file - just scale it in a paint program, or L3DT (in 16bit greyscale!!!) to 1024x1024, or 2048x2048 for starters...

At this stage you'll need to quit thinking temporarily about actual realworld size and start thinking about in-game size...

Effectively - you need to decide what size your project is going to be... this is a juggling act between Heightfield Resolution and Cell Size basically...

EG:

1024x1024 heightfield with 10m cell size = 10240x10240m terrain (10x10km)

1024x1024 heightfiled with 5m cell size = 20480x20480m terrain (20x20km)

2048x2048 heightfield with 10m cell size = 20480x20480m terrain (20x20km)

I'm pretty sure SnakeMan (?) and/or the PMC crew had a chart detailing this, but I can't remember where...

At this point it's best to consider the limitations both of Visitor, and of the game engine itself - while still bearing in mind the data resolution of the DEM, etc we considered earlier (told you it was a juggling act!)

Visitor has been known to misbehave with 4096 heightmaps on occasions - people have reported problems placing objects in certain areas...

A cell size of less than 5m may look awesome, but performance hassles and crashes were reported with Betons experimental 2m ground res island...

Photoshop doesn't really like images bigger than around 30000x30000 - problems have been reported both making and importing Satellite images bigger than that (I tried a 28672x28672 and it worked -but took literally HOURS to import) - since for decent appearance you want to keep your satellite mask 10x your heightmap - so 1pix = 1m in distance view, theres another upper limit being imposed right there...

Unless you want to be all year making your first map (not a good idea, trust me - I bit that big .50cal :(), you should really aim for a managable standard project size like 10x10km - the classic size... Takistan is just a little bigger than that - and it's plenty big enough... since you're talking Afghanistan where theres an awful lot of not very much you might be able to push it to 20x20km - but bear in mind you then have to fill that convincingly with objects...

Of course you could go much larger - 100x100km isn't impossible, but then you'd need to use larger and cruder cell sizes... So what? you might say... if I'm using 90m res DEM, 75m cell size is OK... Well, yes it is, but its the "transitions" between cells that are important - you may indeed get a beautiful sweep of terrain 50km across - all authentic looking - for flying over... when you land it'll look a whole lot cruder on the ground - one cell is labelled at 90m height - so it is... the next cell is labelled 180m - so it is! - you'll get an enormous "step" where really what you wanted was a smooth gradually rising hillside... if that step were broken down into a series of 10m gradations the "step" wouldn't be so obvious.... back to 10m cell size right there!...

1:1 or realworld scaling is a bit of a holy grail to some people - usually non-island designers... it's important to bear in mind that in Arma 2 almost everything IS scaled anyway to a certain extent... plus - we expect battles to last a few hours - not a few days...

In the real world Loch Ness in Scotland is about 20miles+ x about a mile wide - a formidable barrier! It's huge - trust me, you could hide a brontosaurus in there! :D In my terrain, its about 2 miles by 300m - still a formidable barrier, it'll still take you a while to leg it to the nearest bridge... it'll be a "game" while though - 15 minutes holding the "w" key is enough to make the point - no need to make players run for 6 hours, it would stop being fun long before they got there!... Besides - my map is 20x20km - if it was 1:1 all you'd get would be 2/3 of Loch Ness and some hills - maybe one village - coz in the real world - thats what its like! Don't be afraid to scale - a little... it's OK...

There's a place for 1:1 of course - if you're determined to depict a relatively small area as accurately as possible...

A final point on scaling then I'll shut up since I've forgotten the question anyway :) - scaling heights.....

In the old days of "landscape visualisation" it was all a bit crude on the GIS 3D display front, but even then we knew that if - for the purposes of display, you scaled say a 100km x 100km area to 10x10km then, if you reduced the heights by exactly the same proportions, it looked awful flat... the trick was to use "vertical exaggeration" - carefully - around 1.2 - 1.4 was usually OK... what this basically means is that if you reduce by a factor of 10 on the X & Y coordinates - only maybe scale by a factor of about 7 on heights...exact proportioning just doesn't look right...

B

Edited October 18, 2010 by Bushlurker

[stormridersp 11]

Thanks for the answer guys, I succeded in some different ways, but still stuck in others.

What I did:

1- I've downloaded a Photoshop CS4 plugin called Avenza Geographic Imager 3.2, which allows me to import Geotiff elevation maps (SRTMs) and Georeferencing. The problem is that it is a very expensive plugin, 540+ € to register, so I stick with the time trial which is already almost expired.

2-Downloaded from USGS the 2 quads containing Helmand elevation (SRTM, in geotiff format, geo, not UTM (srtm_49_06 and srtm_50_06).

Downloaded the 2 quads of land coverage for the texture, (Sat Imagery, again, geo, not UTM. (Q3164_28m_GEOG_NC and Q3264_28m_GEOG_NC).

3- Opened it using Geographic Imager, tiff/srtm and merged the two quads using Geo Mosaic. Specified the Coordinate system to Geodedic Herat North. Did the same with the Land Coverage. Now I have 2 big images, one containing the merged elevation and the other them merged sat image.

4- Using Geocrop, I cropped the sat image region containing Helmand. Using the same tool, I geocropped using the "crop to another documents extend" to crop automatically the same area that I cropped the Sat Image. It is important to note that Both Geotiff elevation and Land Coverage be downloaded using the same referencing: Either Geo or UTM.

5- Exported both to Geotiff (1024x1024 for the heightmap) and (2048x2048 for the texture) and Opened in L3DT Pro trial and Changed the vertical limit to 0-400m (dont know if this is necessary but I did so).

6- Created the terrain.pbl and edited the Min and Max Height to 00.000000 and 400.000000 (correct me if I'm wrong here)

class cfg

{

PNGfilename="terrain.png";

squareSize=20.000000;

originX=0;

originY=0;

minHeight=00.000000;

maxHeight=400.000000;

};

7- Extracted Takistan's pbo to my Data folder, inside P:\ and choose the 4 textures I would like to use while I'm still in testing phase and just want to make it work now, trying later to grid the map and trying 4 textures per square grid.

8- Created a layer.cfg using Opteryx Visitor 3 Quick Tutorial Example and edited them RVMATs.

9- Created a new layer on top of the Land Sat and painted it using the color pallet configured by layer.cfg.

Is there any error from the begining to this point I should be noted for before continuing???

Cheers

[shezan74 11]

Steps until this point are correct, and looks good if your skill is similar to your effort to create a good map. Nice job :)

Just a question: where you get the georeferenced sat pics for free? I'm curious for another world area that could be covered by satpics...

[stormridersp 11]

Thanks mate, I can say I'm still improving my skills, while not letting my motivation falls on the way!!!

http://afghanistan.cr.usgs.gov/flash.php

this is afghanistan's published resources by USGS. It contains almost its entire area covered in different types of maps and resolution. Just click on the drop down button, choose the type of map you're looking for, select the squares and quickdownload.

[bushlurker 45]

Seems everyone is making maps of Afghanistan at the moment...! there's data everywhere!

One thing worth keeping an eye out for are "shapefiles"... often in ESRI (ArcGIS) format, they'll load into almost any basic GIS setup and overlay just fine on your other georeferenced data... then you can crop it out in raster format at any scale you want - confident it'll overlay perfectly with everything else...

you get lots of data in vector/shapefile format... the obvious one to look out for is Vegetation/land cover/irrigated areas - these can save you a lot of time when you get to the Forest Tools stage...

HERE's an example site...

I haven't looked at the actual data, but "irrigated areas", "land cover", "lakes", "cultivated areas"... all ready-made in polygon format sounds awful promising ;)

If you're making a realworld area its well worth searching around to see what data is available - "imaginary terrain" guys don't have all these options, and mask-making is a lot harder... realworld guys have it easy in comparison, it's often just a question of searching thoroughly...

B

[stormridersp 11]

Thanks Bushlurker, that seems very promising indeed. I'm startingmy research on that now. The more we can automate our workload, the faster, easier and better our maps will get.

After downloading some of the shapefiles provided at the link you recomended, I found out I don't know how to open and/or use a shape file so I get to this website. It can be very useful for anybody willing to follow this path too.

http://landtrustgis.org/technology

Yesterday I tried the Map Editing Starter Pack, but I could not make it to work cause I still only have OA Standalone here and many of the texture or objects included in the config file relates to Arma2. So I'm one step back now. I need a working config file related to Takistan map and data only. Any suggestions?

Edited October 29, 2010 by stormridersp

[bushlurker 45]

I think that map starter pack is really for Arma 1... not sure though...

OA only island making? Thats a first I think... There's gonna be a couple of snags... firstly, the models are a new version, and until we get a tools update you'll need to tweak the ones you want to use... search this section for "OA Objects"...

Similarly, the road tool in Visitor requires mlod road models - the ones provided with Visitor 3 are for Arma 2, the dusty style OA roads won't work directly, since they're binarized...

BIS traditionally release appropriate road models, so everybodys kinda still waiting on that stuff...

I'd seriously recommend buying a copy of Arma 2, just so you have the complete Combined Ops thing, it also gives you access to all the Arma 2 models of course, and Combined Ops is well on the way now to becoming the "Standard"...

For a working OA config, you could always start by taking a look at the one for Takistan...

B

[stormridersp 11]

This is the result of a multilayered image containing the srtm elevation map, a sat land coverage, rivers/roads and landcover shapefiles. The question now is how to use this to automate our effort in importing to Arma2.

ShapeMaps Layered by rider.storm71, on Flickr

The bad news is that I screwed up here. I was having problems with Buldozer not showing names at buttons, and messed with files. Now for some reason, not even my ingame mission editor is working properly. Cant doble click to create objects....wierd! OA Standalone, sorry BIS, but it *&@#%. I dont recomend it to nobody. And the original Arma2 or even Combined Ops is not found anywhere here in Spain. I've ordered then from amazon my arma2 copy, but due to some unknown reason, its now 1 month since i've being waiting for the shipment to arrive from the UK to Spain........

Hard times... Meanwhile, my Global Imager expires and L3DT pro is on its way too.

[bushlurker 45]

Bad news on the visitor/editor/tools screwup... might be worth a reinstall - maybe backup the whole P:\ drive first and selectively replace stuff afterwards...

As far as the GIS data is concerned... now you improvise!

For shapefiles... well, theres several types - point, line and polygon basically...

We don't have a tool that makes use of point data yet - Rocket was working on a script that translated point data into object placement, but I think he got posted somewhere and has been AWOL from the forums for a while now - so I guess thats on hold... Maybe Shezan, our new resident coding genius, will come up with something for world tools eventually... theres almost always "place name" point data - a way of translating these directly into "named locations" or something might be handy... Probably the least useful data though, points...

Line data is usually for rivers (not much use to us - terrain limitations) and roads... Road data is more useful - it can't be translated directly into import scripts (so far - Homer will be back - eventually... last I heard from him he was decorating his girlfriends house or something :)) - so we'll see... Meantime, road data often has associated "metadata"... as do all shapefiles actually... in the case of roads, it may consist of a "name" and a "classification" - depends on the data...

You can call up the metadata in a table usually and search it - do things like "show me all roads classified as A roads", and it will - you can then colour all of these green, output it as a raster layer and use it as a road guides overlay for Homers tool... do the same with B and C roads - colour your roads to match in the Visitor road tool and get going - replace all the green guidelines with A roads, the red with B and the white with C - or whatever...

Final shapefile type is polygons - the most useful... again, the metadata may contain several types - you can display only the ones labelled "forested areas" for example - colour them all some colour - export the raster overlay and use it directly in Shez's Forest Tool for auto forest placement...

Theres no real guidlines or set procedures or even set tools for doing all of this stuff - you just have to improvise and make it all up as you go along...

This is the fun part... ;)

B

Edited October 30, 2010 by Bushlurker

[stormridersp 11]

Since I'm out of visitor editing for the next few days/weeks/months til my arma2 copy finally ships, I'm back to the drawing board: heightmap editing and theorical planning.

I have been wondering for some time now about rivers. Searched the forum and found some old posts, usually without any recent update, so no conclusions taken. Status Quo.

What about rivers? Is it possible? How? Did anybody checked the river at Zargabad map? That looks exactly how I would like Helmand river to be.

If the problem was not already too much, Helmand river runs downhill, with a difference of almost 60m in altitude (ingame resized) from Kajaki located at the northeastern edge to Lashkar Gah near the south western edge. Is there any definite updated solution for this problem? If not, I can only think in only solution. Re-designing the heightmap, so everything stands like 1 metre above sea level, with no altitude difference, basically, a flat map, with few mountains and relieves along the map. To think is easy, to make it happens is another story which I can't tell and need help there.

That's it for now.

Cheers mates

[prowler.wolf 0]

The only way I have gotten rivers to work is by cutting a trench to below sea level (like zargabad) from a terrain that has no slope in it. If you are going off of a real world terrain you might want to consider making one from scratch so you are able to eliminate any sloping. Otherwise you are SOL and messing around with pond objects is a waste of time. :/

[stormridersp 11]

The only way I have gotten rivers to work is by cutting a trench to below sea level (like zargabad) from a terrain that has no slope in it. If you are going off of a real world terrain you might want to consider making one from scratch so you are able to eliminate any sloping. Otherwise you are SOL and messing around with pond objects is a waste of time. :/

Thats what I was afraid. Any suggestion on how to modify a real heightmap to remove its slope? Better if its automated.

Options I have in mind:

1- Photoshop clone tool or Color replacer...

2- L3DT raising ground

3-Photoshop, cutting out and then pasting back in mountains and terrain into a new levelled ground

4- Suggestions?

If I use Photoshop as my tool, what settings should I apply in order to be able to import directly into Visitor3 or at least compatible with L3DT, to export later?

Cheers

[bushlurker 45]

I think Homer Johnston explained the 8bit / 16bit thing somewhere, but I can't find it, so I'll summarise...

For Photoshop, or any other bitmap/raster editing program, make absolutely sure you keep your heightmap in 16bit format - massively important!

Visitor needs a 16bit greyscale .png for importing, L3DT will accept .png, .bmp, .geotiffs and a few more obscure formats, but the important thing is the 16bit part...

An 8bit greyscale has 256 shades - so that means only 256 possible height values - bad news if your range is 0-400m - using 8bit format means all your height data for each "cell" will be shoehorned to fit the 0-255 range - not good...

Worse still, most of your heights will be fractional - eg: 12.26m - no room for that either with such a narrow range of values...

16bit images give you 65535 possible values - more than enough...

Watch out for that - 16bit all the way...

As for actually editing the heightmap to remove slope... hmmm, I dunno...

Theres basic "clipping" tools in L3DT for editing overall height ranges - you can also do basic "filtering" in Global Mapper if I remember correctly, but I'm not sure exactly how you'd approach this...

I have similar problems with rivers - I've had to skip a few so far - Mighty Scottish salmon rivers too, like the river Tay - which is pretty big, but which is also at an average height of 100m or more! - I tried bulldozing Loch Tay (from where the river flows) to sea level - it looked like an enormous hole in the ground! No good at all...

Right now Loch Tay is pond objects and the river is abandoned... which results in places like "Corrbridge", "Spean Bridge", etc - where there are neither bridges, nor rivers... or "Granton-on-Spey, with no river Spey to be "on"... ;(

B

[oldbear 390]

I believe that Beton has shown us a trick for such an issue on his Hürtgenwald map.

As I havn't seen the .pew, I can't say if Beton is really using it but it seems so ..., well he had laid pond objects in rivers bed parts, each part being separated from the other by rocks or dam or whatever other obstacle.

Of course the waterfalls are missing, but I believe it can be a mean to get small rivers, of course with "dead pond water", in altitude.

But no way to get a rushing mountain torrent in-game.

Edited November 2, 2010 by Old Bear

[stormridersp 11]

That sounds good to me. Separating objects could be used as crossing tracks.

[stormridersp 11]

Screenshot3 l3dt by rider.storm71, on Flickr

Screenshot l3dt by rider.storm71, on Flickr

One question. Im taking the river at sea level path to simulate rivers, but in L3dt I set my heightmap to be from 0 to 600m and set the heightmap to be at 1m and dug the river to 0m. Will this work with Visitor3?

Edited November 9, 2010 by stormridersp

[stormridersp 11]

These are the first ingame screenshots. The map currently has only a low-res satmap and only 1 texture, which is grass.

The next step I plan to import a new sat map, high-res, and re-do the mask with takistan textures. I will also tweak the heightmap, it looks way too smooth.

Comments and opinions are very much welcome!

arma2OA 2010-11-21 22-03-29-38 by rider.storm71, on Flickr

arma2OA 2010-11-21 22-10-51-52 by rider.storm71, on Flickr

[stormridersp 11]

So, after testing a few maps I came to the conclusion that a Helmand Map including more than one city is not actually possible, so Im changing the project to Musa Qaleh, a remote DC on the northwestern area of Helmand.

I`ve already downloaded and applied a Hi-Res Satellite Texture and also drawn the Mask in PS.

Everything is working fine! :)

Next step is to drop objects and more vegetation. Mmm..I still need to learn that!

Here some screens

---------- Post added at 10:59 PM ---------- Previous post was at 10:02 PM ----------

Here some screens:

Musa Qaleh by rider.storm71, on Flickr

Musa Qaleh by rider.storm71, on Flickr

Musa Qaleh by rider.storm71, on Flickr

Musa Qaleh by rider.storm71, on Flickr