Convert DEM to xyz for Visitor 3 Using MicroDEM

[DC39 1]

Hello all!

I'm following the ZGM Arma 2 Map Making Tutorial (can be found on armaholic but I can't post links yet) in my first attempt at a terrain project for Arma 2.

I haven't gotten very far into the tutorial as it says to use 'Global Mapper' to convert my DEM file to xyz, but Global Mapper isn't free and I can't find a trial key for it, so instead I am using MicroDEM.

This has presented a big problem because I haven't been able to find any in depth guides for the program relating to Arma 2 and Visitor 3 compatibility - yes I HAVE searched the forums, and armaholic, and google... lol

SO, basically I have a DEM file, which I know how to open in MicroDEM, and I think I've worked out how to crop (or 'subset') the area of terrain that I want. (But I'm not sure about that :/)

My first question is how can I crop or subset part of my DEM at a specific size? I want to make a map 15km by 15km, I have the long & lat co-ords for the area I want to use from my DEM but not sure how to get an exact size - which I think is necessary for Visitor.

Next i obviously want to export it as an xyz file, so my second question is what settings I need in the export process to make the xyz suitable for Visitor.

Again I've played around with the software and have a vague idea of what to do, but don't know what settings I need to make it visitor compatible, I've tried a few different combos but no luck so far.

When I try to import the xyzs I create with MicroDEM the width and height are not what expected (don't know how to get exact measurements) and my 'square size' is 1.0 instead of 10. (so atm result size = 4096 sqm)

Any help or pointing me towards where I might find a solution would be greatly appreciated.

Thanks! DC

[bushlurker 46]

Hi DC39!

Eeeeh..... tricky question!

MicroDEM is a great little program which does all sorts of good things usually only found in the very expensive programs like Global Mapper, but it's not the most obvious program in the world to use at first...

Worse still, I don't actually have it installed right now, nor have I used it in a while, so I'll have to work from memory.....

Hmmm...

OK - first you want to crop - MicroDEM does call this "subsetting" as you've noticed...

There's a couple of ways you can do this... Easiest is just to use the crop tool (probably called the "subset tool" though... it's a wee sort of dotted rectangle icon in the menu bar... just drag a rectangle from Top Left to Bottom Right to define the area you want...

You have coordinates though? That's handy...

There's another option - under the menu item "Modify" - Something like "Map Extents or Areas" and then the "subset corners" or "something corners" option...

You should be able to enter coordinates for that Top Left and Bottom Right pair and it'll come to the same thing - you'll have defined a square...

*Remember Visitor only likes square heightmaps!!

Assuming you've defined the actual area you want successfully - export it as a file.....

Don't worry about scale or cell sizes, or even .xyz format at this stage..... MicroDEM outputs a fair number of different formats - none of which I can remember :(

Look for .BT (Binary Terrain) - use that one if it's there... if not, maybe .HFZ or .HF2? .TIF (Geotiff), if necessary, I guess...

Next... go HERE and grab a Free copy of L3DT Standard Edition.... this is the best free heightmap editor around, and you'll probably want to come back and play with it again sometime, but right now we'll just use it as a file converter...

Load your saved heightfield - it should display as a greyscale image.... Now go to the top menu...

"Operations/Heightfield/Change Horizontal Scale" - set this to 7.50 meters (this is your "cellsize")

Then...

"Operations/Heightfield/Resize Heightfield" - set this to 2048

That should produce a 2048x2048x7.5m heightfield - Same size as Chernarus & Takistan... approx 15.3km x 15.3km

Now go to "File\Export\Export Active Map Layer" and export the heightmap as .xyz

Assuming I've remembered the MicroDEM bit OK, that should get you started at least..... ;)

B

[lecholas 2]

Hi DC39!

Eeeeh..... tricky question!

MicroDEM is a great little program which does all sorts of good things usually only found in the very expensive programs like Global Mapper, but it's not the most obvious program in the world to use at first...

Worse still, I don't actually have it installed right now, nor have I used it in a while, so I'll have to work from memory.....

Hmmm...

OK - first you want to crop - MicroDEM does call this "subsetting" as you've noticed...

There's a couple of ways you can do this... Easiest is just to use the crop tool (probably called the "subset tool" though... it's a wee sort of dotted rectangle icon in the menu bar... just drag a rectangle from Top Left to Bottom Right to define the area you want...

You have coordinates though? That's handy...

There's another option - under the menu item "Modify" - Something like "Map Extents or Areas" and then the "subset corners" or "something corners" option...

You should be able to enter coordinates for that Top Left and Bottom Right pair and it'll come to the same thing - you'll have defined a square...

*Remember Visitor only likes square heightmaps!!

Assuming you've defined the actual area you want successfully - export it as a file.....

Don't worry about scale or cell sizes, or even .xyz format at this stage..... MicroDEM outputs a fair number of different formats - none of which I can remember :(

Look for .BT (Binary Terrain) - use that one if it's there... if not, maybe .HFZ or .HF2? .TIF (Geotiff), if necessary, I guess...

Next... go HERE and grab a Free copy of L3DT Standard Edition.... this is the best free heightmap editor around, and you'll probably want to come back and play with it again sometime, but right now we'll just use it as a file converter...

Load your saved heightfield - it should display as a greyscale image.... Now go to the top menu...

"Operations/Heightfield/Change Horizontal Scale" - set this to 7.50 meters (this is your "cellsize")

Then...

"Operations/Heightfield/Resize Heightfield" - set this to 2048

That should produce a 2048x2048x7.5m heightfield - Same size as Chernarus & Takistan... approx 15.3km x 15.3km

Now go to "File\Export\Export Active Map Layer" and export the heightmap as .xyz

Assuming I've remembered the MicroDEM bit OK, that should get you started at least..... ;)

B

I don't want to overcomplicate things, but...

Aren't Aster DEMs in an unprojected geographic coordinate system (WGS84)? If yes, wouldn't that mean that cutting a square raster (ie. with even pixel size edges, eg. 2048x2048) from it would result in a representation of non-square-like area?

Edit: Ok, I've glanced through the ZGM's tutorial and looks like it's covered there for Global Mapper. One thing to remember when you use Microdem instead of Global Mapper is to define a projection in Microdem as well (I suppose it allows for that - if not, I'd suggest using for example opensource QGIS).

Edited March 2, 2012 by lecholas

[DC39 1]

Bushlurker:

So I found the corners bit you were talking about, in my version of MicroDEM; 'Modify' - 'Map Area' - 'Keyboard Corners'

and then when I got to export the only option I can find that you listed is .TIF - options are save as image - (colour or grayscale geotiff) & save as DEM - ASCII XYZ, DTED, USGS ASCII, BIL, ASCII Arc Grid, Grid Float, and then a list of options under 'Caveat Emptor', so I've got as far as exporting my Height map DEM as a smaller Geotiff image.

I think I need to look through the l3dt instructions a bit more because there is a baffling array of options for projects etc which look very interesting - I'll get back to you properly when I've worked out how to convert, as at the moment it says the area I want to use as a height map is too big for the free version.

Thanks for the advice, I think using L3DT should make things easier, I'll give a full description of what I did when I get it done right so others searching the forums will have a little guide.

Lecholas:

Not sure quite what you mean by 'define a projection' but I've downloaded the Opensource Quantum GIS and I'm having a look at that next, thanks/ :)

Cheers guys, get back to you in a couple days with final thoughts, sorry I took so long to reply - hectic work schedule this month :(

DC

[rocket 9]

Have you tried Global Mapper? I think they have a trial version. I used it in conjuction with L3DT on several occasions. Also, Aaron at L3DT is a great guy and very easy to deal with if you have questions. This makes exporting in different projections (to grayscale and XYZ) extremely easy straight to Visitor 3. It might be worth a shot, I found it much easier to use than MicroDEM and used it at my old work alot.

[DC39 1]

Hi Rocket, yes I have tried Global Mapper, it's the program advised in ZGM's tut so it was my first stop - trouble is he uses an older version than I could download, and AFAIK the new version wont let you save or export ANYTHING, quite annoying really, which is why I've been trying out other soft. like microDEM. Maybe you know a way around this using L3DT? I haven't really had a chance to use either yet, I'm VERY new to the whole mapping thing, still getting to grips with how the directories and tools work.

[rocket 9]

You can export grayscale heightmaps using MicroDEM (I think as a GeoTIFF, but its been a while since I used MicroDEM). Have you tried importing into Visitor with a grayscale map? I wouldn't have a pipeline direct from MicroDEM to Visitor as the image will need some cropping, levels editing, etc... to get it correct. It will probably require some cleanup also.

A second option is to create the terrain in L3DT and then export as a grayscale image, couple with the correct details in a pbl file, and then import directly into Visitor that way. To achieve the exact effects I wanted, I once was importing/exporting between Global Mapper, L3DT, photoshop, back to L3DT, then into visitor (for my Aoraki map). As always I think the easiest thing is to start simple, then make small steps. See if you can get a very simple, small, grayscale heightmap imported into visitor no problems. Then try something a little bigger once you've got a good feel for the process.

[mondkalb 1087]

Creating_a_Heightmap with L3DT and saving for V3

[lecholas 2]

Creating_a_Heightmap with L3DT and saving for V3

Well, that's a way for creating a heightmap (and what is more, for randomly creating it). I think that the thing that DC39 wants to do is to use an existing real-world data from ASTER DEM.

Lecholas:

Not sure quite what you mean by 'define a projection' but I've downloaded the Opensource Quantum GIS and I'm having a look at that next, thanks/ :)

Cheers guys, get back to you in a couple days with final thoughts, sorry I took so long to reply - hectic work schedule this month :(

DC

A projection is a cartographic method for unfolding or unwrapping an ellipsoidal approximation of Earth's surface into a flat plane. There are many ellipsoids (one of which, the most common thanks to it being used by a GPS, is WGS84) and for every elipsoid there are many projections, global or local (eg. UTM zones, older soviet pulkovo1942 and many, many national projections (or 'grids') as for example Polish PUWG1992. The thing here is to remember that geographic degrees are not flat plane coordinates. They must be projected onto a plane in one way or another. For most Arma purposes I'd assume that UTM projection with a correct zone would be a good choice. Unless you're making a map based on a historical scanned map which is in a different coordinate system (or you have other reasons) as I am at the moment.

I'm personally using QGIS combined with other software (GIMP, L3DT, Willbur). I'm using my hand-made DEM (from contour lines from a historical map), but if I were to use an ASTER DEM my workflow would be as follows:

- download and install QGIS; open it and go to Plugins->Download more Plugins (the names of menu items here and below can differ as I'm using a localized Polish verision of QGIS) and find and install 'Rectangles, ovals digitizing' plugin

- download a for the desired area DEM (I assume you already know how to do it)

- load it into QGIS

- go to Settings->Project settings and tick 'project on the fly' and change the projection to the one which is suitable for you (eg. UTM with the correct zone)

- convert the DEM layer into proper raster (e.g. GEOTiff), remember to tick the option named something like 'Add to layers after finished' (not sure if that's required)

- reproject the resulting raster to your desired spatial reference system (i.e. projection, eg. UTM), Raster->Change Projection; settings: source srs: WGS84, target srs: UTM with your zone, for your purposes 'Nearest neighbour' method of resampling should be good but you may try the remaining ones; again check 'Add to layers after finished'

- create a new shapefile: Layer->Create new layer->Shapefile; make sure it's polygon-shapefile and that the srs (projection) is set to your desired projection (UTM); don't worry about adding any attributes apart from the default FID

- create a new polygon in your new shapefile: select the layer, click 'Edit mode' (a button with a pen or something), then 'Add polygon', and then, using 'Rectangles, ovals digitizing' plugin draw a 'Square from the middle' (AFAIK there is no, more suitable for us, 'Square from the edge' option in the plugin yet) that will cover your desired area (if you're planning for example a 10240x10240 meters map the side of the square should be 10240m - could be tricky with QGIS, I took a shortcut and created the square shapefile in ArcGIS to be sure its side is as close to 10240m as you can get; in QGIS you can for example use the measuring tools, or maybe there are some plugins that I'm not aware of which serve that purpose; or find some other tool just for a purpose of creating a shapefile with a square polygon of the desired dimensions and importing it into QGIS - on thing to remember that such a shapefile should be natively in the target srs, in our case UTM). This will be our cake-cutter layer for cutting all the different layers we ever need (e.g. Sat mask; if you could acquire a land cover layer for the area you could then cut it with the cake-cutter and get a mask for Sheazan74's brilaint World Tools for generation of randomly placed vegetation that way); stop editing (the pen button) and save changes to the layer

- then, at last, clip your DEM raster with the cake-cutter layer: Raster->Clip raster; select the name of your source projected DEM raster and a name and destination for the clipped raster

You should receive a greyscale DEM raster corresponding to the desired area. You can resample it to the desired resolution (eg. 1024x1024) in any program for digital graphics (eg. IrfanView) and you should be able to load it either in visitor or in L3DT. One thing you need to check is to check the min and max pixel values which should correspond to minimal and maximal heights in the covered area and which you should put into right places in Visitor (pbl file) or L3DT - the form when importing the heightmap. But that's a different story.

I may be not precise in many points but I don't have time now for writing a comprehensive tutorial. Maybe some time later if there is some interest. It's not the prettiest method in the world but it works. If you have any questions feel free to PM me and I'll answer if I can.

[Homer Johnston 0]

I haven't fiddled with terrains in years, but back in ancient times I always used 3DEM to convert a GeoTIFF to a terragen .ter, and then loaded that into Wilbur & converted again to a 16 bit png. Took me a while to figure out how to do it, but it always worked just fine for me ;) can't recall how I cropped an area, though.

[memphisbelle 99]

Creating_a_Heightmap with L3DT and saving for V3

Mondkalb, your tutorial is quite well and covers all questions of terrain making newcomers. But by using L3DT by the way you suggest, you end up in always create a new hightmap becuase the one before dont fit your wishes.

So I highly suggest to go through this tutorial. This is didicated to L3DT too, but it explains the use of the Blank designable map, by that one can use ALL possibilities of L3DT in conjunction with the own ideas.

[vasf_jeffs 0]

There isn't really anything new here, but as I've just gotten back into fiddling with terrain I thought I'd share the steps I've used for getting it into visitor. You don't need L3DT, but I'd highly suggest it. If I'm doing something redundant please let me know so others won't have to suffer the same fate.

Assuming you have your project folder, name space, tools installed, etc, etc and are looking for dem data... here's what I've done.

1. I downloaded the dem data from the seamless server (geotiff).

2. Searched all the forums, read a bunch of tutorials and lost some hair…

3. Downloaded / Installed Wilbur

4. Downloaded / Installed microdem (note I didn’t use it)

5. Downloaded / Installed 3DEM

6. Downloaded / Installed L3DT

Getting the dem into game

a. Open dem 3DEM

1. Optional? Operation -> Change Project -> Convert to UTM

(I did this as the dem looks “stretched†after applying this it looks more “normalâ€. I’m not a GIS guy but I imagine there is a reason for this (curvature of the earth?)

2. Operation -> select smaller area -> Click and drag area (make sure it will be bigger than your actual map size) OR Calculate the size of the map based on meters and extract the exact size by altering the Northern or Southern Edge and the Western or Eastern Edge by subtracting (or adding the number of meters needed for the exact map size you will be using (i.e. 5120 , 10240, 20480. If you do this you can skip the crop in Wilbur.

3. File -> Save Terragen Terrain -> Entire Terrain

B. Open Wilbur

1. File -> Open -> (choose Terragen Surface (*.ter) -> select file you saved from 3DEM. The file should load at this point. But it isn’t the right size for your map.

2. Choose select (the dashed box on the tool bar) -> style -> Fixed Size -> then enter in the map size (i.e. 512x512 or 1024x1024, etc) click on the map so the Area you want is in the selection box.

3. Surface -> crop to selection

4. Edit what you want here (i.e. water heights) or do it elsewhere. (I like L3DT)

5. Write Down MIN and MAX Height (Surface -> Find Min Max)

6. Save As PNG Surface

C. Put the png in your working folder (name it terrain.png) and make sure the terrain.pbl contains the min and max info you wrote down earlier.

D. Open visitor and terrain -> import from picture and select the terrain.pbl and it should import just fine.

I have noted that if I open a file in wilbur after it has been edited the height fields get screwed up, so one I have the min and max I don’t put it back in Wilbur.

[NeedMicrodemHelp 10]

Hello I'm new here I'm having a problem merging four USGS topo DEM quadrangles of San Francisco using MicroDEM (it appears Admin of this Vbulletin forum has disabled attachments so I can't upload error screenshot or post Imgur link until I'm a member for one day..) I keep getting error message my (free) install of MicroDEM missing some ".dbf" file, and then will only open window for "output merge" folder location, not open a 'select merge files' window to click on the 4 .dem files to merge them. I've tried uninstalling/re-installing MicroDEM multiple times to no avail getting same error/result.

Anyone out there that has a properly working version of MicroDEM that has successfully merged DEM files (..or other DEM merging software that'll do the trick), and then save merged file as single ASCII XYZ file and email it to me? (not sure how big it would be - I have gmail file limit of 25 MB, but can also do cloud upload/download if need be). I need a single ASCII XYZ to supply a CNC model company so owner can open in AutoCAD and create 3D model for me for a project.

Help is greatly appreciated thank you in advance!!! :D

Edited March 16, 2014 by NeedMicrodemHelp

[person915 11]

Hello I'm new here I'm having a problem merging four USGS topo DEM quadrangles of San Francisco using MicroDEM (it appears Admin of this Vbulletin forum has disabled attachments so I can't upload error screenshot or post Imgur link until I'm a member for one day..) I keep getting error message my (free) install of MicroDEM missing some ".dbf" file, and then will only open window for "output merge" folder location, not open a 'select merge files' window to click on the 4 .dem files to merge them. I've tried uninstalling/re-installing MicroDEM multiple times to no avail getting same error/result.

Anyone out there that has a properly working version of MicroDEM that has successfully merged DEM files (..or other DEM merging software that'll do the trick), and then save merged file as single ASCII XYZ file and email it to me? (not sure how big it would be - I have gmail file limit of 25 MB, but can also do cloud upload/download if need be). I need a single ASCII XYZ to supply a CNC model company so owner can open in AutoCAD and create 3D model for me for a project.

Help is greatly appreciated thank you in advance!!! :D

I have a working version; here's what you should do:

When a program gives you an error like it's missing a file, just let it happen. For a program developed by and for such smart people, MicroDEM is EXTREMELY intuitive and is a very good program, so when it knows what it wants and it wants something, then most likely it will ask for it and you should let it do whatever it wants to do to get it, like downloading the file, for instance. If it's not wanting to do that then... well... I don't know what to tell you. All I know is that there were some extra files I needed to download after install.

I still haven't managed to get an .xyz out of it, but thank you so much for digging up this thread! It deals with exactly what I am trying to do but couldn't figure out. Saved me from making a new thread. Anyway, good luck to you.