[TUTORIAL] A guide to importing roads from real data

[_SCAR 139]

Dear all,

Those of you who have tried importing roads from real data might have experienced the pain. There are multiple tutorials out there, but most of them give bits and pieces and it's hard to understand the exact workflow that needs to be followed.

 

FYI, reference is mainly from these tutorials:

• Terrain Processor
• PMC Editing

 

Software that you'll need:

• Terrain Builder
• Global Mapper
• QGIS
• SAS Planet
• Photoshop / Gimp

 

Some of these software are not free, however there are trial versions that you may consider using. Let's get started.

 

 

SETTING UP TERRAIN BUILDER & EXPORTING THE REFERENCE HEIGHT MAP

 

Fire up Terrain Builder. Click on "Mapframes > Add mapframe":

 

 

Leave the default Zone 31N selected and click OK:

 

 

Set the name of the Mapframe, the output root folder (in your P drive), Easting to 200000 and Northing to 0:

 

 

Click on the "Samplers tab" and choose your map size. I've set everything to have a map size of 10,240Km:

 

 

Click on the "Processing" tab:

 

 

Then click on "Rebuild Terrain":

 

 

Confirm, and you will now have an empty terrain layer in Terrain Builder. We are going to export it so that we will be able to use it as a reference, further down in the tutorial:

 

 

We're going to save it as reference-terrain.asc.

 

 

DOWNLOADING ROADS

 

There are multiple ways that you can get the roads shapefiles from Open Street Map. You can go on the https://www.openstreetmap.org/ website and export a specific area if you'd like, but we're going to download the data from http://download.geofabrik.de/

 

Head there, and select the country of your interest. In our case, we're going to need the data from Afghanistan:

 

 

Download the .shp.zip format, and unpack it somewhere on your drive.

 

 

 

SELECTING A PRECISE AREA

 

Open Global Mapper and start a new project by selecting "Load Default Data":

 

 

Click on Find address:

 

 

Enter the name / coordinates of the area you're interested in (in this case Feyzabad, the map I'm currently working on):

 

 

Click to center map on result:

 

Let's now add some visual reference by clicking on the icon to connect to online data:

 

 

Select "World Imagery" and click on "Connect":

 

 

Use the zoom tool to zoom on your area.

 

 

You can now enable the Control Center from the menu:

 

 

So you can enable / disable the layers to view either the world imagery or the countries default data.

 

We are now going to select our precise area of interest. Click on the "Create rectangle/Square area" tool:

 

 

Draw an area (doesn't need to be precise) on the terrain where you want your area to be. In the menu that pops up when you release the mouse button, enter the name of your reference area (I've called it "border"):

 

 

We can now better move the selection to your area of interest. Click on the Digitizer Tool:

 

 

Select your area, then on the move tool:

 

 

This allows you to move your area of interest around:

 

 

Once you're satisfied with the position, we're going to adjust this area to be a perfect square in terms of meters. Right click on the area and select "Vertex Editing > Edit Feature Vertices":

 

 

In the menu that pops up, we're going to change all of the vertices distances so that they reflect our map size, i.e. 10,240m. To do so, click on the first vertex, then on the "Edit Length" button, and enter the value 10240 meters:

 

 

Once you're done, you will have a perfect square of 10,240m x 10,240m (look at the "Length" column):

 

 

Click on "Copy to clipboard" to be able to access these references later:

 

 

These are my coordinates:

Idx,Longitude,Latitude,Length,Total Length,Heading
1,70° 30' 01.1798" E,37° 09' 41.4202" N,10.24 km,---,90.0°
2,70° 36' 56.2072" E,37° 09' 41.4496" N,10.24 km,10.24 km,180.0°
3,70° 36' 56.2072" E,37° 04' 09.2811" N,10.24 km,20.48 km,270.0°
4,70° 30' 01.6831" E,37° 04' 09.3107" N,10.24 km,30.72 km,0.0°
5,70° 30' 01.6831" E,37° 09' 41.4793" N,---,40.96 km,---

Use a converter tool to convert these coordinates in UTM. I used this online converter tool to do so:

http://home.hiwaay.net/~taylorc/toolbox/geography/geoutm.html

 

 

The result of my conversion is to have the following 4 points:

top left
37.0692447463377,70.5003277260268
UTM: 4103606.716118635,633376.3161381673

top right
37.0692447463377,70.6156131229525
UTM: 4103774.7515539424,643625.6939804782

bottom left
37.1615056060246,70.5003277260268
UTM: 4113842.6755561405,633214.540454661

bottom right
37.1615056060246,70.6156131229525
UTM: 4114010.864559334,643451.4784080114

Note that the UTM zone of my area is 42N.

 

Now it's time to import the roads downloaded from Geofabrik. For the purpose of this tutorial (importing roads) we're only interested in the gis.osm_roads_free_1.* files:

 

 

Import the gis.osm_roads_free_1.shp file to Global Mapper (a simple drag & drop will do), and you will see the roads surrounding your selection:

 

 

Now we need to convert the project to use UTM. Click on "Configure":

 

 

In the resulting popup, set the Projection to UTM (the correct zone should already be set for you, if not just use the one that resulted from your computations here above):

 

 

You might see your areas tilting a bit.

 

Now, let's crop the roads to our area of interest. With the Digitizer Tool active, and only the roads and your border layer visible, select your border, and right click to select "CROP - Crop Loaded Features to Selected Area(s)":

 

 

 

Ensure that you consider only lines and that you create a new layer:

 

 

Name your layer:

 

 

Verify that the projection of your new layer is correct:

 

 

and then click on OK.

 

You can now hide the layer with the full roads, and if you visualize only the newly created "roads_clipped" layer and the border you'll see only the roads you're interested in:

We can now export those clipped roads:

 

 

Export them as "shapefile":

 

 

Specify your destination file for your lines (in my case, roads_exported.shp) and that "Generate Projection" is selected.

 

 

 

 

PREPARING ROADS FOR IMPORT

 

Open up QGIS. Open the options:

 

 

Ensure that your projection is set to WGS 84 / UTM zone 31N by default:

 

 

Click on OK and start a new project.

 

Now import (drag & drop will suffice):

• The reference height map that you exported here above from terrain builder (reference-terrain.asc).
• The roads_exported.shp that you exported from Global Mapper.

 

Right click on roads_exported and select "Zoom to Layer":

 

 

You will see:

 

 

Ensure that both layers are set to the proper CRS: right click on both layers, select "Set Layer CRS" and select "WGS 84 / UTM Zone 31N":

 

 

Do this for both layers. If you don't see roads anymore, just right click on the layer, "Zoom to Layer" again.

 

Now it's time that we move the roads to the appropriate place, which is Easting 200000 and Northing 0, as per the data we entered when creating our Mapframe in Terrain Builder.

let's compute how much our roads are off, in terms of UTM:

 

Our top left corner UTM coordinates are (see computations above):

4103606.716118635, 633376.3161381673

We need to make them become:

0, 200000

Hence we need to shift the coordinates:

X shift: 200000 - 633376.3161381673 = -433376,3161381673
Y shift:      0 - 4103606.716118635 = -4103606,716118635

 

Bring up the Processing Toolbox with CTRL+ALT+T, and search for the v.transform algorithm. Once it appears, double-click on it, which will bring up the dialog. Enter the shifting values that we just computed:

 

 

Clic on "Run". This will create a new layer "Transformed". Right-click on this layer, and select "Zoom to Layer". You will see:

 

 

The black square is our reference-terrain level, so the roads are now in the right position for Terrain Builder. Great!

 

However, you can notice that the roads are slightly rotated in comparison to the reference terrain. We can fix that by applying some rotation. I'm going to choose -1 and apply it with v.tramsform again:

 

 

This results in roads being rotated but also shifted southwards:

 

 

Right click on the upper "Transformed" layer and select "Toggle Editing":

 

 

Now use the Move Tool to move your roads so that they are fully contained in the black area (the reference terrain):

 

 

Repeat if necessary.

When you're done, we can export those roads to be used in Terrain Builder. Right-click in the uppermost "Transformed" layer, and select "Save as:"

 

 

Export the layer as ESRI Shapefile (ensure that the proper CRS is set, which should be the default):

 

 

Here I've saved the file as roads_final.shp.

 

We now need to set the appropriates attributes for roads.

 

 

In the resulting excel-like table, toggle editing mode:

 

 

Click to create a new field:

 

 

Create a new numerical field called ID:

 

 

Now we need to populate this field. Click on "Open field calculator":

 

 

and fill the data to look like this:

 

 

The ID field needs to contain the roads reference as specified in your roadslib.cfg file of your terrain project. Basically, we are populating the newly created ID field based on the values contained in the fclass field by using this expression:

CASE
  WHEN fclass is 'primary' THEN 1
  WHEN fclass is 'secondary' THEN 2
  WHEN fclass is 'tertiary' THEN 3
  WHEN fclass is 'residential' THEN 4
  WHEN fclass is 'service' THEN 4
  ELSE 5
END

Click on OK. You ID field will now be properly populated:

 

 

Now let's remove the unneeded fields. Click on the delete field icon:

 

 

Select all the fields except the newly created ID:

 

 

Click OK. Save Edits:

 

 

Quit the Attributes Table and If prompted, save the layer.

 

 

 

PREPARING THE SATELLITE IMAGE

 

Open SAS Planet, and click to enter a selection area by coordinates:

 

 

Enter the coordinates as exported by Global Mapper:

 

 

This will bring up the selection manager. Select "Bing Maps - satellite" (or your preferred satellite source) and then click on the first zoom level that has a resolution higher than your terrain size (in this case, you can see that with zoom level of 18 the resulting image size is greater than 10,240 x 10,240): 

 

 

Click on "Start" to download the tiles:

 

 

Once the process is finished, click to select the last area:

 

 

Move to the "Stitch" tab:

 

 

Ensure to select the same Map source (in this case, "Bing Maps - satellite") and level of zoom (here, 18), select where to save your file and click on "Start". This will save your satellite image as a single JPG file.

 

Open up this image in Photoshop (or other editor of choice) and change the size of the satellite image to match the size of your terrain (you will have to un-tick "Constrain Proportions"), in this case 10240:

 

 

Save your resulting satellite image as a BMP file. Do not save it in other formats, as those might take significantly longer when imported in Terrain Builder (we're talking about hours in comparison to few seconds).

 

 

IMPORTING EVERYTHING BACK IN TERRAIN BUILDER

 

Back in Terrain Builder, import the newly created satellite image:

 

 

Now import your roads (roads_final.shp):

 

 

When prompted, leave unchanged:

 

 

And there you have it. Enjoy your freshly imported roads:

 

 

And if you select them and open up the Database Properties:

 

 

You are going to see the correct road ids:

 

 

 

 

Many thanks to the various tutorial writers that originally helped me, and to W4lly63 for his great support.

 

Best,

_SCAR.

[W4lly63 82]

Great job bro !

[Yuval 29]

Damn the result looks great!