rossoe

Having posted a previous tutorial regarding working with Terra Incognita and QGIS to manage satellite imagery ready for use in Terrain Builder - I have since discovered that due to an update to GDAL (2.4.0) one of the key steps in that tutorial using 'Clip Raster by Mask Layer' no longer works perfectly! So I reworked the tutorial by getting a bit more under the hood and using GDAL command line where possible, this also simplifies the process a little and reduced even further processing on the data. This process has often been described using Global Mapper, but I wanted to detail the equivalent steps within the open source (free) GIS app - QGIS. The goal was to achieve an export satellite image and heightmap with the same or better quality than would be possible with Global Mapper. The following software versions were used for this tutorial: QGIS v3.4.3 Terra Incognita v245 I have referenced Terra Incognita because I felt it was the easiest way to obtain satellite data in the oziexplorer map format - see this tutorial for help using Terra Incognita. Don't worry about getting the perfect square sat image using Terra Incognita, just grab slightly bigger area than you will actually use for this project - QGIS steps below will cut to the perfect square. First an overview of the steps required: For Heightmap: Load heightmap into QGIS (drag and drop) Set QGIS to appropriate CRS for location of your HM Create shapefile & generate perfect square using Advanced Digitizing panel Obtain extents from square feature Run 2 GDAL commands First to set cell size, CRS and then clip to shapefile square extents Second to convert to .asc file For Satellite image For single tile export Load in just the '.map' files generated from Terra Incognita into QGIS Merge rasters Obtain extents from square feature gdalwarp command will set cell size, CRS and then clip to shapefile square extents For 4 tile export (large projects 40960 size and above) Load in just the '.map' files generated from Terra Incognita into QGIS Merge Rasters Create grid – 2 x 2 Feature (using square generated above - heightmap step) Move each quarter to its own shapefile layer Obtain extents from each quarter feature Run gdalwarp command for each quarter to set cell size and CRS and clip to shapefile square extents And now detailed steps for: Heightmap This tutorial covers single raster heightmap as per the kind you will find on opentopo here, I will detail steps for heightmaps spread across muliptle rasters in another update. as they will require additional steps likely including Merge / Build Virtual Raster commands. This section does not cover any re scaling of heightmap, which I would recommend against anyway. Remember Grid size x Cell size should = the same size (meters) as your source heightmap selection. Drag and drop your heightmap '.asc' into QGIS If you do use opentopo for source of Heightmap, make sure extract both the .asc and .prj files before proceeding with below. They should both be in the zip file from opentopo. Set QGIS to appropriate CRS for location of your HM Clicking on the CRS section bottom right within QGIS – will bring up the Project Properties for CRS Select the CRS to match your real world data, in this example its ‘WGS 84 UTM zone 20N’ because the terrain is from the Montserrat in the Caribbean. See also UTM projection. This will ensure your QGIS project space works appropriately with any other data you want to add - eg road shapefiles etc. Terrain Builder will only use UTM 31N but we'll get to that later. Create shapefile & generate square feature using Advanced Digitizing panel Top menu – Layer > Create Layer > New Shapefile Layer Save to project folder location Change Geometry type to ‘Polygon’ Match CRS so it is the same as that just configured in above step Right click on your shapefile in layers window & select ‘Toggle Editing’ From menu – View → Toolbars → Advanced Digitizing Toolbar Select ‘Add Polygon Feature’ Select Enable advanced digitizing tools To create a perfect square, left mouse click for your top left starting point, move your mouse to the right a little, then press ‘d’ on keyboard, type in the exact width (distance) required (in this example ‘20480’) then immediately press Enter – be careful not to move the mouse before pressing Enter or it will mess up the number. Press left mouse click and you will have drawn your first horizontal line which turns red. By default this tool snaps to 90 degree angles, making it easy to draw your lines. Now to draw the vertical line start moving mouse your down, press ‘d’ again and type in same figure as above, press Enter and another left mouse click, repeat the process for the last 2 lines of the square. See a short video example using the Advanced Digitizing tool: After the last left mouse click from the step above, the tool is still waiting to plot more points, so to finalise the square do a right mouse click This will then bring up an attributes dialogue – just type any number and Enter Save the above edits to the shapefile by clicking on the layer & selecting ‘Toggle Editing’ Obtain extents from square feature Activate the toolbox if it is not already available Start typing 'vector' in Processing Toolbox search bar, and you will see 'Vector information' Double click it and make sure your square input layer is selected - then just click Run You will only need to copy the Extent figures, which we will use within the GDAL commands in the next step Run the first GDAL command which will - set CRS, cell size and then clip to shapefile square extents From menu - Plugins > Python Console Click 'Show Editor' The blank window on the right is where you will paste GDAL commands Now you are ready to edit the command below to match your data - sections in bold are the parts to edit: import os os.system(r'''gdalwarp -t_srs EPSG:32620 -wo SOURCE_EXTRA=1000 -tr 5.0 5.0 -srcnodata "-9999" -r cubic -ot Float32 -of GTiff -te 576787.687480 1841104.815839 597267.687480 1861584.815839 D:/Arma/Heightmaps/Opentopo/output_srtm.asc D:/Arma/QGIS/Montserrat/converted.tif''') Some explanation of the key parameters within above command: -t_srs EPSG:32620 CRS to match where the heightmap is from in the world. -tr 5.0 5.0 The desired resolution of your heightmap, match with Cell size set within your Mapframe properties in Terrain Builder: -te 576787.687480 1841104.815839 597267.687480 1861584.815839 The 4 long figures represent the extents of your square, and should be replaced with your own figures obtained in previous step. Input path: D:/Arma/Heightmaps/Opentopo/output_srtm.asc Output path: D:/Arma/QGIS/Montserrat/converted.tif Update the paths to match where you've stored your source heightmap, and where you want the output tif saved to. You will have to wrap your paths with quotation marks "..path.." if it contains any blanks (spaces), or just remove blanks entirely to avoid the confusion. Run second GDAL command - converting tif from above step to a Terrain Builder friendly .asc Remember to change the paths in below command to match where your files are located import os os.system(r'''gdal_translate -of AAIGrid D:/Arma/QGIS/Montserrat/converted.tif D:/Arma/QGIS/Montserrat/final.asc''') Satellite image For single tile export Load in just the .map files into QGIS Merge Rasters Top Menu - Raster → Miscellaneous → Merge Select all your .map files as input layers change Output data type to ‘Byte’ as the default is ‘Float32’. Select ‘Save to File’ under Merged – as the default is to a temporary file. Obtain extents from square feature You should already have this from working on heightmap above 576787.687480 1841104.815839 597267.687480 1861584.815839 Run GDAL command import os os.system(r'''gdalwarp -t_srs EPSG:32620 -wo SOURCE_EXTRA=1000 -tr 1.0 1.0 -r cubic -of BMP -te 576787.687480 1841104.815839 597267.687480 1861584.815839 D:/Arma/QGIS/Montserrat/merged.tif D:/Arma/QGIS/Montserrat/mont.bmp''') Some explanation of the key parameters: -t_srs EPSG:32620 Sets the CRS -tr 1.0 1.0 Sets the resolution of your satellite image to 1mtr per pixel - which is recommended. Make sure it matches your settings within Terrain Builder. -te 576787.687480 1841104.815839 597267.687480 1861584.815839 Clips to shapefile square extents For 4 tile export Detailing additional steps required for a 4 tile export – which would suit those terrains that are 40960px x 40960px and higher. Load in just the .map files into QGIS Merge rasters - same as above (single tile export) Create grid – 2 x 2 Feature You will need to have created the square shapefile first – see above steps Top Menu – View → Panels → Processing Toolbox Processing Toolbox Menu – Vector Creation > Create Grid Set Grid type to ‘Rectangle (polygon)’ & the Horizontal & Vertical to half the distance of your main square Set it to save new grid feature to .shp file After clicking 'Use Layer Extent' Select the shapefile layer for your square After clicking Run, your original square will then be Covered with a new grid of 4 tiles – perfect quarters of your original square Move each quarter to its own shapefile layer Because you will need to export each quarter separately for use in Terrain Builder, we will move each quarter onto its own shapefile layer. Click Select Features - And select the first quarter: Top Menu – Edit > Copy Features Then… Edit → Paste Features As → New Vector Layer Save shapefile layer to something obvious I’ve used ‘TL’ for top left. Repeat for the remaining quarters and you’ll have something like below, ‘Grid’ can be deleted. You will then have 4 tiles which we can use to gather each quarters extents Obtain extents from each quarter feature Copy extents to clipboard And list them out for safe keeping TL - Extent: (576787.687480, 1851344.815839) - (587027.687480, 1861584.815839) TR - Extent: (587027.687480, 1851344.815839) - (597267.687480, 1861584.815839) BL - Extent: (576787.687480, 1841104.815839) - (587027.687480, 1851344.815839) BR - Extent: (587027.687480, 1841104.815839) - (597267.687480, 1851344.815839) Run GDAL command for each quarter to set cell size and CRS and clip to shapefile square extents import os os.system(r'''gdalwarp -t_srs EPSG:32620 -wo SOURCE_EXTRA=1000 -tr 1.0 1.0 -r cubic -of BMP -te 576787.687480 1841104.815839 597267.687480 1861584.815839 D:/Arma/QGIS/Montserrat/merged.tif D:/Arma/QGIS/Montserrat/TR.bmp''') Loading your assets into Terrain Builder I would recommend saving copies of any files to be used in Terrain Builder into a separate terrain project folder. This will prevent your source project files being adversely affected, and potentially no longer working within QGIS - particularly relevant to those changes mentioned below. Heightmap (.asc) edits Open your asc in notepad and change the following to ensure it loads in Terrain builder: ncols 4096 nrows 4096 xllcorner 576787.687479999964 yllcorner 1841104.815839000046 cellsize 5.000000000000 NODATA_value -9999 ncols 4096 nrows 4096 xllcorner 200000 yllcorner 0 cellsize 5.000000000000 NODATA_value -9999.0 Save to a new asc, whilst maintaining the original for use in your QGIS project space Before loading the asc file into Terrain Builder, delete the .prj file. Otherwise it will not import into TB. Satellite image Before loading the satellite image into Terrrain Builder delete the .bmp.aux file Managing the project assets within QGIS Save your QGIS project so that you can return to export further data at a later date. Layers can be tidied up to hold the essentials ready for the next time you want to re-run GDAL commands, or export anything else like road shapefiles or mask layers etc. Road shapefile creation Load into QGIS (drag and drop) road shapefile In this example I got a good dataset from - https://data.humdata.org/ Clip vector to square shape Top Menu - Vector > Geoprocessing Tools > Clip Save features from clip to new shapefile - selecting correct CRS UTM-20 Top Menu - Layer > Save As Set road shapefile to CRS - 31N RMB on road shapefile > Set CRS > Set Layer CRS Set QGIS project CRS to UTM- 31N (bottom right corner): Collect extents from your square shapefile: Extent: (576787.687480, 1841104.815839) - (597267.687480, 1861584.815839) The first two values will be used in the v.transform step below. v.transform on road shapefile using calculated extents from square shapefile Your heightmap asc must adhere to Terrain Builders required values of easting 200000 and northing 0 Your road shapefile also needs to line up to the same values - we will use v.transform to achieve this So taking the extent values we grabbed in previous step - We will do the following calculations to bring these values to (200000 and 0): Easting: 576787.687480 - 376787.68748 = 200000 Northing: 1841104.815839 - 1841104.815839 = 0 -376787.68748 -1841104.815839 Having figured out the correct figures plug them into the v.transform process Drag and Drop into QGIS your heightmap asc - the one you are able to load into Terrain builder - already adjusted to (200000, 0) Set CRS on above asc to UTM - 31N Right click on asc in Layers > Zoom to Layer In Layers panel drag the asc below your transformed road shapefile - so you can see the roads layered above the asc Your roads should now be perfectly aligned above your heightmap Setting ID and ORDER fields RMB on transformed shapefile layer > Toggle Editing RMB on transformed shapefile layer > Open Attribute Table Click Delete field - select all fields and press OK New Field - Name 'ID' - length 0 New Field - Name 'ORDER' - length 0 Toggle - Multi Edit mode ID - Enter 0 - click Update All Select ORDER from drop down - Enter 1 - click Update all Click - Switch to table view and check that ID column all show as '0' and ORDER all show as '1' Click Save RMB on transformed road shapefile - Toggle Editing to save the changes Switching to Terrain Builder - Load your road shapefile Top menu - File > Import > Shapes Click OK Perfectly overlaid within Terrain builder: 3D Preview using Qgis2threejs If you want to see an instant 3d preview of your terrain within QGIS you can with the amazing plugin Qgis2threejs First install it: Top menu > Plugins > Manage and Install Plugins RMB on the Merged layer and select Zoom to Layer Then select both the Merged raster, and the original source heightmap - deselect all other layers Start the Qgis2threejs plugin - Top Menu > Web > Qgis2threejs > Qgis2threejs Exporter Select source heightmap and nothing else RMB on heightmap > Properties Set the Resampling level to 6 Set Resolution to 400% Enjoy browsing around your terrain in 3D: You can also export your terrain to a browser interface if you want - File > Export to web Coming Soon: - Assisted image classification - for creation of mask Above tutorial is also available on Github - https://github.com/rossoe/Arma3_QGIS

+3

I would recommend getting in the Arma Discord server and asking the man himself - he may reply and help re his own tools. These 'external tools' are rather special with the attention to detail that's gone into them. I don't think you need to re-export the WRP for roads, just need to binarise Map data. Here's an idea of the things you need to do following certain changes:

without Mikero's tool's there are no Arma terrains I had to install a few of them twice over, and in certain order to eventually get them to play together.

you can tweak lots config settings to minimize performance hit, only way to tell is by giving it a go - it's on Steam so easy to add into the mix!

AI driving is vastly improved with the latest VCOM AI release. It has a totally separate driving module, so you can just turn that on if desired. and it has a great debug mode where you can see how it's navigating around obstructions. Good video of the developer working on mod

Hi LoRo, There is another guide here for Road shapefile work - https://forums.bohemia.net/forums/topic/221322-qgis-real-world-data-tutorial/ towards the end.

Updated with guide on 3D Preview within QGIS using Qgis2threejs plugin

Updated with road shapefile section - who needs Global Mapper!

Try using this tool - May help identify what is going on.

I would highly recommend trying v3.2 of VCOM found here - https://github.com/genesis92x/VcomAI-3.0/branches it's got a lot of nice fixes, and also brings back the very clever driving improvements that were previously a separate mod. It won't be too long until it's released to published version on steam.

The BI dev corrected himself "I meant just for the scope of Warlords, like purchasing transport vehicles, aircraft etc. :)"

If you look at the Github for this mod you will see that it's far from abandoned - https://github.com/genesis92x/VcomAI-3.0 Genesis showed his work on updates for the coding on this mod only last Friday -

Not sure but seems to work ok on one of my maps - messes with distant roads a little, but alot nicer than stock http://www.armaholic.com/page.php?id=19525

Try 'BaBe midTex' mod

Better late than never! sorry SCAR didn't spot your query. Re the CRS - see below steps I only changed the road shapefile CRS, as my heightmap was already on UTM zone 31N (coming from Terrain Builder). Also in this instance both my road shapefile and heightmap were originally exported from a Global Mapper project to the exact size required for a terrain in Terrain Builder. This made it easier to check original easting & northing coords before they are switched to 200 000, 0 which Terrain Builder requires. Steps I took using Affine Transformation tool in QGIS 2.18 - Load in currently used heightmap (asc) from your Terrain Builder project: When skirting the mouse near the bottom left corner, figures should be close to - 200 000 easting and 0 northing coords. And if it's from Terrain Builder it should be CRS - 'UTM zone 31N' Now add in road your road shapefile Depending on the source of shapefile, It'll possibly be on different CRS to your heightmap: Right click the road layer - 'Set Layer CRS' Switch it to match Heightmap - 'UTM zone 31N' The shapefiles original easting & northing coords (exported from Global Mapper) would have been: xllcorner 174068.079 yllcorner 15603007.959 Which lines up with a rough check above on the bottom left of the road shapefile We still need to get it to 200000, 0 so that the roads line up over the heightmap. So if I do the calculation for xllcorner - “200000 - 174068.079” equals 25931.921 value. And for yllcorner we need to get it to 0 so will simply use -15603007.959 Enter the above figures into Affine Transformation tool - Which brings the road shapefile back in line with the figures you want 200 000, 0 ready to use in Terrain Builder I'll soon detail the steps to do everything in QGIS rather than jumping between Global Mapper and QGIS like above.

I wonder if someone could help me figure out how to get a Shapefile converted - as I'd love to start working on roads on my terrain. I've been following: http://pmc.editing.wiki/doku.php?id=arma3:terrain:road-shapefiles-real-world-data and also: http://pmc.editing.wiki/doku.php?id=arma3:terrain:qgis-reproject-transform-tutorial and I'm constantly getting stumped at the v.transform stage in QGIS. I know the shapefile produced during the Global Mapper stage of the tutorial works ok as I tested throwing it in TB: But of course I have to carry out the transform step so that it matches up with the mapframe. (I'm using v18 of GM so following the QGIS steps) I load into QGIS my heightmap asc exported from TB which has mapframe properties: xllcorner 200000.000000 yllcorner 0.000000 Then load my Shapefile I apply WGS 84 / UTM Zone 31N to both layers I know the coordinates from an asc header exported from Global mapper with user created features grid are: xllcorner 174068.079 yllcorner 15603007.959 which near enough matches up with above image (pointer was not in exact spot) So I assume the goal is to bring above figures back to - 'easting 200000 and northing 0' So I calculated to return the coordinates to those of arma3 terrain builder I need - X: +25931.921 Y: -15603007.959 Using v.transform I input above figures: I don't do any scale changes as I assume I don' t need it. I click Run & then zoom to new Transformed layer - and this is where I get confused - as I think it's supposed to then line up with my heightmap layer in the view but's it's not -see below The coordinates don't seem to have changed at all? In fact even though I am zooming to the new Transformed layer the view is still actually sitting on iowroad3. it's like the new layer has nothing in it. If I save off the new Transformed layer - it's just 1kb shp file - so does not look like it holds anything? Am I doing something stupid with my transform calculations? or using the wrong asc? it's not clear from tutorials which heightmap asc you should be loading into QGIS, one mentions the asc exported from TB, the other notes grabbing the one from Global Mapper. they both have completely different header coordinates!

Where is the Image 1 example from?

So I tried to spend a bit of time reviewing the process of getting sat images through QGIS and into Terrain Builder Essentially you just load the .map files into QGIS - Terra Incognita will generate tiled images with corresponding .map files, these .map files hold the CRS details within them. You will only need to drag the .map files into QGIS’s central pane. They will automatically link back to the images (jpg’s, tif’s), located where you originally saved them. I did an analysis of 2 different routes to process the rasters within QGIS: The very basic steps after that: For single tile export · Load in just the .map files into QGIS · Merge Rasters · Align raster to 1mtr cell size and re-project to appropriate CRS for its geo location · Change the ‘Project Coordinate Reference System’ (CRS) to match CRS in previous step · Create shapefile layer · Create square Feature using Advanced Digitizing panel · Export sat image ready for Terrain Builder using ‘clip raster by mask layer’ For 4 tile export · Load in just the .map files into QGIS · Merge Rasters · Align raster to 1mtr cell size and re-project to appropriate CRS for its geo location · Change the ‘Project Coordinate Reference System’ (CRS) to match CRS in previous step · Create shapefile layer · Create grid – 2 x 2 Feature · Move each quarter to its own shapefile layer · Export 4 sat images ready for Terrain Builder using ‘clip raster by mask layer’ If you look here you can find the entire process which I documented and stuck in a word file: https://github.com/rossoe/Arma3_QGIS or on PMC wiki: https://pmc.editing.wiki/doku.php?id=arma3:terrain:qgis-real-world-data-tutorial

I posted on the driving mod thread in Dec - But since then I tested the union of the 2 mods after release of Vcom AI 3 - and it seems you now can run them both at the same time and benefit from the separate driving mod - which I can confirm after numerous runs of the same scenario still performs better than Vanilla driving AI skills - particularly in instances when there is a large road block for example model of crashed helicopter, vanilla could not make it around at all, Vcom AI driving got around it no problem! Try and test it yourself - you should see the same results, just put loads of different hazards in a road and waypoint a vehicle to drive through them all.

I'm trying to set the following: VCM_StealVeh = false; //Will the AI steal vehicles. from within the hpp files: \@Vcom AI V3.0\addons\userconfig\VCOM_AI\AISettingsV3.hpp \@Vcom AI V3.0\userconfig\VCOM_AI\AISettingsV3.hpp I can of course do it via CBA in game, but just wondered why it's not possible via settings files?

Ben - I noted above that seperate Driving mod seems to work well with new update.

Amazing work, and it seems to have sorted a issue I had where I wanted to run both Vcom AI and VCOM AI Driving together. Before it seemed to cancel out the great benefits from VCOM AI Driving, but now I can run both woohoo. Tested several times and clever driving around large obstacles like crashed helicopters etc is persisting. and I can now get the extra benefits of your latest Vcom AI to boot. Can you please confirm if you have completely removed driving elements from the v3.0 Vcom AI? only error I'm seeing so far is this: https://imgur.com/a/y1Gjtsx happens when I place a load of soldiers in the way of vehicles route. Thank you for your hard work on this one.

Bit more digging and even though imagemagick is showing 8bit - when I check in irfanview it was still 24bit! https://imgur.com/a/2XZUH so converted to 256 colour in irfanview and it now works in SurfacePainter with 40960 mask tiff Wonderful

hashlych, I've been testing using a 40960 mask image and I can't get Surface Painter to write back to the tif initially my tif was 24bit, but I shifted it to (indexed colour) 8bit in Photoshop and made sure it's lzw compressed - file is 4.28mb Here is a check on the tif prior to testing in SP using ImageMagick: E:\test\8bit\Final>magick identify -monitor -define registry:temporary-path=E:\Windows\Temp -limit memory 16mb -verbose iow_8bit.tif load image[iow_8bit.tif]: 40959 of 40960, 100% complete Image: iow_8bit.tif Format: TIFF (Tagged Image File Format) Mime type: image/tiff Class: DirectClass Geometry: 40960x40960+0+0 Resolution: 72x72 Print size: 568.889x568.889 Units: PixelsPerInch Type: Palette Base type: TrueColor Endianess: LSB Colorspace: sRGB Depth: 8-bit Channel depth: Red: 8-bit Green: 8-bit Blue: 1-bit Transparent color: none Interlace: None Intensity: Undefined Compose: Over Page geometry: 40960x40960+0+0 Dispose: Undefined Iterations: 0 Compression: LZW I'm getting these flags coming up but it never finishes the writing to file? https://imgur.com/a/aOriG Tried on 20480 project and that completes the image write in about 5-8 seconds. Any chance Surface Painter could be made to us a different algorithm to support large images?