Add Realistic Mist and Fog to Topography in QGIS 3.2

I recently came across a great tutorial by John Nelson in which he demonstrated how to create map of Switzerland in the style of Edward Imhof, the famed Swiss cartographer renowned for his hand painted maps of Switzerland and other mountainous regions of the world. John’s map used traditional hillshading, multidirectional hillshading and crucially, a translucent topographic layer that created a mist like appearance he likened to the sfumato technique used by painters since the Renascence.

I followed John’s tutorial in QGIS 3.2 and I was quite pleased with the initial result below. However, the process creating it is a bit too complicated for a tutorial so I set about simplifying the process and rather than imitating Imhof’s distinct style, my goal this time is realism.

The heart of the effect involves the very clever idea of using the topographic layer as a subtle opacity mask to simulate mist, fog and atmospheric haze. Have a look at the image below taken on March 17th, 2005 by NASA’s Terra satellite. This is the industrialised Po valley of northern Italy, surrounded by the Alps and Apennine Mountains that rise above the valley’s hazy pollution. The haze adds a sense of depth to the surrounding hills and mountains. It’s not uncommon to see fog and pollution in satellite imagery that gives way to the clear air in high mountains e.g. northern India and Nepal, China, Pakistan and India. Creating a similar mist effect in QGIS is actually quite simple.

First download topography for the Alps and Po region (a 68.55 Mb GeoTiff file derived from freely available EU-DEM data I resampled from 25 to 100m resolution). Next, make sure you have the plugin QuickMapServics (QMS) installed (menu Plugins – Manage and Install Plugins). This great plugin provides access to over 1000 basemaps.

Load the GeoTiff file into QGIS (Raster – Load) and rename the layer Hillshade. Right click the layer to open the Layer Properties window. In the Symbology panel, next to Render Type, choose Hillshade. Change the altitude to 35 degrees, Azimuth to 300 degrees and Z Factor of 1.5 (illuminating the landscape from the top left). Finally, change the Blending mode to Multiply. Click OK to close the dialogue.

To add the basemap layer, Esri World Imagery (Clarity), type “ESRI clarity” in the QMS search bar to find and add the basemap; Go to View – Panels and activate the QMS search bar if it isn’t initially visible. Make sure it’s the bottommost layer.

Oh, that’s a bit disappointing, we only increased the relief little a bit. It’s missing the vitally important mist layer.

To create mist, right click the Hillshade layer and choose Duplicate. Rename the new layer Mist and make sure it’s above the Hillshade layer. Now open the Layer Properties window of the layer, we’re going edit it’s attributes to make it look like mist.

Change the Render type to Singleband Pseudocolor and use 0 and 3000 for the min and max values (limiting maximum latitude of the mist to 3000 meters). Then open the colour ramp window by clicking on the Color ramp and enter these values:

• Left Gradient – HSV 215 15 50 and 75% transparency
• Right Gradient – HSV 215 15 50 and 0% transparency

Close the Color Ramp dialogue. In the Layer Properties window, and this is very important, change the Blending mode to Lighten. Click OK to close the Layer Properties window.

Wow, we have mist!

(View Full Size)

The mist effect looks great. It certainly adds a lot of realism to the topographic map, it now looks quite like NASA’s images. This is just a quick and basic map so there’s lots of scope to improve the effect. Play around with the colour of the mist layer and its opacity, or even brighten the Hillshade layer underneath. See what effects these changes have.

Here’s another example below. In this example I duplicated the hillshade layer and set the second hillshade layer to Multidirectional Hillshading (yes, QGIS 3.2 has Multidirectional Hillshading). I then adjusted the transparency of both hillshade layers so they blended together nicely. I then replaced the basemap with another duplicated topography layer that I coloured using the gradient sd-a (by Jim Mossman, 2005) using the cpt-city plugin. And lastly, I doubled the opacity of the mist layer turning it into a milky fog. I think it looks great!

(View Full Size)

What next? Well, there’s lots of possibilities. Perhaps download Martian topography and add mist to the bottom of Valles Marineris?

References:

Eduard Imhof – Biography

TV documentary about Eduard Imhof

The Map as an Artistic Territory: Relief Shading Works and Studies by Eduard Imhof

Haze in northern Italy – NASA Terra Satellite

Tzvetkov, J., 2018. Relief visualization techniques using free and open source GIS tools. Polish Cartographical Review, 50(2), pp.61-71.

Adding ESRI’s World Hillshade layer to QGIS

You may have seen my earlier tutorial where I described how to make nice looking hillshaded maps in QGIS using SRTM elevation data. Well, we don’t have to stop with just one hillshade layer on a map, it is possible to overlay multiple hillshades; a procedure that can increase the visual quality and detail. The following image is the hillshade we made before. Once you re-create a hillshade, following the previous tutorial, you can head to the next step (note that brightness and contrast settings may be different due to changes in how QGIS generates and displays hillshades).

We can improve the SRTM hillshade further by adding ESRI’s World Hillshade layer, which uses multi-directional illumination (also called a Swiss Hillshade in tribute to the celebrated Swiss cartographer Eduard Imhof). In addition, World Hillshade has a much higher resolution than SRTM 30m data in some regions of the world, it is 2m for most of the England and Wales, 10m for most of the US, 5m for Spain and 3m for Holland etc. The only drawback is that the style of this layer is somewhat controversial, some love it, some hate it, it looks like it’s illuminated from above, but mixing it with the SRTM hillshade obviates some of it criticised flaws.

To add the World Hillshade layer in QGIS go to the Layer Menu – Add Layer – Add ArcGIS MapServer Layer – click New and add the following URL:

https://services.arcgisonline.com/arcgis/rest/services/Elevation/World_Hillshade/MapServer

Notice QGIS 2.18 no longer needs a plugin to add ESRI layers, it new has this functionality built in. Also, open the url in a browser such as Firefox, it brings up a webpage that describes the layer. We also see links to other other layers. Yes, they can all be added to QGIS by simply taking the URL of the webpage that describe the layer and connecting to it via the ArcGIS MapServer Layer connector.

Name the layer World Hillshade and click Connect, then click and highlight the layer it connects to. Finally, click the Add button to add the layer to the canvas.

Next, we need to adjust the properties of the World Hillshade layer to properly overlay it above the SRTM hillshade layer. Make sure the World hillshade layer is the topmost layer. In the Layers Panel, right click Layer properties and in the window that opens up, click Style (if not visible). Next, change the Layer Blending mode (under color rendering) to Overlay. Adjust the layer’s brightness to around -20 and leave contrast at 0. If you find the scene is still too dark, brighten the SRTM Hillshade by increasing the layer’s brightness. You may also have to change (lower) the Min value of the Min – Max value boxes. Leave the contrast at 0 for the SRTM hillshade. Also, don’t brighten it too much as it might become washed out, loose detail, especially in bright areas. Play around the controls, settings may vary depending on the SRTM data you download and the version of QGIS you use.

Here’s a comparison in Ireland, a ring like structure of hills with a central peak. No, it’s not a meteorite crater. It’s a different kind of geological marvel, the Slieve Gullion Complex and its ring dyke; the deeply eroded remains of a 410 million year old Caledonian volcano. The SRTM hillshade is on the left and World Hillshade + SRTM hillshade is on the right (click on the image, it’s best appreciated full size):

We can see the World Hillshade + SRTM Hillshade layer shows much finer detail. We see a parallel array of roughly north-south orientated lines, these are fractures and faults that cut the Slieve Gullion Complex that were perhaps enhanced by glacial erosion. Also, look carefully, there seems to be some roads meandering across the landscape (hint, bottom of the map and right of the scale bar). You should get even better results with higher resolution World Hillshade data. We also notice that bending SRTM derived hillshade with World Hillshade adds a naturalistic illumination not apparent in multi-directional hillshading. So we have the best of both worlds, a high resolution hillshade and realistic looking illumination.

Hope you found this tutorial helpful.

References:

Baxter, S., 2008. A Geological Field Guide to Cooley Gullion, Mourne & Slieve Croob [pdf]. Geological Survey of Ireland, Dublin. p. 43-53.

Imhof, E. 1982. Cartographic Relief Presentation. Walter de Gruyter GmbH & Co KG.

Using Trigonometry To Place And Orientate Labels

Geologists display the dip and strike of rock layers on geological maps using a dip and strike symbol, where dip in degrees indicates the maximum angle a rock layer descends relative to the horizontal. However, it is not directly possible in QGIS 2.18, using basic label settings, to place and orient a dip label next to a dip and strike symbol.

However, there is a way around this issue using Trigonometry and editing the layer’s Attribute Table. This method may be useful for controlling the position and orientation of labels around point features in general. The first step involves adding values to the Attribute Table. First, add these two new columns:

• Angle – 0° is North and values increases clockwise up to 359°
• Distance – label distance from a point feature

You can add Angle and Distance values to these columns manually or use the Field Calculator (see below) to add values if you have lots of points. Also, I chose Map Units (not millimeters) for Symbol Size, Font Size and Distance for my map, as I prefered to keep symbol size, font size and position of labels fixed when zooming in and out.

Note – I use Strike (Angle) and Label Distance (Distance)  in my Attribute Table

The next step is to control the position of the label around the points using trigonometry. Right click the points layer and choose:

Properties – Labels – Placement

Check that Offset From Point is checked and then click the Data Defined Override next to the Offset X, Y boxes and choose Edit. The Expression String Builder will appear. Enter the following expression in the Expression String Builder window:

to_string ( ((-1) * ( “Distance” )) * cos ( radians ( “Angle” ))) ||’,’|| to_string (((-1) * ( “Distance” )) * sin ( radians ( “Angle” )) )

The expression takes the angle and distance values from the Attribute Table (edited earlier) and calculates an X, Y label position relative to the point feature. You may also optionally control the angle of a symbol or icon itself via:

Layer Properties – Style – click Data Defined Override icon – Edit

Then enter the following expression in the Data Defined Override dialogue:

“Angle” – 90

Finally, to control the rotation of label text, so text follows the orientation (angle) of a rotating symbol or icon, choose:

Layer Properties – Labels – Placement – Data Defined – Rotation

Click the Data Defined Override Icon again and then choose Edit. Enter the following expression in the Data Defined Override dialogue:

(“Angle” – 90) * -1

The following geological map of the Old Head of Kinsale in southern Ireland shows the results of the above procedure. We see that the dip labels rotate and currently follow the orientation of the dip and strike symbols (note that the points are at the intersection of the T symbol).

Geological Survey of Ireland – Creative Commons Attribution 4.0 license

You may have several different symbols, of various sizes, each requiring an appropriate label distance expressed in the Attribute Table. It took me a few tries before I found the right distances for my geological symbols, from 90 to 230 meters distance depending on the symbol size and type.

Lastly, the expressions “Angle” – 90 and (“Angle” – 90) * -1 were necessary in my case because I needed to place my labels next to the dip and strike symbol’s barb. You may need to use a different expression e.g. “Angle” and (“Angle”) * -1, or a value other than 90° depending on the symbol used and the prefered label placement location. Some trial and error is may be required to find the correct label position.

Creating a Tissot’s Indicatrix in QGIS

The task of projecting, or unfolding the spherical Earth onto a flat map, is an age old problem in cartography. Projection almost always introduces distortion, most projections cannot preserve angles, areas and distances at the same time, they may be conformal (angle-preserving), equal-area (area-preserving) or equidistant (distance preserving) but not all at once. The only exception is a Globe, which preserves angles, areas and distances perfectly. Thus a projection is a compromise.

The choice of projection depends on a map’s use, scale and audience. Conformal projections, for example, are preferred for nautical charts or small scale maps because they locally preserve angles necessary for navigation and survey drawings. Equal-area projections are best suited for maps of broad continental region as they preserve the relative sizes of countries, seas and oceans and allow comparison between regions. Finally, there are hybrid projections that minimise the distortion by merging conformal and equal-area projections, these can be used to create visually pleasing maps of the entire Earth (for a guide to selecting a map projection see Fig. 9 in Jenny (2011), link below).

But how does one measure the degree and type of distortion in a map projection?

One elegant method was developed in the 1880s century by the French cartographer Monsieur Nicolas Auguste Tissot, the Tissot’s Indicatrix (or Tissot’s Ellipse). This mathematical contrivance consists of a grid of infinitely small circles that measures the degree and type of distortion caused by projection. While Monsieur Tissot’s approach is mathematical, involving infinity small circles, his technique can be approximated overlaying a regular grid of large circles and crosses to a map.

The Indicatrix Mapper plugin for QGIS by Ervin Wirth and Peter Kun creates a Tissot’s Indicatrix by adding a vector layer of circles and crosses in a gridded pattern on a map. The degree and type of distortion of the Tissot’s Indicatrix reveals the class of map projection as follows: –

• If a projection is conformal, the area of circles and sizes of the crosses will change while the shapes of circles remain the same and intersection angle of the crosses will always meet at 90 degrees e.g. Mercator projection

• If a projection is equal-area, the area of the circles will remain the same while the shape of the circles change and intersection angle of the crosses will not always meet at 90 degrees e.g. Mollweide projection or Hammer projection

• If a projection preserves neither property, the area of the circles and their shape will change, and the intersection angle of the crosses will not always meet at 90 degrees e.g. Robinson

After adding the Indicatrix Mapper plugin to QGIS (menu Plugins – Manage and Install Plugins) first add a basemap using the OpenLayers plugin e.g. Bing Aerial layer, then click the Indicatrix Mapper icon and run the plugin using default settings. You can then select different projections (lower right in world icon QGIS) to see the effects of various protections on the Tissot Indicatrix. If the circles appear as squares after selecting a different projection, right click the Circles layer in the layers panel, then select the Rendering tab and deselect the Simplify geometry check box. Also, turn off the basemap layer when using different projections, unfortunately the OpenLayers plugin only supports Google Mercator projection (EPSG 3857). To create the basemap below, that can be displayed using different projections, I styled vector data downloaded from Natural Earth and OpenStreetMap.

Mercator Projection – the area of the circles and size of the crosses increase towards the poles but their shape remains the same.

Mollweide Projection – the area of the circles remain the same but their shapes are distorted, the crosses do not always intersect at 90 degrees.

Robinson Projection – both the area of the circles and intersection angle of the crosses circles vary.

It is important to note that a Tissot’s Indicatrix generated in QGIS is an approximation of mathematical ideal, we are not no longer dealing with infinity small circles. As a result, here will be some minor distortion visible towards the edge of a map independent of the projection used; notice that the circles in the Mercator projection nearest the poles are not quite symmetrical or the circles at the edge of the Mollweide projection do not appear to preserve area as they should. This anomalous distortion can be minimised by reducing the size and spacing of the circles and crosses created by the Indicatrix Mapper plugin. However, despite these limitations a Tissot’s Indicatrix elegantly reveals the distortion present. This is something to important to understand when when choosing a map projection.

References:

Jenny, B., 2012. Adaptive composite map projections [PDF]. Visualization and Computer Graphics, IEEE Transactions on, 18, 2575–2582.

Tissot, M.A., 1881. Mémoire sur la représentation des surfaces et les projections des cartes géographiques. Gauthier-Villars.

ArcGIS REST API Connector Plugin for QGIS

Last year we described a command line method that adds ESRI REST layers in QGIS. Well, a team at the Geometa Lab in the University of Applied Sciences Rapperswil (HSR) Switzerland, have released a plugin for QGIS that adds ESRI REST layers via a GUI (Github page). The plugin is experimental so you will need to tick the box “Show also experimental plugins” in the settings panel of the “Plugins – Manage and Install Plugins” dialogue in order to add the plugin to QGIS. The following URLs lists numerous REST layers in the plugin’s GUI:

http://services.arcgisonline.com/arcgis/rest/services

http://basemap.nationalmap.gov/arcgis/rest/services

http://services.nationalmap.gov/arcgis/rest/services

Reference:

REST API Connector Plug-in Wiki Page

Create great looking hillshaded maps in QGIS

In this tutorial I will show you how to create a Hillshaded topographic map in QGIS. We will be using Shuttle Radar Topography Mission (SRTM) data, a near global Digital Elevation Model (DEM) collected in February 2000 aboard NASA’s Space Shuttle Endeavour (mission STS-99). The mission used a X-Band mapping radar to measure the Earth’s topography, built in collaboration with the U.S. Jet Propulsion Laboratory, the U.S. National Imagery and Mapping Agency (now the National Geospatial-Intelligence Agency), and the German and Italian space agencies.

The raw radar data has been continuously processed and improved since it was first collected. Countless artefacts have been painstakingly removed and areas of missing data have been filled using alternate data sources. The version we will be using is the 1 Arc-Second Global SRTM dataset, an enhanced 30 meter resolution DEM that was released last year. It is a substantial improvement over the 3 Arc-Second / 90 meter SRTM data previously available for Ireland. SRTM elevation data can be downloaded from the United States Geological Survey’s EarthExplorer website.

When first loaded into QGIS (via Add Raster Layer), the DEM is displayed as a rather uninformative black and white image.

It is therefore necessary to apply a suitable colour ramp that accentuates topography. While it is possible to create your own colour ramp, or use one of the colour ramps provided by QGIS, superior colour ramps can be downloaded using Etienne Tourigny’s Color Ramp Manager (Plugins – Manage and Install Plugins). After the plugin is added to QGIS, go to the Plugins menu again and choose the Colour Ramp Manager.

In the window that pops up, choose the full opt-city package and click check for update. The plugin will then download the cpt-city library, a collection of over a hundred cartographic gradients (version 2.15). After the package downloads, quit the dialogue.

Back in QGIS, right click the DEM layer to bring up the Layer Properties dialogue. In the Style tab, change the render type from single band grey to single band pseudocolor. Then click new color ramp and new color ramp again, choose the cpt-city color ramp to bring up the cpt-city dialogue. Click topography and choose the sd-a colour ramp. While this is an excellent colour ramp, I find its colours are a bit too strong for my liking.

Still in the Layer Properties dialogue, change the min and max values to match your DEM’s lowest and highest elevations values and click classify, this applies the new colour ramp. Next, change the brightness to 30 and lower the contrast and saturation to -20. Click OK to apply the new style and quit the Layer Properties dialogue.

Next we need to create a Hillshade layer from the DEM, a 3D like visual representation of topographic relief. This is achieved via the menu Raster – Analysis – DEM (Terrain models). There is one small catch, the hillshading algorithm assumes the DEM’s horizontal units are in meters (they are decimal degrees). We need to enter a scale correction factor of 111120 (in the Scale ratio vert. units to horiz. box). Once that is all done, select an output path to save the generated hillshade and click OK. Generating a hillshade may take up to a minute depending on the size of your DEM.

After the hillshade is created, open its Layer properties dialogue. Change the min and max values to 125 and 255, increase its brightness to 45 and contrast to 20. Finally, switch the blending mode from normal to multiply. This allows the DEM beneath the hillshade to show though. Click OK to apply the new style.

If you followed these steps correctly you will have created a fine looking topographic map similar to the one below. It’s also possible to create contours but that’s a tutorial for another day.

Technical note:

There are two hillshading algorithms available in QGIS, one by Horne (1981) and another by Zevenbergen and Thorne (1987). Jones (1998) examined the quality of hillshading algorithms, he found the algorithm of Fleming and Ho€er (1979) is slightly superior to Horne’s (1981) algorithm. Zevenbergen and Thorne’s (1987) algorithm is a derivation of Fleming and Ho€er’s (1979) formula. QGIS uses Horne’s (1981) algorithm by default.

References:

Horn, B.K., 1981. Hill shading and the reflectance map. Proceedings of the IEEE, 69, 14–47.

Jones, K.H., 1998. A comparison of algorithms used to compute hill slope as a property of the DEM [PDF]. Computers & Geosciences, 24, 315–323.

Zevenbergen, L.W. & Thorne, C.R., 1987. Quantitative analysis of land surface topography. Earth surface processes and landforms, 12, 47–56.

Oceancolor Data Downloader v1.0 for QGIS

Sea Surface Temperature data downloaded by Oceancolor Data Downloader.

The Oceancolor Data Downloader is a new plugin for QGIS from the Mapping and Geographic Information Centre of the British Antarctic Survey that downloads Oceancolor and Sea Surface Temperature data from NASA’s Oceancolor website. The plugin currently downloads three datasets:

• MODIS AQUA chlorophyll concentration
• SeaWiFS chlorophyll concentration
• MODIS AQUA night time Sea Surface Temperatures

The data accessed includes daily, 8 day, monthly and yearly composites, all of which can be saved to disk while downloading. Future plans for the plugin include additional access to other datasets such as ocean Net Primary Production, selection by bounding box, the ability to save in other formats, a progress bar etc.

I used the plugin to download global Sea Surface Temperatures for the 13th Jan 2015. I then used shapefiles from Natural Earth to create a simple basemap. I finally chose the IBCAO Polar Stereographic projection (EPSG: 3996) to create a map centred on the North Pole.

If you use the plugin to produce published research, please cite:

Importing CSV files into PostgreSQL using the DB Manager in QGIS

There is very useful tool in QGIS that can import very large CSV files into PostgreSQL rapidly and reliably. The DB Manager’s “Import Vector Layer” tool. Contrary to its highly misleading title it can import CSV files as well. Open the DB Manager (menu Database – DB Manager). Then select the database where you want to store your table and click the “Import layer/file” icon.

From the Import Vector Layer GUI, locate our CSV file on disk and enter the name of your new table in the Table box and click OK. Yes, it’s that simple. Proceeding this, you may need to select an text encoding scheme, files created on Windows often use ISO-8859-1 (Latin-1) instead of UTF-8 encoding. In my case, I was able to import a large statistical data set describing the energy efficiency of 525,500 Irish homes (432 megabytes) into PostgreSQL in ~15 minutes. After the CSV file is imported, you can optionally add it to your project using the DB Manager, right-click the table and select Add to Canvas. Don’t use the “Add PostGIS Layers” menu, it’s not a PostGIS layer.

And one more useful tip. You can convert Tab delimited text to CSV using QGIS. Load a Tab delimited text file into QGIS using the Add Delimited Text Layer GUI, then right click the imported file in the layer panel and save it as a CSV file.

Go2streetview plugin for QGIS

A very handy plugin for QGIS I use day to day is go2streetview by Enrico Ferreguti. The plugin adds an icon to the tool bar in QGIS and when selected I can click a road or street on a base map and a window will open that displays the Google Street or a Bing Maps Bird’s Eye view of the location. The camera’s direction and location is highlighted by a blue marker. I use the plugin when tracing boundaries of parks, open spaces and foot paths from aerial imagery. If the imagery is blurred or the view is obscured by trees, I click a point on a nearby street to see the location up close. The plugin works wherever Google Street view and Bing Birds Eye has coverage.

For example, in the screen-shot below notice there is a footpath leading to a bus shelter that’s not mapped by OpenStreetMap. I know where it is now, I will add it to my map.

Plugin: go2streetview

CartoDB wins best “high-growth web entrepreneur” at the 2014 European Web Entrepreneur of the Year Awards

CartoDB, the FOSS powered web mapping solution, was honoured at the 2014 European Web Entrepreneur of the Year Awards along with three other companies at Dublin’s Web Summit on November 7^th. The awards, presented by a European Commission backed body, were announced after a six month competition that involved public voting across four categories. CartoDB won the award for best “high-growth web entrepreneur”. CartoDB now has a growth rate of over 15% per month and customers in over 30 countries.

In September, a partner company of CartoDB, Kudos Ltda., released a plug-in that allows QGIS users to view, create, edit and delete data stored on their CartoDB accounts. Here is a map I created with the help of the new CartoDB plug-in that shows mountains and hills across the contiguous United States in the form of a heat map.

CartoDB and QGIS illustrate the exciting convergence between web hosted and desktop GIS, where interactive maps created in QGIS can be quickly published on the web and viewed by a worldwide audience.