The OSM dataset is a rich database of information about the world. Not only can the data be used to render beautiful and useful maps, it can also be used to do some nifty calculations in conjunction with other open datasets that exist. One such potential use is in cleaning up and improving elevation datasets such as the public domain SRTM or ASTER datasets. Both of these datasets offer elevation coverage over much of the world but they are limited in resolution (SRTM is 90 m per pixel, ASTER is 30 m per pixel).
Although these elevation datasets can be useful for very basic things like drawing 25 m contour lines on topographic maps, if you try to use these elevation models for more sophisticated things like drainage calculations, you will get very weird results due to a number of issues. First, almost no rivers are 30 m wide or more, so the river channel can’t be modeled at all. Second, noise in the elevation datasets often results in river valleys with “bumps” in the river that make it back up and flood large areas that in the real world would really flood because there is a stream draining the water out of the valley. Lastly, in many areas streams, drainage ditches, and other such things often run in close proximity to buildings and other such features. In these areas trying to fix the elevation to correctly predict drainage ends up changing the elevation of buildings as the pixels are so large the whole neighboorhood must be raised or lowered to make the drainage calculations work. The example shown below shows a couple of these issues, and other areas suffer from them in much more signifigant ways.
The original ASTER data at 30 m per pixel.
In order to fix the problems outlined above, and allow for other useful operations it is therefore worthwhile to upsample the horizontal resolution of these elevation datasets. The GRASS GIS system has tools like r.resamp and r.proj which can do things like cubic interpolation which provide nice smooth datasets at higher resolutions (in my example I am using 3 m per pixel, a 10 fold step up from the ASTER data I started with).
The data resampled to 3 m per pixel using r.proj with cubic interpolation.
This resampled data looks much more visually pleasing and if things like contour lines were computed from it they would be smoother, etc, but it still does not necessarily accurately reflect the drainage network topology for flood simulations. This is where the the OSM database comes in.
Because we map streams in OSM and because they are often traced from aerial imagery with resolution on the order of .1 meters per pixel, even very small streams and canals can be recorded with their exact position and shape accurately recorded. Furthermore, we can measure things like the width or depth of the waterway to further improve our data about the feature. This high precision stream path can then be “carved” into the higher resolution elevation model to allow the stream to cut through the small “bumps” intorduced by the cubic resampling. In the example below a 5 pixel wide (i.e. 15 meters wide) channel is cut into the dem everywhere there is a river or stream in OSM. The stream is first “leveled” by iterating over all the pixels in the stream and for each one setting it to the minimum of its own value or the 8 pixels surrounding it (the 3x3 neighborhood). During this leveling step if the pixel is on the stream centerline is is carved down 5 meters below the minimum neigborhood value, if it is along the edge of the stream it is only carved 2 meters below the minimum. Finally, after the carving is done the whole map is run over with a 5x5 neighborhood average to smooth out the river channel into something roughly U shaped to approximate a channel carved out by water, but still ensuring that the center of the channel will be free from major bumps that impede the flow of water in the flooding calculations. The result can be seen below, notice that not only can we now see the irrigation lines in the fields in the center of the image, we can also begin to see the rough cut in erosion pattern in the mountainous regions as well.
The final 3 m elevation model with the stream network from OSM carved into the surface.
The final result is quite nice, both visually and topographically. This is just a first step in this process to demonstrate the basic idea. I plan to do much more in terms of varying the width and depth profiles of the waterways to more accurately model the drainage system and I also want to look at integrating other sources of elevation data from the OSM database such as natural=water which shows a level contour line in the elevation profile at high resultion, natural=cliff and barrier=retaining_wall which indicate “jumps” in the elvelvation model, and also things like road and railway embankments, etc. The 3 m per pixel chosen is just small enough to nicely represent the width of a built up road or rail line or to represent roads “sunk” down a few inches from the surrounding grade due to things like curbs, etc. I hope to hear suggestions on other things from OSM I could use, or how the OSM data itself could be improved to better allow for these kinds of analyses.
EDIT: As requested in the comments below, here is the ‘difference’ calculated with r.mapcalc between the upsampled DEM and the carved DEM. The values in the “flats” i.e. anywhere that is not a river range from about -1 to 1 m and the river are as deep as 7 meters in the places where they cut through a “bump” caused by the cubic resampling process.
The “difference” between the 3 m DEM and the carved DEM.
-AndrewBuck
Discussion
Comment from Hedaja on 17 April 2014 at 06:41
thats really nice and interesting. I didn’t expect that you could use OSM data for this. Another source for elevation data could be the uploaded GPS tracks. I know that GPS elevation data isn’t very good but I think it’s worth giving it a try. I really would like to see a comparision between your elevation models in a flooding calculation.
-Hedaja
Comment from mcld on 17 April 2014 at 06:56
This is indeed a lovely idea for fine detail! Nice work Andrew.
Regarding OSM GPS-track elevations (as Hedaja suggests): they’re VERY poor accuracy in general. I tried inferring the terrain of the UK from OSM GPS-track elevations and the results came out really badly. The problem is not so much that the elevations are “noisy” because if we have enough measurements then variation gets averaged out - the problem is that they seem to have weird biases pushing the estimate in the wrong direction. (You also get problems due to GPS tracks recorded in aircraft!)
I didn’t find an obvious way of cleaning up the OSM GPS-track elevations to make them useful, but of course others may be cleverer than me.
Comment from Nick McW on 17 April 2014 at 07:25
Ingenious use of OSM data, and nicely implemented - many thanks for posting this Andrew.
I thought it’d be interesting to see the stream-adjusted image minus the unadjusted one to highlight the ‘corrections’.
Comment from AndrewBuck on 17 April 2014 at 15:06
@Hedaja @mcld Yes, in theory you can get elevation from GPS traces but there are a lot of problems. What I am trying to do is to take a different approach. Rather than trying to determine the absolute elevation of each pixel accurately, I am more interested in using the “hints” that the OSM data gives us about the relative elevation of a pixel to other nearby pixels. The ASTER and SRTM data are already pretty good once you get up to more broad scale elevation (beyond about 100 meteres or so), this is just to fill in the details and make them more suitable for really small scale mapping applications like high zoom level contour lines or for things like flood simulation, or terrain models for games, etc.
Regarding this “relative” elevation hints, this work grew out of an idea I have been working on for a long time. I started trying to document some of it a while back at the following wiki page. Feel free to add any other tags that might be relevant. Don’t worry about filling in the other columns if you don’t understand them, but anything that might tell you about the elevation of the ground either below or nearby the mapped object.
http://wiki.openstreetmap.org/wiki/OpenTerrainModel
@Nick I added the elevation difference from the carving, that was a great idea, it will really help me understand where my algorithm works and doesn’t work going forward.
-AndrewBuck
Comment from JohnDoe23 on 17 April 2014 at 16:02
At the FOSSGIS2014 there was a talk about routing with SRTM.
There are also artifacts when there’s an area where buildings are (for example Frankfurt’s banking district:12 meter difference in altitude (but buildings are over 100 meter high)
you can see some graphics about these problems in this talk: https://www.youtube.com/watch?v=c0TehKCX4Ao#t=2m45s
Wouldn’t it be possible to take into account landuse=forest, natural=wood and building=yes also?
Comment from imagico on 18 April 2014 at 11:53
Looks interesting, i am not sure how your approach ensures a hydrographically correct elevation model, according to my understanding your smoothing step will inevitably create local minima along the course of the streams.
My own attempts in that direction (see here for lakes and here for rivers) work by enforcing the hydrographic condition locally in an iterative process which then converges to a hydrographically sound elevation model. Depending on the source data it makes sense to implement additional constraints, for example in case of SRTM elevation along rivers is nearly always overestimated. One major problem with all such techniques is you inevitably also use detail from the original data.
The most serious problems when using OSM data for this purpose are the inconsistencies in the data, especially of course wrong direction of streams but also lack of clear differentiation between level water surfaces (lakes) and flowing water.
Comment from AndrewBuck on 19 April 2014 at 13:31
@JohnDoe23 Yes I am aware of the aretfacts, I think forests is a big part of why there are a lot along rivers because that is where the trees grow best due to the water. I do plan to take many things from OSM into account, not just landuse, but more sophisticated things as well, for example railroad tracks act much like a contour line since they cannot run at a steep grade, so you can gather elevation info from them just like you can with streams, etc.
@imagico Those are very interesting posts and I want to read through them all again in more detail. It would be very interesting to talk to you about this, can you connect to mumble (talk.hotosm.org) sometime or contact me on skype (my username there is the same as my name on this site except with a ‘40’ at the end of the name)?
-AndrewBuck
Comment from Nick McW on 24 April 2014 at 14:06
Hi again Andrew, and thanks very much for posting the “difference” plot. Fascinating stuff. I did a very quick comparison of the contours on your map with the terrain apparent on Google Earth (assuming the contours are for the carved DEM, that is); in this case, I wondered if a wider “cut” would more realistically model the rather V-shaped gulleys - or perhaps a larger smoothing window to further extend the channel effect. Anyway, I guess this will all vary greatly depending on the nature of the terrain, and you mentioned that you’ll experimenting more with the channel profiles, so it’ll be great to see the results (if and when time allows!).
Comment from AndrewBuck on 24 April 2014 at 14:11
Yeah, I plan to try to make the channel width and profile variable so that the original dem cut very roughly with the streams to force the dem to drain properly in the areas where we know how it should drain. Then run the drainage calculation and use the results to determine the signifigance of each stream. Finally, knowing this go back to the original dem and re-carve it, now using variable stream widths.
-AndrewBuck