OSM Progress timelapse videos with QGIS and Ohsome API
Posted by 2hu4u on 13 November 2024 in English.Finished result:
https://www.youtube.com/watch?v=xUkLQj29vE4
Preamble
I have been interested in making “before and after” comparisons of mapping progress for a while. The rich and beautiful OSM-carto style as it appears in standard OSM is a particularly important element of what I wanted to achieve, but it remains quite difficult to render historical map data in this style.
This guide mostly follows the ohsome guide from 2018 but taken further and with updates. Please be aware that I have done little to optimise the workflow so far; this is more of a proof-of-concept at the moment.
I started out mapping in August 2020 and mapped almost exclusively in my old hometown whilst I was stuck there during COVID. I spent a lot of time mapping this area very comprehensively and wanted to visualise my work. Big shoutout to user John Bek who greatly helped with building tracing, which is quite tedious. The Blue Mountains towns of Katoomba, Leura and Wentworth Falls featured are now some of the most comprehensively mapped in Australia.
Previous method
The most genuine way of rendering historical tiles is from the planet file. I previously tried running an OSM tile server in docker with data extracted from the pbf file downloaded from geofabrik. In windows I installed docker in an ubuntu VM and then installed the
“minimal effort” tile server following the github instructions. After a lot of messing around, I was able to generate tiles for a particular moment in time:
I think the way the pbf was spliced made the large natural=wood
relations disappear, so elements are missing unless I generated tiles for a much wider area.
Either way a lot of effort, and I never figured out how to procedurally generate tiles at different timesteps.
New method
Software requirement
- Internet browser (I hope you have this already)
- QGIS (version must be 3.14 or newer)
- ffmpeg
Get the data
I have been using ohsome visualisation tools for a while, but only quite recently I became aware of the ohsome API endpoint for Full History extraction of a region of interest. The first step is to get the bounding box coordinates of your area of interest. Tip: in OSM, the url is formatted like this:
https://www.openstreetmap.org/#map=19/{y}/{x}
where {y}
is the latitude in [-90,90] and {x}
is the longitude [-180,180]
Note down the lat/lon pairs x₁,y₁ for the northwest corner and x₂,y₂ for the southeast corner.
Now you can construct the API GET request. Amend the following example as necessary and paste it into your browser URL bar.
The bboxes
parameter is structured like bboxes={x_1},{y_1},{x_2},{y_2}
as below.
https://api.ohsome.org/v1/elementsFullHistory/geometry
?bboxes=150.298548,-33.679397,150.391572,-33.743226
&keys=highway
&properties=tags,metadata
&showMetadata=false
&time=2008-01-01,2024-11-01
&types=way
I have split up the request by querystring parameter for readability. Change bboxes
, keys
, time
and types
.
You’ll now want to repeat this step using the following combinations:
Key | Way | Relation | Vector geometry |
---|---|---|---|
highway |
Yes | No | Linestring |
waterway |
Yes | No | Linestring |
railway |
Yes | No | Linestring |
building |
Yes | Yes | Polygon |
landuse |
Yes | Yes | Polygon |
natural |
Yes | Yes | Polygon and Linestring |
amenity |
Yes | Yes | Polygon |
leisure |
Yes | Yes | Polygon |
These categories should cover the vast majority of what you can see rendered in OSM-carto. You might want to also download power
as linestring for power lines, etc.
You will get 13 GeoJSON files out of this; you can simply drag them into QGIS. Note that when downloading type=way
geometry, in QGIS you will be given a choice of what geometry type to use as a vector layer. For instance, natural=*
ways may be either a polygon (area) such as in natural=wood
or Linestring such as in natural=cliff
. I recommend you import them both separately if required.
You should also download relation areas (multipolygons) for common area types according to the table. Conveniently, ohsome API will also clip large relations to the bbox that you chose.
Altogether you will probably have 14 vector layers.
Styling in QGIS
The most time-consuming part is styling everything to look like it does on OSM. I suggest adding OpenStreetMap as a layer on top so you can colour-sample directly. Start by setting your QGIS project canvas colour to the OSM blank land colour #F2EFE9 (Project > Properties… (Ctrl + Shift + P) > General > Background Colour).
Highways
After importing your GeoJSONs into QGIS, you can open the Layer Properties and go to the Symbology tab. Set the type to Categorized, set the filter to your key (eg. highway
) and click Classify.
At this point, you’ll probably get a bunch of junk mixed in with the common highway types. Just delete the ones you don’t want. Then change Categorized to “Rule based”.
You don’t really need to do that many modifications here, just a few like discriminating service roads from driveways and parking aisles etc. I didn’t bother with the more subtle stuff like tracktype
, surface
and access
etc.
You now want to decide on a zoom level to encompass all your data, and consider the resolution of your final animation (I recommend 1080 x 1920). My map scale for the animation is 1:16255, which is between zoom level 15 and 16, I picked that just arbitrarily.
OSM roads are composed of an inner stroke and an outer stroke. You can symbolise this with two lines, one slightly thicker than the other. For mine, major roads were set to 1.8mm and 2.0mm for the thin and thick lines.
Minor roads such as link
s, residential
, unclassified
etc should be thinner. I used 1.3mm and 1.5mm. For all cases, make sure you set the Join and Cap style to “Round”.
If you want to be able to render your map at multiple zoom levels, you’ll need to adjust the thicknesses for each zoom level and enable Scale range such as below, according to this guide. This will be quite time consuming.
After you’ve finished copying all the colours, you’ll want to set the rendering order. Click “Symbol Order…” button and set the order as below;
The aim is to have the thick, dark outer stroke to level 0 and the thin, lighter stroke to higher numbers according to the road classification.
Finally, in the Temporal property tab, enable Dynamic Temporal Control and set the Limits to “include start, include end” and set the Start field to @validFrom
and the End field to @validTo
. Note that this is a fairly new feature in QGIS and performs much better in my experience than the plugins that the original ohsome blog post used.
You can now check if everything is working by opening the Temporal Controller Panel (clock icon) in one of the toolbars up the top of the QGIS main UI. Try scrubbing through the timeline and see how the data changes.
I ended up turning driveways and parking aisles off because they encumbered the map too much at my zoom level.
Other stuff
This is what my Leisure non-relation polygons look like. These are the only categories I kept.
Most of them are fairly straightforward, however some things like leisure=natural_reserve
are less intuitive and also drawn above everything else, so you may want to separate it into its own layer.
To get the green stroke that is only inside the polygon, I set the effect “Inner glow” with zero blur radius. Make sure you uncheck “Source” otherwise the inner part won’t be transparent. I set it to #B6DEB6 and slightly transparent just by estimating it, I’m sure you could probably figure out the actual correct colour from OSM-carto repository but this will do for now.
Here are my settings for leisure=garden
.
Some other interesting ones are natural=cliff
;
wetland=marsh
;
and natural=scrub
and natural=wood
were basically the same but with different SVG symbols that I thought were a fairly close match from what I already had installed. I didn’t really try that hard for it to be accurate because most of this stuff you can’t really see anyway.
For waterways, I set it to rule-based where intermittent=yes
was the only rule for the dashed line style, and then everything else was rendered the same.
You can check this imgur album for other settings.
Finally, make sure your layers are all in a sensible order. I put linestrings and nature reserve over the top of everything else. Landuse was on the bottom; amenities and leisure above that and then natural on top of those. OSM-carto has a lot of rules for this but the goal is not really an exact replication here.
Note that I didn’t render labels or point icons, this is on purpose; I feel like it isn’t really appropriate for a timelapse video.
Animating everything
Make sure all layers have the same Temporal settings. Now in the Temporal Controller, check to make sure everything looks good. Set the Step interval to 1 month, make sure your viewport is roughly in the centre of your Area of Interest, Save the project, and now you should be ready to export the animation. Set the resolution to something sensible, I suggest 1080 x 1920. I tried 4K but my laptop struggled hard. Again, I suggest testing it with a short timespan first to make sure your settings are dialled in correctly. It should save the frames as png images in the format usually “projectname0001.png” etc.
Compile the video with ffmpeg
Open your terminal and change the directory to the folder with all your png frames. Here is my script below. It includes a date counter in the format YYYY mm
in the top left corner. You can get rid of the -vf
flag if you don’t want the counter. Make sure you change the input name to your own one.
ffmpeg -framerate 24 -i "openstreetmap timelapse%04d.png" -vf "drawbox=x=w-150:y=h-45:w=150:h=45:color=black@0.8:t=fill,drawtext=fontfile=NotoSans-Bold: text='%{eif\:2008+floor(n/12)\:d} %{eif\:(mod(n,12)+1)\:d}': fontsize=30: fontcolor=white: x=10: y=10: borderw=1: bordercolor=black" -pix_fmt yuv420p output.mp4
The drawbox
and drawtext
draws a transparent background for the date counter and the text with the counter. This increments the month every frame, and the year every 12 frames, starting Jan 2008.
I found that the video won’t play for some people, or in Windows media player, or upload to Reddit without adding the pixel format flag -pix_fmt yuv420p
.
Wrapping up
If you have trouble with any of the steps, or suggestions on how to improve the workflow or the rendering, please leave a comment. Good luck and have fun.
Also check out the discussion on reddit here.
Discussion
Comment from rphyrin on 13 November 2024 at 10:01
Really cool.
I wonder if you could export and share those OSMCarto-like QGIS symbology settings so we can skip the time-consuming setup and benefit from your hard work instead 😂
Comment from 2hu4u on 13 November 2024 at 14:10
Thanks for the comment. It was my intention to share the styles, I tried exporting the qml style and loading it onto a previous version of QGIS on a different PC but it wasn’t playing nice with me, but I didn’t troubleshoot it very hard yet so there’s a good chance I was doing something wrong, probably something simple. Will put a bit more work into getting more tags set up and fine tuning. Perhaps I’ll put it on github. Watch this space
Comment from PlaneMad on 13 November 2024 at 19:27
Fantastic resource, thank you for the writeup. Watching these animations for any place is fascinating and it really should be more simple for anyone to do it for their area.
Thats is a ton of work to get the QGIS styles right, would love to also play around with the styles to try this out.
Comment from Geonick on 17 November 2024 at 21:04
Hey 2hu4u: You can share QGIS symbology using the QGIS Resource Sharing plugin https://qgis-contribution.github.io/QGIS-ResourceSharing/ .
Comment from Geonick on 17 November 2024 at 21:06
And regarding the time consuming styling: Soon the QGIS plugin “AIAMAS - AI-Assisted Map Styler” will also support line and area symbology. This plugin allows quick symbolization and automatically suggests fitting colors and symbols for newly loaded vector layers. Feedback welcome! https://plugins.qgis.org/plugins/AIAMAS/
Comment from joost schouppe on 19 November 2024 at 21:42
Awesome work, thanks for sharing. I did similar stuff back in 2014 with osm history data, but it was way more complex at the time. And the tools aren’t maintained anymore. And my maps aren’t online anymore it seems…https://www.openstreetmap.org/user/joost%20schouppe/diary/21826