PostCards from the Edge: A Tour of OSM Data Analyses + Visualizations (SOTMUS 2019)
Posted by Jennings Anderson on 19 September 2019 in English (English). Last updated on 1 October 2019.At State of the Map US a few weeks ago in Minneapolis, Minnesota, Seth and I presented a session titled:
PostCards from the Edge: A Tour of OSM Data Analyses + Visualizations
The recording and description of the presentation is available here.
Our goal was to curate a collection of OSM data visualizations from over the years that tell the story of OSM’s evolution, both as a map and a community, as well as highlight a few innovative data visualizations that show new ways to interact with OSM data to learn more about an area of the map.
We produced this spreadsheet (same as the table below) with links and author information for each of the visualizations that we showed and discussed in the talk. Since many of them are interactive, we chose to link to the original source:
Visualization | Author | Year |
---|---|---|
2 weeks of bicycle courier data in London | Tom Carden / eCourier | 2005 |
OSM Node Density | Martin Raifer | 2013-present |
Man-made vs. Natural feature density | Jennings Anderson | 2016 |
Object Density | Jennings Anderson | 2019 |
Non-diverse Mapping Density | Jennings Anderson | 2019 |
Haiti Earthquake Response | Mikel Maron | 2010 |
Edits with HOT | Jennings Anderson | 2019 |
HOT Project Activity Timeline | Martin Dittus | 2015 |
The life cycle of contributors in collaborative online communities—The case of OpenStreetMap | Daniel Bégin et al. | 2018 |
Timespan of OSM Contributor Engagement | Jennings Anderson | 2019 |
Cartographers of North Korea | Wonyoung So | 2019 |
Pipelines | Tim Meko, Washington Post | 2016 |
City Street Network Orientations | Geoff Boeing | 2018 |
OpenStreetMap past(s), OpenStreetMap future(s) | Alan McConchie | 2016 |
Optimal Routes by Car from the Geographic Center of the Contiguous United States to all Counties | Topi Tjukanov | 2017 |
A few of the visualizations were from my OSM research work, so I’m compiling them here:
Man Made & Natural Features in OSM
Made with tile-reduce & datamaps, this rendering of OSM data shows natural
features (such as ways tagged as natural=coastline
) in blue and all other features in orange. Do you know what those large orange rectangles in the Barents and Kara Seas are? View them on OSM.
Object Densities at Zoom level 12
Also made with tile-reduce, this visualization shows the density of objects in OSM as calculated by the number of objects in each zoom-level 12 osm-qa-tile.* At first glance, this figure shows there are few parts of the map that have no data. This is misleading, however. This is really a diverging color scheme where areas that appear blue or purple are unmapped. There are 0-100 objects representing areas of more than 60 square kilometers. In reality, these purple dots are showing us where we know something is there (such as the name of a town, a road, a river, etc.), but it has yet to be more completely mapped.
*Zoom level 12 tiles represent the area of about a small city. Their area decreases at higher latitudes, so normalizing against this would absolve cartographic sin. However, having done this and seen little affect to the message being conveyed here, I present the raw, non-normalized numbers.
Object Densities Broken Down by Contributor Count
Less than 10 mappers since 2018
More than 10 mappers since 2018
These two visualizations show the same density counts as the previous map, but exclusively show only tiles where more than or less than 10 mappers have been active since 2018-01-01. For many parts of the world, these appear to be a population density map (as many maps do). The takeaway here, however, is that while there may not be a lot of contributors active everywhere, there are at least a few contributors active most everywhere.
Contributor Lifespans
These charts are recreations of a chart first presented in Bégin et al. 2018. These charts are all derived from data obtained by querying the history of all OSM changesets (just under 70M) on the OSM public dataset on Amazon AWS with Amazon Athena.
Both axes represent time and each dot represents 1 user. Users that fall along the x=y diagonal are on-time contributors: Meaning their first edit and their last edit are on the same day. The vertical lines that begin to appear represent times when many users made their first edit (x-axis), and then some users continued to contribute for days, weeks, months, and years, creating the line.
Users along the top are still active, meaning their most-recent edit in OSM was near the time when we downloaded the data. The thick line across top means that there are many users who frequently edit the map, regardless of when they made their first edit.
All contributors
Contributors with at least 1 changeset with the text osmgeoweek
Contributors whose first edit was in 2015.
The impact of HOT editing on the growth of OSM
This figure shows the number of changes to the map per day, as calculated from all of the changesets in OSM. The area between the blue and orange lines represents edits in changesets that include the term “hotosm” in the comment.
Comment from Rovastar on 20 September 2019 at 03:28
Excellent stuff. I love the data side of OSM and have been following it for years.
for the “Timespan of OSM Contributor Engagement” in the graph and mentioned in your talk what is the big super active block of from mid 2016 til now was down to maps.me announcing on there splash page they used OSM and allowed editing and I agree. Had a great positive boost to OSM. Also at a similar time the PokemonGo effect is in play too, from when the game was launched it soon because clear that features like water, footpaths, etc in OSM had an influence on the game mechanics and from there people were editing OSM to get add these features (mostly legitimate edits but some rouge ones too) and the thick-ish vertical line (jan 2017) maybe is when they switched the ingame map in south korea to use OSM (previously the whole world was from Google map data)
You can see the impact of that here http://osmstats.neis-one.org/?item=countries&country=South%20Korea&date=28-1-2017 from about 10-15 users a day editing in the country to 1200+ in a single day
and the thick line vertical around November 2017 is when the whole world ingame map switched to OSM.
More details in this diary entry on the timelines: https://www.openstreetmap.org/user/WoodWoseWulf/diary/382394
(also other random things with OSM boost stuff recently Brazil went from around sub 100 daily mappers to I think 800 or something because some big youtuber gamer in Brazil that plays pokemon go posted a video saying you should edit Openstreetmap as it is based, even Portugal got 50% boost in there edits the day that video came out too)
But the visual map hasn’t been updated in nearly 2 years ingame I expect there will be another boost in editors when they get update the visuals again OSM has grown about 30% in nodes/ways/etc since then. ALso if maps.me ever did another splash page saying “we use OSM” when you open the map I imagine any boost in numbers.
Years ago you used to get big boosts in the numbers whenever OSM got in the news but we don’t get as much now and there are so many different areas.
And one small gripe is the “The impact of HOT editing on the growth of OSM” graph. I was so confused thinking I don’t believe HOT contributors are nearly 30-40% of all changesets as I have never seen those numbers looking at daily stats. Then I realized you didn’t start the Y axis at 0 you started about 1.3 million. Sorry just a personal bugbear of mine when looking at data visualizations it allows seems to misled me.
Oh you have probably seen it already and these are old but itoworld dis some great animations of OSM edits
https://vimeo.com/49003594
Hope that helps
Comment from Jennings Anderson on 22 September 2019 at 08:34
Thanks Rovastar!
You bring up some great points here, and I think you’re right, it’s a combination of maps.me and various games. Regardless, the change in mapping contribution patterns is dramatic.
Sorry to mislead with the y-axis; here’s another version of the graph that’s not time-bounded
Comment from Jez Nicholson on 30 September 2019 at 09:25
I see that ITOWorld animations have been mentioned…but, you’ve really got to include “OSM 2008: A Year of Edits” https://vimeo.com/2598878 for its historical relevance. Over the years many, many OSM talks have begun by showing it, to the point of it becoming a cliche.
Comment from Taktaal on 1 October 2019 at 20:38
Do you have any idea what event caused the large number of new editors in like May 2008, and the simultaneous quitting of editors in May 2011 and 2013?
Comment from Jennings Anderson on 1 October 2019 at 20:47
Yes, @Taktaal: So this visualization is a colorized, recreation of the visualizations in this paper by Bégin et al. with newer data.
That paper attributes the 2008 increase events to “The German journal Der Spiegel comparing OSM to Wikipedia” in May. Also, the mass dropping of users in mid 2011 is the removal of users / redaction work for those that didn’t accept the new license terms (as identified by Bégin et al.).
As for editors leaving in May 2013, I’m really not sure? Any ideas?
Thanks! Jennings