OpenStreetMap logo OpenStreetMap

dkbenjamin's Diary

Recent diary entries

Tl;dr: this is a somewhat technical post, so if you are interested primarily in finding alternatives to OSM Analytics, please scroll down to the last section. This post is being created in conjunction with a Community Forum topic as well, so any feedback or comments should be directed there.

Background

What is OSM Analytics?

For the past 8 years HOT has been supporting the hosting and maintenance of OSM Analytics. This service was built by Martin Raifer, hired by HOT with support by the Knight Foundation. It was originally developed to help people analyze and visualize OSM data it multiple ways:

  • Density and distribution of OSM features such as buildings, roads, hospitals, amenities, places and waterways from aggregated low zoom levels (world scale) all the way to individual features;
  • Total count of OSM features over different areas, including any admin boundary, user defined area of interest or custom GeoJSON;
  • Recency of edits displayed in a timeline graph, allowing users to select specific time periods and visualize the corresponding edits on the map
  • Visual distribution of features mapped by OSM contributors with different levels of experience, allowing interactive selection of experience levels and automatically displaying relevant features
  • Gap analysis, showing areas of estimated completeness with regards to buildings mapped;
  • Change over time, allowing users to compare density, distribution and total count of OSM features in any area at any zoom levels, between any two years from 2007 to now
  • Top OSM contributors and top distinct tags for any user defined areas, allowing interactive filtering through the timeline slider selector

Technical functionality

On the backend, OSMA relies on OSM QA Tiles and a data “cruncher” hosted by the HeiGIT lab at Heidelberg University. Until 2018 the QA tiles were processed daily and hosted by MapBox, then they were migrated over to HOT infrastructure. The OSM Analytics API continues to be hosted by HeiGIT (documented here). Additional charting functionalities for using and visualizing OSM Analytics data in other web applications is documented here. An example of these functionalities in action can be seen in the Open Cities Africa project website, where charts and statistics are sources interactively from OSMA. Any update to the OSMA data will automatically reflect in each web page.

History

Not much has changed with this tool over the years. This year OSM Analytics received about 2200 unique visits, seeing a gradual decline since we started tracking in 2019. HOT did not invest in updating dependencies, fixing bugs (Martin has generously continued to support in this effort without recognition), or optimizing performance of the OSM QA Tiles processor since the initial development and release. When HOT took over hosting and running the QA Tiles processor, we reduced the frequency of updates for QA Tiles to weekly, as this was the majority of the cost to the service overall.

Alternative analysis tools

At HOT, our Tech & Data team’s current focus is on facilitating mapping activity with our tools and enabling everyone in our focus regions to map, with a mission to map areas home to 1 billion people. Our technology stack aims at uniting people in our focus countries around mapping: We are improving the Tasking Manager, focusing on improving the quality of data being added to the map and making mapping more effective by leveraging ethical, open source AI modeling. We are prioritizing where humans are most in the loop through coordinated mobile data collection of higher-level data by adding more detailed and accurate tag information. In the meantime, we realize that leaves a gap for some users of OSM Analytics.

Since 2022, we are partnering with HeiGIT to really understand the contributions to the map and analyze the data being created by humanitarian mappers. The ohsome dashboard provides statistics on map data currentness and saturation, but it was not expressly built to replace OSM Analytics. HeiGIT has also produced the ohsome History Explorer (ohsome Hex) to explore edits on OSM.

HOT has also invested heavily in building a data processing engine called Underpass, which is just now entering the prototyping stage where data can be analyzed at scale. Other recent tools can provide similar statistics as well, such as Kontur’s disaster.ninja. From HOT’s perspective the APIs available from HeiGIT and Underpass will have a greater scope of analysis, higher accuracy, and flexible use-case to provide as a service to the OSM community.

We are seeking your feedback: Alongside this diary post we have also created a topic on the Community Forum where we can engage in a fruitful discussion about the future of OSM Analytics, alternatives, and currently missing functionality.