Exploring the growth and temporal mapping patterns in OSM in North America

The following figures are from my OSMUS Connect 2020 Talk. Additionally, I’ve included the relevant queries to reproduce these datasets from the OSM public dataset on AWS (See this blog post). For this work, I used a bounding box that encompasses North America.

Starting with the big picture…

This year we are averaging about 900 active mappers each day, with significant growth in the past few years:

SELECT 
    DATE_TRUNC('day',created_at) as day,
    COUNT(DISTINCT(uid)) as user_count,
FROM changesets
WHERE min_lat >  13.0 AND max_lat <  80.0 AND min_lon > -169.1 AND max_lon < -52.2
GROUP BY DATE_TRUNC('day',created_at)

How did we get here?

This next graph quantifies a mapper’s first edit in North America by month. For example, in August 2009, 1,700 contributors edited in North America for the first time. In January 2017, close to 7,000 contributors edited in North America for the first time.

SELECT 
    uid,
    MIN(DATE_TRUNC('month',created_at')
FROM changesets
WHERE min_lat >  13.0 AND max_lat <  80.0 AND min_lon > -169.1 AND max_lon < -52.2
GROUP BY uid

Putting those previous numbers in a bit more context, here is the comparison to the Global OSM Community:

Next, let’s break this down a bit further: Based on when a mapper made their first North American edit, how long did they stick around mapping? Personally, I prefer using the metric of Mapping days, which counts the number of distinct days that a mapper has been active. In this way, we’re counting mappers equally whether they edited 1 or 100 objects that day.

The highlighted blue line represents only the number of mappers who started mapping in North America and continued on to map more than 7 days (ever). For reference, I’ve circled the two peaks that are annotated in the first chart: Of the 1,700 mappers who started in August 2009, just over 200 of them continued to map more than 7 days over the past 11 years. Of the nearly 7,000 mappers that started in January 2017, just under 500 of them stuck around for 7 more days of mapping.

SELECT 
    uid,
    MIN(DATE_TRUNC('month',created_at') AS first_month,
    COUNT(DISTINCT(DATE_TRUNC('day',created_at))) as days_mapping        
FROM changesets
WHERE min_lat >  13.0 AND max_lat <  80.0 AND min_lon > -169.1 AND max_lon < -52.2
GROUP BY uid

If we increase the threshold to 30 days of mapping, we see significant growth since 2017 in the number of contributors that map in North America and then map for more than 30 days, about 50 mappers beginning each month:

Overall, we continue to growth in the OpenStreetMap US and North American mapping community. Recent years have seen an increased rate of growth, especially in the number of mappers with sustained mapping activity (being continually active).

What time—and on which days—do mappers contribute to North America?

This violin plot shows the breakdown of what time of day (In US Eastern Time) mappers were actively mapping in North America in 2011. The line through the middle represents the median time for mappers active each day, meaning that most mappers were active around 10am Eastern Time, each day, with little variation.

The green box highlighting the activity between 5am and 11am Eastern time represents the bulk of the activity, where the area is the widest. Though, this seems a bit early for the US, given that is only 2am on the West Coast; however, that is 10am in Europe. I think what we see here in 2011 are European mappers early in the day, and then North American mappers coming online throughout the day.

SELECT 
    DATE_TRUNC('hour', created_at) as hour_utc,
    COUNT(DISTINCT(uid)) as num_users
    COUNT(id) as changesets,
    SUM(num_changes) as num_changes,
FROM changesets
WHERE min_lat >  13.0 AND max_lat <  80.0 AND min_lon > -169.1 AND max_lon < -52.2
GROUP BY
    DATE_TRUNC('hour', created_at)

…and the median number of mappers per hour throughout the week:

…and finally mappers active per hour over time (in this case between July and September):

Now, let’s see how these evolve as we add more years

Between 2011 and 2014: violin plot looks very similar, generally we see growth: up to ~30 mappers per hour

In 2017, we see the first major changes. The shapes of the violins in the top line have not changed much, and the medians for 2011-2017 are very similar. However, the middle plot shows a major increase in hourly mappers on weekdays, but fewer on weekends. The bottom chart shows a more discernible weekly pattern, with more mappers active during the week than on the weekends:

In 2019, we start to see a difference in the violin plots as well as continuation of the weekday/weekend trend observed in 2017: Growing difference in the number of mappers active per hour on weekdays from weekends:

And finally, 2020. The shapes of the violins on top have changed, with a median editing time now of 3pm EST for weekdays. In the middle, we see more than 100 mappers active per hour on weekdays with far fewer active on weekends. On the bottom, the difference between the peaks and troughs of 2020 are the largest, with more than 100 mappers active per hour on weekdays than weekends.

So what does all of this tell us?

These charts show two trends between 2011 and 2020: First, growth. We expect to see this increase in mappers per day/hour when we compare back to the earlier figures in this post.

Second, the change in daily and hourly temporal patterns illustrates a shift in when contributors are actively mapping in North America. We cannot necessarily say what time of days mappers are active because we do not know a mapper’s local timezone, but most importantly, there has been a shift from a median of 10am (EST) to 3pm (EST) on weekdays.

Additionally, the evolving weekday / weekend pattern suggests that many (more) contributors are active during the week, potentially during school or working hours. The timeline also matches the rise of paid-editing in OSM, though the number of active paid editors does not account for all of these activities. There is more to investigate in these temporal patterns, but I suspect that we are seeing a shift/increase in the number of professionals and students that are using / contributing to OSM during working or school hours in North America. The 2020 OSMUS Community Survey saw a number of respondents reporting that they use OSM professionally, which corroborates this trend.

Quantifying these mapping behaviors now (in 2020) gives us a baseline to measure against as these trends continue to evolve.

The queries to reproduce all of my charts using the Amazon Public Dataset are included here to encourage readers to investigate these patterns in other regions. Please share any similar or additional findings you may come across!