OpenStreetMap

StreetComplete Overlays

Posted by westnordost on 15 June 2022 in English. Last updated on 6 July 2022.

StreetComplete v45.0 now has overlays! Finally!

You can load the alpha version from GitHub, the beta and release versions will be rolled out in a few weeks, when the bugs are ironed out. Please report any issues you find!

Overlays are a new way in which data is represented and editable on the map, additional to and orthogonal to quests: Display and edit just one aspect of OpenStreetMap, but allow it to be edited in an easy and efficient way.

Overlays help contributors to comprehensively record the kind of data that has the most value if it is mapped area-wide - for example addresses, POIs, sidewalks, cycle lanes, smoothness and other properties that are used as part of a network (e.g. for routing). In particular, overlays make it easy to identify where the information is missing and see at one glance if the information is (still) correct.

The initial version just includes two overlays to select from, “sidewalks” and “street lighting”. More will follow in the future but can also be contributed by others, just like quests, see the overlay guidelines.

overlay-lit street lighting overlay

overlay-sidewalk sidewalk lighting overlay

Journey

You won’t believe how long of a journey it has been to reach this point. In fact, I already came up with this idea early 2020. It was actually part of my pitch for the Prototypefund project, a grant program by the German ministry of Education and Research for FOSS.

I think the first time I described it was in this ticket in January 2021, because I was seeking feedback on the idea from the community.

mockups one of the mockups from the ticket above

Prototypefund

During the funding, I was only able to work on some technical groundwork for it, most importantly what had been released with StreetComplete v26 (Nov 2020) and v32 (May 2021). The funding through Prototypefund was over, so this topic lay bare for a while.

The changes made however, had some significant advantages on their own:

v26 changed how the OSM data was downloaded and how quests were generated. Before, each quest would do an own query to the Overpass API and place a quest pin on the map for every element that was returned. With v26 onwards, the app would download all the data directly from the OpenStreetMap API first and analyze it for quests locally on the device. Intended as one step towards the technical groundwork for overlays, it turned out as THE biggest but entirely accidental performance improvement for downloading quests…

This comparison shows something else too: The more quests exists, the slower the download, because the more is there to analyze

… and made usage of the app so much more effective that edits made per user more than doubled…

StreetComplete users and edits 2017-2021

v32 replaced the previously very quest-focussed architecture of the data flow completely. The side-effects of this were that it was now more effective to use the app when offline and the app also got a very powerful undo function. I talked about these changes on the State of the Map 2021:

Click the thumbnail to watch to the video

NLNet Fund

But I didn’t forget the original purpose why I made these huge changes to the app’s architecture. Thanks to the NLNet fund, I have been able to work on this once more. Now, also towards the end of this grant, I finally came around to finish this endeavour.

The outcome of it is now this first version of overlays. Are you curious how the overlay is defined in code? This is how it looks like. As with quests, I try to make the interface as straightforward as possible:

MapComplete

If you don’t know MapComplete, check it out: https://mapcomplete.osm.be

So ehh, what’s the difference to MapComplete, then? In fact, the conceptual approach of overlays is very similar to MapComplete themes. From my point of view, there are the following differences:

  • MapComplete doubles as a data viewer. This is a big selling point for MapComplete because it sees itself as a platform for community contributed special interest maps with an editing function. By making it easy to setup own themes, it strives to get all those “special interest” maps like the hydrant map, vegan restaurants map, beer map, table tennis map and whatever else is out there under one roof with something that is easier to set up than an own website. Plus of course, it includes an (easy) editing function, living the notion of OSM as a true wiki map

  • StreetComplete overlays are more strictly curated to be most quick and efficient to use on the go, as both the overlay and quest guidelines apply. This however makes the scope of for which data overlays are suitable in StreetComplete narrower. In particular StreetComplete overlays (will) only exist for data that makes sense to update on-site, as this the app is meant to be used while on a survey only.

  • MapComplete is a web-based app that necessitates being online, while StreetComplete is an Android app that works well offline and this applies to its overlays, too, of course

The End

I have now effectively dedicated more than two years of my life to more or less full-time development on StreetComplete, some time of it financed by a grant, some time of it out on my own account.

Since I have now reached my long-term goal and because the NLNet grant is soon coming to an end, I will soon need to open a new chapter in my life and end my intensive involvement in StreetComplete development in order to get a proper job.

Don’t worry, thanks to the crowd funding through liberapay, Github Sponsors and Patreon I will be able to continue to maintain the app on the side, but development will somewhat slow down of course. It is truly difficult for me to disengage from this project after such a long time to the point where it will actually be a challenge for me.

Also, not to forget the other regular and irregular contributors that continuously work on further improving the app whose names you have certainly read many times in the changelog. Certainly, and I am very happy about that, StreetComplete is one of the most active community driven OSM projects right now.

Discussion

Comment from Mateusz Konieczny on 15 June 2022 at 05:50

Congratulations! It looks great, though I worry a bit about performance.

I will be able to continue to maintain the app on the side, but development will somewhat slow down of course. It is truly difficult for me to disengage from this project after such a long time to the point where it will actually be a challenge for me

Good luck with your other projects! I hope that you will be able to balance StreetComplete and other projects with everything ending in good shape.

Comment from Fnordson on 15 June 2022 at 07:07

Thank you VERY MUCH for what you’ve done to OSM and of course for the recent feature explosion. The app managed to motivate people around me who didn’t even know where OSM appears in their life to not only learn about it but also to participate in improving it.

What a fantastic project.

Comment from Mateusz Konieczny on 15 June 2022 at 07:52

thanks to the crowd funding through liberapay, Github Sponsors and Patreon

links are missing so lets provide them if someone wants to fund work by Tobias:

https://liberapay.com/westnordost

https://www.patreon.com/westnordost

https://github.com/sponsors/westnordost

Comment from tordans on 15 June 2022 at 09:46

About feature:

Thanks a lot for this work and feature, it looks great and is needed!

I wonder, is there a description of the use cases that the overlays will solve in this version of the feature? Especially: I wonder what the conditions are that make a line green/pink/red. Could I use this feature to re-survey an area? (See also https://github.com/streetcomplete/StreetComplete/discussions/3644.)

About “the end”:

I really wished the OSM(F) would find a way to support work like yours in a more long term capacity. Reading the effort it took for you to build this feature and get the funding to do so shows very clearly that work like this has to be funded and the funding really should become easier.

Best of luck finding a great fit for your future work!

Comment from Mateusz Konieczny on 15 June 2022 at 10:21

Could I use this feature to re-survey an area

Yes, that is the whole point of it :)

Comment from Mateusz Konieczny on 15 June 2022 at 10:26

I really wished the OSM(F) would find a way to support work like yours in a more long term capacity.

While I would personally benefit from OSMF funding full-time StreetComplete development, I want to point out that in case of OSMF paying people to do work there are several more important projects.

Like making OSM projects GDPR compliant, see https://www.openstreetmap.org/user/SimonPoole/diary/399367

Or second paid sysadmin. Or more work on iD

Not sure is JOSM requiring any support?

StreetComplete is great, but I would not say that it is the most important project in OSM.

(though if someone or something would be willing to fund my work, I would be enthusiastic - right now it NLnet foundation via NGI Zero Discovery grant program)

Comment from westnordost on 15 June 2022 at 11:28

I wonder what the conditions are that make a line green/pink/red.

The coloring depends on the overlay. Right now, the only color convention is red for “information is missing” and black for “no” (e.g. no sidewalk). This may change. Early mockups envisaged to show a legend, but I dropped this idea because it adds effort for overlay contributors + translators and the user can simply tap on one specifically-colored element to see what the color means.

Could I use this feature to re-survey an area? (See also https://github.com/streetcomplete/StreetComplete/discussions/3644.)

Right now, the “OK” button is greyed out if you did not change anything. So, setting e.g. check_date:sidewalk while not otherwise touching the tagging is currently not possible in this first version.

However, the idea is to enable the thing you mentioned in the future by adding some kind of setting for the overlays in which you can specify to color elements last checked more than X days/weeks/months/years ago as if they haven’t been tagged at all yet. E.g. if you set this slider to “1 day”, probably the whole map except those you just answered will appear in red. For such elements, the OK button shall be not greyed out. I did not implement this yet because I did not see a use case that’s worth further delaying this initial version of the overlays.

Comment from westnordost on 15 June 2022 at 11:59

I worry a bit about performance.

I do too. #4079 will produce a relief.

But otherwise, I pushed the map rendering library tangram-es really far beyond what it is made for, and it shows. Nevermind possibly migrating to MapLibre, I think if something is done here, the more prudent move would be to do something similar what Bryan Housel and Ben Clarke did with (rap)iD: Use a hardware accelerated game/general purpose rendering library to render the OSM data itself.

Comment from CapitaineMoustache on 17 June 2022 at 20:49

Congratulations on your incredible work! StreetComplete is a wonderful application that I always recommend to new OSM contributors! It’s pretty, smooth, has a sense of progression with those little trophies and is incredibly easy to use. I’m always amazed at how accurately the tags are translated and illustrated and the quests filtered by country, it’s really high precision and so respectful of the OSM communities.

In short, one of the main if not the most indispensable application when one is completely new to OSM but wants to help (and even later when one has become a seasoned contributor).

Thanks again for your work!

A Frenchman with a moustache :{

Comment from chris_debian on 26 June 2022 at 13:03

Thank you, Tobias.

You’re work has been exceptional, and has made OSM map contributions much more accessible, which has increased the ‘currency’ of the map data.

Looking forward to seeing you in the community, in the future.

Cheers,

Chris, chris_debian 2E0FRU

Comment from realjep on 8 July 2022 at 23:10

Sincere congratulations! It’s an amazing work that is really useful and motivational.

Log in to leave a comment