OpenStreetMap

Observe - cross-platform, offline, field mapping tool for OSM

Posted by geohacker on 13 May 2019 in English (English).

Over the last few months, we’ve been building an offline-first field mapping tool for the OpenStreetMap ecosystem called Observe. Observe makes field surveying and verification easy for mappers, and works on iOS and Android.

Motivation

Field verification is an important part of keeping OSM data accurate. So far, field mapping exercises are largely manual, cumbersome, or require internet connectivity. OpenStreetMap has an active mobile editing ecosystem, but doesn’t offer the same editing experience as iD for beginners. Most often, mapping campaigns need a tool as good as iD that allows edits from the field to verify existing data and improve data quality.

Observe is a cross-platform, offline-first field mapping tool for OpenStreetMap, perhaps the first of its kind. Our primary goal was to build an application that makes field observation easy, and to provide a comparable experience to iD on Android and iOS — with some success. Observe focuses on browsing OSM data and allows users to add new points or verify existing information. The edits made offline are stored on the phone and uploaded when the mapper goes online. Observe is a product of several iterations of user research and a couple years of conceptualization.

Using Observe

Today, we’re launching development builds for both Android and iOS. You can download the Android APK from here or request for access if you’re on iPhone (you’ll have to submit a request with your email, then download and install a profile that’s sent to your email, we’ll then let you know when you can install the app). The app will be configured to use the OSM development API by default, as opposed to the production API. If you don’t have an account on the development API, create one. We want to let community members test Observe thoroughly and to give us valuable feedback before releasing a version for the production API.

Functionality

Observe allows mappers to add new points, move/delete existing points, and edit attributes of existing ways. Observe does not support complex way drawing operations or relation editing at the moment. To learn more about how to use Observe, read the user manual. Observe creates one changeset per feature. Past experience has led us to make this choice - it’s easy to revert the single defective changeset without losing many good changes. The app also supports a very rudimentary resolution of conflicts by allowing to see the changes upstream, and allowing to continue uploading.

Presets

Observe uses presets from iD. We’ve tried to include as many as possible, but there could still be certain combinations missing since this is an MVP. To keep the MVP performant, a handful of features are not available for editing. These will still be available on the base map.

Offline mapping

You can download areas of interest for editing offline. Observe will allow you to download areas of up to about 400 square kilometers, but it’s still dependent on your device, storage space, and quality of network. Edits made while offline are stored on the phone, and uploaded when the device is back online. The status of each upload can be seen in the contributions screen. When creating an offline pack, Observe downloads OSM data, base map tiles, and satellite imagery tiles for that area. When the user pans the map, while connected to the internet, these areas are automatically refreshed to fetch latest data.

Use-cases for Observe

We think Observe is excellent for the following use-cases:

  • Quickly map POI features - Observe is best for adding POIs. As you walk down the street, it only takes a few seconds to add a shop, bus stop, or any point of interest to OpenStreetMap.

  • Community mapping parties and field mapper exercises - Observe will work great particularly for mapping parties that identify clearly what feature to map beforehand. For instance, gathering a group of people to map trees in a neighborhood could be a really easy task with Observe.

  • Verify names of streets, shops, or other features - A lot of times we want to verify and correct tags of existing features. Observe allows us to edit tags and to add new tags to ways.

Development and future

Observe is built using react-native, and Mapbox GL. There are some really cool ideas on the list, and we’re exciting to see what feedback you have. On the roadmap are:

  1. Clear outstanding bugs, technical debt, and validate decisions. These are documented on GitHub.
  2. Tasking Manager integration
  3. Validation - both preset level and geometry level
  4. Notes - add a layer of OSM notes, and potentially add notifications
  5. Integrate Nominatim for search

Feedback

We’re looking for feedback. You can open a ticket in http://github.com/developmentseed/observe and let us know any bugs or issues you come across while using Observe. We’ve only tested Observe on a handful of devices, so if it doesn’t work on your phone, let us know. Any reasonably performing phone with Android 8+ should work well, as well as iOS 10.

Comment from ᚛ᚏᚒᚐᚔᚏᚔᚋ᚜ 🏳️‍🌈 on 13 May 2019 at 17:09

Interesting! It’s always good to see more developments in mobile mapping, it’s an area we need to improve and especially limited editing apps.

FYI your image at the top doesn’t work. I’d try the app, but there’s some pretty bleh privacy terms, why do want to know what apps I have installed and what calls I make?

You say “Observe creates one changeset per feature”, does that mean you make one changeset per edit? Did you know that StreetComplete initially made one changeset per edit too, but received (justified) criticism.

Comment from geohacker on 13 May 2019 at 17:27

Hi Rory, thanks! You can ignore the TestFairy privacy policy because we’re not sending them any data from the app. It’s just an easy way to drop the apk and ipa files, instead of having them in different places. If you’re on iPhone, you’d be sending test fairy your device id. That can’t be avoided if you want to download the app because apple doesn’t allow test builds unless the device is verified.

Thanks for pointing out about StreetComplete. I’ll take a look! We’re not against batching changes, just thought of trying this but very open to feedback. Thank you!

Comment from mvexel on 13 May 2019 at 22:27

I am excited about this! I downloaded the preview for Android and fired it up. I’ll try and share more feedback but here’s some initial points:

  • “Save” suggests that you’re saving locally. Instead this commits to OSM.
  • It connects to the OSM dev server which makes it hard to really try it out.
  • It wasn’t immediately obvious to me if any presets are available for POI types. It just asks for a common name and it looks like you can manually add tags.
  • The zooming / panning has some inertia that doesn’t feel right to me.

Observations made without reading the manual :)

Comment from alexkemp on 13 May 2019 at 22:31

Are the tiles that you download compatible with MAPS.ME (‘MapsWithMe’)? I’ve already got a few Gigabit of files downloaded with their app and do not want to duplicate the same files.

Comment from Zverik on 14 May 2019 at 07:16

One changeset per feature is worrying: I collect 100+ pois per field trip, and if this results in 100 changesets, I doubt people using osmcha would be happy.

Comment from Zverik on 14 May 2019 at 08:07

So… Had a quick test, and to be honest, the editor doesn’t look like it has couple of years of planning. The core might be good (less hassle in downloading stuff than in Vespucci), but interface-wise and OSM-wise the editor has a very long road ahead of it.

  • Presets do not work. Searched for “bench”, searched for “shoes” — no results except for a few branded presets.
  • Workflow in iD: create a point → choose a preset. Here it’s reversed: you click on “Point” and then have no idea where to go next.
  • Adding a building adds a building node. Why? Why is it a separate item in the list?
  • It doesn’t care if I’m on data or on wifi. There’s a big difference: in the former case, I pay money for traffic.
  • Entering a changeset comment for every changeset, one for every feature? How many users would do that, what’s your estimation?
  • Absolutely no way to use the editor with one hand. How many people hold a phone with both hands?
  • Using a mapbox library means there’s no attribution, and after clicking on (i) you suddenly find out there’s telemetry being sent to mapbox servers.
  • After switching a layer, map pans to the user’s location. Quite a surprise, if you were editing in a different region.
  • Turn phone off then on: a blue point of the current location shifts to a road, but clicking the crosshairs button positions the screen to your exact location. Weird.
  • Markers behave erratically: appear and disappear, show as white pins on farther zooms.
  • “Zoom in to edit” message is misleading: it does nothing to the UI, you can still edit and add new points.
  • When logging in, the app requires every permission available, including “create diary entries” and “modify your user preferences”. Are you sure it needs all these?
  • Log in, then edit some, then press “back” — and suddenly you’re on the OSM website, logging in again.
  • Notifications do not disappear unless you explicitly close them. Even after you close and open the app again.
  • The UI does nothing to simplify entering data or collecting extra data for later processing. Like, for a field surveying, it might suggest taking photos or recording an audio message.

I’ve tested the android version. It crashes occasionally, but technical issues are easy to fix, unlike conceptual flaws. Looks like the developers did next to no prior research before developing another POI editor. I suggest you look at:

And all the other mobile editors, including Vespucci, Go Map, MAPS.ME, and OsmAnd.

Comment from Zverik on 14 May 2019 at 08:12

Oops, forgot one important app for reference: https://wiki.openstreetmap.org/wiki/OSMfocus. If your focus is on updating data, then you need to show it. Like, not require clicking around to see the details. And fit more information on the screen: ideally so that a surveyor in a city won’t need to touch anything except the “turn on” button of the phone.

Comment from geohacker on 14 May 2019 at 08:21

Ilya - thank you for taking the time to test, really appreciate this from someone who has experience building maps.me! I won’t try to respond to each of your points, but I’ll open tickets on Github to the ones I think are bugs or absolutely interface problems, and it’ll be great if you chime in there too.

Overall, if you feel like it’s not polished enough, that’s understandable because this is just an MVP and we do intend to improve :)

In my post above, and the guide, we do talk about how presets work. At the moment, we don’t include all presets. Only a subset. And the search is limited to only presets that are compatible with point geometry. So if you don’t see something, that’s likely because of these reasons. Some of the other issues like reseting location while switching layers are known problem that have been captured and have explanation.

We’ll open a separate thread on changeset comments, and batching edits per changeset. I think it makes sense to batch edits into changeset. There are some ideas on the table and would be great to see what everyone thinks.

If you run into specific bugs, would you open tickets so we can try and reproduce? We’ll also need to know what device you’re using. Things like ‘notifications not going away’, for instance, I’m not able to reproduce on an iPhone X and Pixel 2. Same with the problem you report with ‘Zoom into edit’ button.

Thanks for all the links!

Comment from geohacker on 14 May 2019 at 08:23

Hey alexkemp!

Are the tiles that you download compatible with MAPS.ME (‘MapsWithMe’)?

Good question, unfortunately no. The offline tiles from both apps may not be the same. We use react-native-mapbox-gl to download map tiles, and integrate OSM data as XML. And it does sound very challenging to make the data shareable between these two apps. Sorry!

Comment from geohacker on 14 May 2019 at 08:27

Hey Martijn - I’m glad you are excited! I’m going to open up tickets for the things you raised. Some of them like telling there’s presets that can be searched, and also notifying that changes will be uploaded makes a lot of sense.

The zoom/pan inertia is not expected - what phone are you using? At least for a bit, I think we’ll want to point the app to OSM dev. It just gives us more time to fix obvious bugs and design issues. This feedback from you all is absolutely A+.

Comment from ndrw6 on 14 May 2019 at 08:31

I wasn’t able to log in with my usual OSM credentials. “Sorry, could not log in with those details.” Android 7.0, log in page opens up in Firefox.

Map rotation feature can be a bit annoying, it would be good to have an option to lock it down if not needed.

How would you describe (future) differences between Observe, Vespucci, Osmand, Go Map and Maps.me? Especially Vespucci, seems to have similar project goals.

My preferred way of surveying larger areas is to cycle through them while recording continuous video+audio commentary in my helmet camera and a GPX trace in Osmand. When walking (can’t attach any modern phone to a helmet) this could work well with just a mobile application. It would be great to have video and trace recording integrated in the OSM editor.

Comment from geohacker on 14 May 2019 at 08:33

I wasn’t able to log in with my usual OSM credentials.

Hi ndrw6 - you’d have to create an account on the OSM development API. This version of Observe doesn’t talk to the production API because we don’t to add test data there. You can create an account if you go here https://master.apis.dev.openstreetmap.org/

Comment from NunoCaldeira on 18 May 2019 at 17:57

Hi, trying to get a spin on this, however when i try to login on OpenStreetMap via your API it always returns that the password is wrong (and its not, i triple checked on OSM website on mobile and laptop). any suggestions?

Comment from trial on 19 May 2019 at 14:40

@NunoCaldeira , as said on the site and several times in this thread, you need to use a development account, not a normal account. Here the previous answer: “Hi ndrw6 - you’d have to create an account on the OSM development API. This version of Observe doesn’t talk to the production API because we don’t to add test data there. You can create an account if you go here https://master.apis.dev.openstreetmap.org/”

Comment from NunoCaldeira on 19 May 2019 at 14:41

Cheers


Login to leave a comment