OpenStreetMap

ODK Entities for OpenStreetMap

Posted by spwoodcock on 4 April 2024 in English. Last updated on 13 April 2024.

ODK

For those that don’t know, ODK is an incredible suite of tools for field data collection.

The Field Mapping Tasking Manager(FMTM) leverages two of their tools to coordinate field tagging of map data:

  • ODK Central as the centralised server to store survey data.
  • ODK Collect as the mobile app for survey-based data collection on mobile phones (working very nicely in offline contexts too).

Entities

  • ODK Entities were introduced to Central in September 2023, in order to more easily track the same feature over time.
  • As a result, we have a nice way to store a feature, with geometry and properties, in ODK Central.
    • This could quite easily map to the OSM ID, feature geometry, and feature tags.
  • The geometry can then be selected in ODK Collect survey questions.

select-from-map-polygon

How To Use Entities

Create an Entity List

Within ODK Central, a collection of Entities is called an Entity List (or dataset via the API).

Currently the only way to define an Entity List is to submit a XLSForm with an ‘entities’ tab defining the entity_id field:

entities-tab

And a ‘survey’ tab defining additional properties/field for the Entities:

entities-survey-tab

Example generic Entity form

  • This will change in the future, with the possibility to create an Entity List / dataset via the Central API.

Populate Entities

Once the Entity List exists, you must populate the data:

Option 1: via the API. Manually uploading the Entity details.

Option 2: via CSV attachment. Entities can be uploaded as a CSV attachment to a form (in theory, although I have yet to make this work!).

Option 3: created automatically on submission by specifying a ‘geopoint’ or ‘geoshape’ field in your form (and collecting the location in the field).

Changes to FMTM

Until now, FMTM was frankly abusing ODK Central and had a structure like this:

FMTM Project 🔗 ODK Project

FMTM Task Area 🔗 ODK XForm (the survey)

FMTM Geometry 🔗 ODK XForm Attachments

This resulted in potentially hundreds of ODK XForms per project, which is not an ideal usage.

With Entities, the new mapping for FMTM is:

FMTM Project 🔗 ODK Project, ODK Entity List (Dataset), ODK XForm

FMTM Task Area 🔗 Group of ODK Entities (via task_id property)

FMTM Geometry 🔗 ODK Entity

This is a much more logical 1:1 mapping of the lowest level unit we are interested in: OSM features.

A World of Possibilities

  • Now Entities exist, it is possible to update Entity fields, as they are mapped in the field.
  • For example, after tags have been added to an OSM feature in FMTM and mapping is complete, a ‘STATUS’ field could be updated for the Entity as ‘complete’. This would inform other users of ODK Collect that this building has already been mapped.
  • Even more intuitively, the feature could be ‘soft deleted’ after mapping, so it so the geometry disappears entirely from another users phone. Or the Entity label could be updated with a big ✅.
  • There is a whole section on updating Entities from forms in the ODK documentation.

So Where Does OpenStreetMap Come In?

  • The goal of FMTM is to add useful field-verified metadata (tags) to features on a map.
  • For example, we have the outline of a school, but wouldn’t it be great to know: how many stories it is, what the walls and roof are made from, how many people are typically inside the building during the day?
  • FMTM mostly uses OSM data for import and updating these tags (it is also possible to use custom data that does not exist in OSM (yet!).
  • ODK Entities are essentially a representation of these OpenStreetMap features, that can be updated from detailed ODK surveys done in the field.
Location: Kalgodin, Ouagadougou, Kadiogo, Centre, Burkina Faso

Discussion

Comment from PierZen on 13 April 2024 at 18:47

Thanks for this presentation of the ODK entities in the ODK Central server, the Android ODK Collect and the HOT FMTM.

Looking at the FMTM project #90, we see for each task, Buildings OSM vector data over the background OpenStreetMap with a link to download the MB Tiles.

A few questions / Comments about this :

  • What the MB Tiles contain, and what is their role ?
  • About ODK Entities for OpenStreetMap :
    1. Could the objective be to collect / update data about entities for OSM ?
    2. Do plans for the HOT FMTM are to eventually use new infos on the entities to update the OpenStreetMap database ? If yes, which workflow could be used ?
  • Note that no attribution to OpenStreetMap for the background Map as required when using OpenStreetMap. We only see references to MapBox layers and ArcGIS tiles. I hope that HOT will correct this, giving attribution to OpenStreetMap.

Comment from spwoodcock on 13 April 2024 at 19:59

You raise a great point that I forgot to include the section on how this relates to OSM - I updated!

For your questions: - FMTM currently has the default baselayer set to Mapbox Outdoors, which does of course use OSM as one of it’s sources. The attribution is present when selecting OSM as the basemap, but you are right we should probably also add the attribution to the Mapbox layer too - thanks!

osm-attribution

  • MBTiles are Mapbox Vector Tiles a format for storing multiple map tiles inside a single file. They are primarily used for displaying map tiles (basemaps) in offline settings. In our context, they can be loaded into ODK Collect as a basemap for use during offline mapping.
  • Updating OpenStreetMap with tags from field mapping is definitely the plan! The workflow will be built inside FMTM, with the final stage being conflation of our data with existing OSM data (in part using our conflation code).

The workflow would be: - Get survey data out of ODK along with geolocation. - Map each survey question to an OSM tag (this is mostly handled automatically by using good XLSForm field names). - Conflate the generated data with the existing OSM data (i.e. check if we should update existing tags, or add new tags). - Upload the updated tag information to OSM.

Comment from spwoodcock on 13 April 2024 at 20:02

(I forgot a line break between the sentence and bullet points and can’t update the comment, here is the nicely formatted list)

The workflow would be:

  • Get survey data out of ODK along with geolocation.
  • Map each survey question to an OSM tag (this is mostly handled automatically by using good XLSForm field names).
  • Conflate the generated data with the existing OSM data (i.e. check if we should update existing tags, or add new tags).
  • Upload the updated tag information to OSM.

Comment from PierZen on 13 April 2024 at 21:47

Thanks, I also see that your github project osm-fieldwork fir various conversions including odk to osm.

Log in to leave a comment