The problem

We are generating an increasing level of data as a society. An unstated goal of openstreetmap that many contributors subscribe to is “completeness” or “accuracy”, which works fine when you dataset is small, local and high level detail, but less so when scaled up to determining if every traffic light crossing in the world has tactile paving.

So naturally, automation and data imports are where people start to look; and very sensibly there’s a process to propose, review and ingest large datasets.

However, this relies on:

• Expertise and peer review
• Honesty and diligence of the importer to have and execute a QA plan
• A second level of QA tools and mappers to QA and maintain data

What could we do differently?

In the semantic web/linked data world, two big concepts emerged. The first is the semantic web layer cake, which talks about going from “machine readable” to “schemas” to “query” to “proof” to “trust”. In OSM terms these are poi, tags, overpass, a lot of tools like keep right or osmose, and at the moment, human boots on the ground survey.

The concept of 5 star open data is focused on the idea that we have a lot of data locked up in silos - and while it would be ideal to align it to every standard and have the highest quality possible data; 95% of the time it’s better to publish anything at all rather than wait until it’s perfect. So long as data consumers have an idea of the limitations, they can apply judgement when attempting to use it.

What is the current state?

A number of open data portals provide basic indicators of “5 star open data” quality.

In our wiki, we maintain documentation which describes the OSM community’s view on data quality of an external dataset.

We have tags for change sets describing the source.

What specifically would we change?

I’m proposing a set of tools or standard metadata for annotating external datasets and proposed/approved exports; so that editing and conflation tools can reason about the quality of data.

IE, if you have a dataset which is derived from OSM, corrects wrong tags and it has been human verified from a random sampling of 5% of the data? That’s a good candidate for letting a maintenance bot operate on this with minimal oversight, and is potentially 5 star quality.

Have a stream of AI generated shop names from street level imagery? Tag that was 2/5 and have flags for requiring human verification, even if it is one click approval.

What would be the impact?

By having these standards in place, tools that are typically used for bulk imports or conflation can add extra guard rails around the process; and from a community review/import approval perspective it becomes a discussion about the higher risk aspects of an import.

It also then greenlights a degree of automated maintenance activities - after data is imported and mappers are promoted to confirm accuracy in the ground; it then becomes lower risk to trust that data source for bots updating existing attributes.

I just spent a little while writing a feature extractor for water tanks; having come across a well mapped dataset on Kangaroo Island, SA.

While not the easiest, fighting windows 10 to get a working dev environment - vagrant, busted; python on windows lack dependencies, etc; finally some success with the windows linux subsystem…

… robosat is actually pretty simple!

I cant be the first to think this - is there a model zoo equivalent for OSM trained models?

Most Restaurants/Cafes & Shops; etc will have a web presence, sharing their address, opening hours, etc.

Named Restaurants/Cafes or Shops without a Website http://overpass-turbo.eu/s/nkh

Named Restaurants/Cafes or Shops with Website but no opening hours: http://overpass-turbo.eu/s/nki

Named Restaurants/Cafes or Shops with Website but no street name: http://overpass-turbo.eu/s/nkj

Keen to make it even easier to contribute to OSM, I threw a few pull requests at http://github.com/osmlabs/onosm.org - the github repository behind the wonderful OnOSM.org.

Its now possible for someone to use the ‘locate me’ control when adding their business; a small but important improvement for mobile users.

iandees has also been kind enough to clean up a few existing pull requests around internationalization - onosm.org now supports English, Portguese(Brazillian), and Italian. If you’d like to localize it, its really easy - just fork the github repository and create a new file in the locales directory.

There are a few other experiments for it pending, such as ‘sync my business details from a facebook page I own’, or ‘upload details of 5+ locations from a CSV’.

Ultimately, I’d be keen to provide an experience similar to a ‘manage my google listings’ scenario - both google and facebook have seen a lot of uptake from business owners. While business owners dont have quite the same incentive with OSM, I am really keen to make it as easy as possible if they choose to publish to OSM.

There are a few other ideas in the works, like “Invite a business owner to share/update their details”, or “check how my OSM listing appears on open data sites - maps.me, mapillary/openstreetcam, missing attributes, etc”

What do you see as the ultimate, simplest entry point to getting a business onto OSM with accurate, useful details?

Right now I am sitting very close to a place in Adelaide - San Churro. I know its mapped, at least with name.

However if I search for “openstreetmap san churro”, I see:

• Very rich data on opening hours (google), how long people spend at a location
• A business listing and map (google)
• A prompt to claim this business (google)

Contextually, my android phone already knows where I am so is potentially limiting the search to nearby businesses.

Previously I’ve been keen on a foursquare ‘checkin’ app to write to OSM and ask smart questions, as well as RDFa/schema.org markup to render useful results.

I’d like to go further and throw out a few feature requests with a bounty attached ($100 AUD each):

• On android or ios, add a plugin to Owntracks to allow check-in to an Openstreetmap location, publishing Lat/Lon, OSM id or new place name(suitable for a note)
• A way to connect to an Owntracks/Mqtt server and match user location/time to a 30 minute time slot (ie: indicating a likely valid opening hour)
• A public owntracks server for interested OSM users (worldwide? Or just country based)?
• An owntracks plugin for OSMand.

OpenStreetMap has a few assumptions about road surfaces, based on the fact most western cities have paved roads.

From https://wiki.openstreetmap.org/wiki/Key:surface

For roads for motor vehicles there there is normally an assumption that the surface is surface=paved unless otherwise stated. Paved in OpenStreetMap is non-specific and may cover sealed, tarmac, asphalt, bitumen. surface=unpaved is treated as the opposite of paved. More specific tags can used used for surfaces which are normally classified into paved or unpaved for routing purposes. Navigation software should assume that roads-that-are-not-paved will have slower driving speed (and therefore longer driving time) and may be impassable in some weather conditions.

In South America (Rural), Rural Australia, Africa, Haiti and more this is not the default. During activities like a HOT activation, quite a lot of road data gets added without specific surface tags.

In places like South America, Strava route building a big thing - but knowing if you can go on a road bike or a mountain bike is not something routing engines can answer with much certainty.

Alternatively, knowing if you are likely to need a 4WD vehicle in disaster relief is another use case which can’t be easily answered by a routing engine.

In an idea world, it’d be great if:

• ID had some notion of ‘rural’ vs ‘metropolitan’ mapping profiles; and allowed you to create a number of roads with the same surface properties as the last few you’ve traced.
• QA tools (maproulette? similar?) existed that looked at specific countries, finding highway=unclassified and highway=residential without explicit surface tags; plus a relationship to any bounding residential landuse to suggest a value (Big city? Its more likely to be paved residential roads)
• Something like DeepOSM that could guess the difference between “light, sandy coloured road” and “Dark, asphalt road” to suggest an appropriate classification.

Even if these tools weren’t used to write out lots of surface data back into OSM, it would be interesting to see if they could be used to generate a likely profile of an area for routing engines - ie; better answering if a certain section of the planet is likely to be unpaved roads.

Over the past few days I’ve been cleaning up a lot of Strava related user error reports. I’ve managed about 500 or so across the southern hemisphere; with a surprisingly high hit rate of actual issues.

One thing that has been quite obvious is that much of Latin America has a very active hiking and biking population; but a relatively poor routing experience.

There’s two easy ways to help in mere minutes:

Keepright’s Highway-Highway check

http://keepright.ipax.at/report_map.php?zoom=14&lat=-23.58791&lon=-46.65713

Look at the error, and decide if the two ways actually do join.

ID makes it quite easy to make a new junction, occasionally you might need to model a tunnel or bridge instead.

This is well suited to remote mapping; though imagery can be somewhat dated or misleading at times.

Strava’s heatmap enabled error report tool

http://labs.strava.com/routing-errors/

While you may not speak spanish or portugese, a lot of errors fall into the category of “there is a rural/unpaved road which is not traced”; and it’s very obvious from GPS data. Google translate can help immensely in understanding the basics.

Why São Paulo?

I’ve opted for São Paulo as a good place to focus; as it’s home to about 12-20 million people in the core/surrounding metro area - it’s effectively the 3rd largest city in the world.

In particular with the keepright errors; it’s plausible we can get to very few uncertain/unresolved issues in a reasonable amount of time.

This directly benefits anyone using Strava, OSMAnd, Scout, Maps.me for driving, cycling or jogging; which in turn is likely to lead to feedback (street names, for example or better GPS traces).

There’s also a quite active mapping community; who may not realise the impact they can have due to the visual map looking “complete” in a lot of ways.

If you’d like to help, spend a few minutes on either of the above two methods!

Many of the mobile editing tools are aimed at the more experienced end of the spectrum. The online tools are almost universally aimed at the full editing experience. I think however there’s a lot of value that can be extracted with the right questions; in a much more constrained use case.

For example, Foursquare will encourage users to rate attributes of a place they’ve checked into, like “Is X good for lunch?”

We’ve got similar small focus editors - things like http://ae.osmsurround.org/ae/index - but we really lack any idea of “does the user know much about the place to answer questions?”

https://developer.foursquare.com/docs/users/checkins does give us one means to ask for this; as did previous versions of the facebook API - I’m not entirely clear now that posts being tagged with ‘locations’ would allow you to understand a similar kind of relationship.

http://openbeermap.github.io/ almost perfectly describes the kind of experience I’d interested in - it has a lot of domain specific knowledge and guides the user into authoring the brewery tag correctly; ditto http://openwheelmap.org/ - both could be tremendously made more useful if coupled with user history and focused data enhancement tools.

I’m chugging away slowly at SA Road Naming - we’ve refined the processes quite a lot, and gotten into a nice working pattern.

The benefits of this data being in OSM are heavily focused on navigation and routing.

The tasks are typically one of three things:

• Find roads without a name in the JOSM file, download just that area, name it and commit it
• Find a missing road - copy and paste it in.
• Check Keepright or an OSM mirror for Almost Junctions, and fix up any that were created from the last task once a week or so.

The result is about 100 or so roads get added or named every 1-2 days - you can get a sense of the progress so far:

• 22 Mar : 5112
• 23 Mar : 4745
• 25 Mar : 4267
• 27 Mar : 3998
• 28 Mar : 3866
• 30 Mar : 3700
• 31 Mar : 3660
• 3 Apr : 3547
• 4 Apr : 3411
• 6 Apr : 3318
• 8 Apr : 3301

This kind of job is really well suited to a Map Roulette job, but to do that means I have to stop working on actually adding in any content - so I could use your help; so that I don’t sped the next 33 days going slightly batty.

Either:

1. Grab the current file and whack 10 or so road names. If 10 people fixed 10 names each day for 10 days(!) we’d be pretty close to finished.
2. See if you can set up a Map Roulette project - my (stalled) progress to date can be found at https://github.com/CloCkWeRX/osm-scripts/tree/add_maproulette - contributions heartily welcomed.

I decided to search for my local pizza place on google. I got a fairly good hit via their google+ listing, but their actual website was missing.

The node that I’d tagged myself is fairly well described, and I ended up updating both Google mapmaker and OSM.

It occurs to me that even if we’re not trying to compete with google in the local business listings, it would be interesting if we could build on it. If I’ve mapped an area, I want others to benefit from the data I’ve created.

Take for example, amenity=restaurant.

We know about 350,000+ of them:

http://taginfo.openstreetmap.org/tags/amenity=restaurant

325 659 have names. 150 598 have cusines. 44 900 have websites. 36 416 have phones.

Imagine if we were to change

http://www.openstreetmap.org/node/1937862511

to render

http://schema.org/Restaurant

… when we had enough authoritive data.

Google Rich Snippets (https://support.google.com/webmasters/answer/99170?hl=en) are just one of the benefits - schema.org, with RDFa would enable OSM to very quickly join the linkeddata (http://linkeddata.org/) web, and possibly provide a real world application or two in doing so (as compared to http://linkedgeodata.org/ which is great, but I’ve never used as an end consumer for example).

http://data.sa.gov.au/dataset/roads is CC-BY-A and explicitly OK’d to go into OSM; and more importantly contains every road in SA as of about March 2013.

I’d import it if I could be bothered fighting the technical challenges. The main problem is not the pain of converting the shapefile to something OSM understands, but avoiding duplication given the granularity of modelling in both datasets.

Instead, I’m spending a little time using OSM Inspector to find clusters of unnamed roads and fixing them by hand.

I’m doing this by:

• Using QGIS (sudo apt-get install qgis)
• Loading the shapefiles provided
• Changing the rendering to show the road name as a label
• Manually inspecting the shapefile to discover street name + type (it also contains surface, width, etc info)
• Correcting OSM

It’d be nice to evolve this from manual processes into a few automated sanity checks (find a list of all unnamed ways, find the corresponding candidates from the shape file, present to a user), and in the other direction provide the dataset providers a GeoRSS feed of new changes - particularly if people map into new areas; where for example OpenStreetMap currently has the most detailed maps of the St Clair Redevelopment.

If you want to lend a hand, or shepard the import of suburb boundaries through the import process, please reach out :)

I’m still really keen on MapCraft, especially since my original pull request got merged, and added some of the ‘sort by updated time’ behaviour.

This weekend, I’ve

• Added a topic to the chat
• Fixed an annoyance with updating unclaimed slices
• Made timestamps in chat more useful

There’s a fair few open feature requests there, but most are quite old. I’d be quite interested to know what people want to see - I’ve been fixing my own issues first and foremost.

A quick update on the coverage of Adelaide/SA areas, touching on residential buildings, McClaren Vale vineyards and a few others

http://clockwerx.blogspot.com.au/2013/06/openstreetmap-adelaide-status.html

Blogged about @ http://clockwerx.blogspot.com.au/2013/05/consuming-georss-and-querying.html

Relatively low effort to pull together - 10 hours perhaps? from start to finish, split over two sessions.

Demonstrates * SimpleXML/PHP to consume the overpass API and results * Simple Leaflet usage * Rendering of polygons from OSM data.

… are just about done.

I’ve covered Port Adelaide itself, Rosewater, Alberton, Pennington and Cheltenham. Osborne, Wingfield, Athol Park and Woodville / Woodville North are partially completed; plus to break up the boredem, much of McClaren Vale’s vineyards are identified too.