Reflections on 10 years of a changing Open Source Map

It was ten years ago today that I registered to map the highway!

With apologies to the Beatles, on 2007-12-28, I registered an account with OSM, started editing the map, and after returning from Falkirk to Cramlington, uploaded my first GPS survey data.

The map has changed rather a lot in that time, and not just in completeness, but in the diversity of the usage of OSM data. I was surprised a few years ago when visiting Lichfield that the Council was using OSM to show car parks, but shouldn’t have been surprised at all when preparing for a walk up Simonside when the Forestry Commission linked to an OSM map based in a small way on my previous visit!

Back in 2007, OSM in the North East of England included the major long distance roads only. Pioneers with GPS must have driven blindly into the blank ‘Here be Dragons’ space probably rushing towards Edinburgh. Back then, my town of 45k souls was little more than the odd through road so my earliest traces are dendritic studies of suburban planning surveys. Using the classic maze solving technique of ‘follow the left hand side of the road’, I cycled around local housing estates adding detail and getting progressively fitter. The improvement in my legs became important as survey cycle rides became progressively longer, ending up with an hour’s travel to map a distant hamlet in Northumberland before heading back.

Although my heavy commuter bike didn’t get any lighter, other tools of the field surveyor did improve. After my first Garmin GPS II+ GPSr fell victim to sea water ingress whilst teaching young dinghy sailors, I moved to a Nokia N800 - no, not the Lumia 800; the Nokia N800 Internet tablet. This was a full Linux device with a colour touch screen and both Bluetooth and Wi-Fi at a time when tablets were only available from pharmacies!

Using the rather amazing (for the time) Maemo Mapper software and a Bluetooth GPSr ‘puck’ it was possible to both capture a GPX file and more importantly, make field survey notes on the same device.

Interestingly, even back then, proprietary maps cost a lot of money - around £150 for the N800 so OSM was rather useful.

The 640x480 camera was so poor quality, even I could have captured scenes better with crayons, so text notes were the order of the day. GPX files with one point per second didn’t fill up the 256Mb SD card, and street names could be added directly to waypoints - something I recorded for posterity in the OSM Wiki with a screenshot back in 2008:

Pecking away at a tiny on-screen keyboard with a resistive stylus was S-L-O-W (remember, this was 2008!), so I experimented with Photo mapping, and Audio mapping using time synchronisation and the several JOSM plugins to work out where a note was captured (as the recording device didn’t have a GPSr). Years of field survey suggests mixed methods are best - photos for specific facts and general street scenes, audio for moving commentary, waypoints for signs.

Top Tip: A picture can indeed save a thousand words, which can be a long time when listening back to an Audio mapping recording for the forth time because wind and traffic noise mean you can’t quite catch the spelling of that side street name!

Top Tip: Never rely on a single survey method or device. Use more than one GPSr (Garmin models capture inertia differently; phones have poor antenna), take spare batteries, a camera phone, and a pen (and a spare pen :-) ).

Ah - the first mention of JOSM. Like many powerful tools, starting out with JOSM was VERY difficult. I remember finger bending gymnastics trying to trace GPX files and add roads. My 19” CRT monitor must have looked like a sunflower as the screen was surrounded by a halo of yellow sticky notes containing scrawled keyboard shortcuts and tags.

After only ten years practice, I can trace about one building every four seconds in a dense area and probably have deformed fingers from jumping between A to add, S to select, B to add a building, SHIFT to re-align (and no, I can’t remember the keys - they just happen). Unfortunately, I can see future archaeologists interpreting the pattern of muscle attachments in my hands as a sign of RITUAL behavior rather than Open Cartography!

After completing the survey of all streets in a 30km radius from my home, Microsoft announced in 2010 that Bing Maps could be used for tracing into OSM. This changed mapping from requiring direct GPSr field survey for ALL activities, to an iterative process of ‘armchair mapping’ using aerial imagery to rough-out road and building geometry first before capturing ground-truth, road names, actual position data, and gazetteer details with a physical visit.

Comparing the data captured with early Bluetooth GPSr (only capable of tracking 8 satellites) with modern imagery and Garmin GPSr, the results are surprisingly good. Sure, you can show spacial offsets using Trimble or Leica differential survey kit, but the features are mostly within the width of the road. Almost - smaller roads have less GPS data so are less accurate; wider larger roads have more traffic, so more traces, so more averaged centrelines.

Bing still has a lot more aerial imagery than myself (even after a DC-3 survey):

With my locality mapped, my pattern of mapping has changed from area survey to maintenance, adding new developments, checking Note questions, and occasionally wondering why my past self interpreted a trace in such an obviously wrong way (with the benefit of 20:20 high-res aerial hindsight!)

Re-visiting early areas based on ground survey before aerial imagery does highlight some common issues:

• Moving GPSr tend to use inertial averaging giving exaggerated corners. The impact is junctions can have poor geometry, with a straight 90deg T-junction skewed into a more complex Y or L, often reversing the ‘give way’ priority.

• Straight roads aren’t. Road engineers have big toys which don’t like corners, so roads like to be straight as long as the terrain allows it. Early GPSr and parked cars conspire to make roads more curved than reality.

Top Tip: The easiest way to map a straight road is not to put bends in it! Simply - take out unnecessary extra points, and line up the remainder (e.g. the JOSM L-key).

So, after ten years of GPSr field cycle survey, how should you commemorate the milestone?

Well, a year ago I decided to map EVERY DAY for my tenth year - and somewhat to my own surprise, have managed the feat. After almost 9000 edits and 620 traces, Pascal Neis excellent tool shows 366 mapping days in the last year:

My 2016-2017 Mapping History

As well as mapping from a bike, I have also captured images and tracks whilst on a narrowboat on the UK canal system. Using both Garmin GPSr, and a Raspberry Pi complete with camera (much cheaper than a GoPro), I decided to arm-chair map several journeys from Lichfield to Stafford, Stone, Stoke on Trent, etc.

I ended up adding both urban and rural detail from Lichfield, down the S&W past Penkridge, to Autherley Junction, up the Shroppie to Chester, then back to Middlewich, Kidsgrove, then up the Macc mapping Congleton, Macclesfield, Marple, and New Mills (to name but a few), before stopping at Bugsworth (AKA Buxworth - the canal is older than the road signs :-) ).

This has taken some creativity, from using iD on a Linux desktop, OsmAnd on a phone, and even JOSM running fine on a Raspberry Pi 3 with official touch screen and 4G dongle whilst on a train back from a business trip!

JOSM on a Raspberry Pi 3 on a Train

Top tip: Use a scroll-wheel USB mouse as a touch screen is just not precise enough when moving at 200km/h…

Remarcably, for an embedded ARM teaching board, JOSM is quite usable on the RPi3 even when mixing GPX survey data with aerial imagery - as long as you use TAB to hide the sidebar to make the most of the small screen.

Thanks for all the Nodes Ways and Relations!

So, here’s to the next ten years of OSM contributions both ground-truth and armchair, and thanks to all those other mappers, surveyors, wiki-fiddlers, sys admins, devs, DBAs, OSM Foundation, organisers, and not least our ex-BDFL who have made it all possible!

Where next? Well, there’s three on-going construction sites to check, and a new one reported via a note… better get the GPSr and bike out!

James