OpenStreetMap

dónal's Diary

Recent diary entries

Productive day of mapping...

Posted by dónal on 6 August 2023 in English.

It’s a long weekend this weekend in Ireland so I found some time today to catch up on some OpenStreetMap tasks.

Primary objective: Send welcome messages to new contributors in Ireland.

Objective status: Completed. \o/

Other observations:

  • new contributors add lots of good information. some mistakes but we also did that at the start.
  • lots of buildings still to add in Ireland.
  • impressed by the amount of organisations trying to contribute updates. Saw new contributions from libraries, public transport organisations, business improvement organisations.
  • A new contributor with a username that may signify they work in the waste industry has been adding information for new estates. Love it!
  • Terracing tool rocks! Added a whole estate in Blarney, Co. Cork in about an hour using it today.
  • Map Roulette (see maproulette.org ) is cool.

Map of recent edits for dónal

Map of Ireland showing openstreetmap edits by dónal

Location: City Centre, Centre A ED, Cork, County Cork, Munster, T12 EK53, Ireland

OSM Ireland Map&Chat - July 2022

Posted by dónal on 21 July 2022 in English.

Quick recap from this month’s OSM Ireland Map&Chat.

Attendance: 4-5 contributors

Topics

Heritage trees in Ireland

Interesting to learn that the Tree Council of Ireland have a registry of trees and a number of trees have been classified as Heritage Trees. See https://data.gov.ie/dataset/heritage-trees-of-ireland

Brian H. wanted to know how to map these and it turns out there were no entries for heritage= so we fixed that by adding an entry for the Tree Council of Ireland. \o/

New building projects open on https://tasks.openstreetmap.ie/

The existing set of projects were mostly complete so Heikki mentioned that some new projects had been opened up. I started working on Leitrim since I know parts of the county quite well (lived in the neighbouring county of Sligo for a few years). Managed to get some easy tasks completed equating to 2% of the work. :)

Sledding hills in Toronto

Rob had a query about how to map a mound in Toronto he had seen. We concluded that man_made=mound was probably acceptable (some existing usage) but it turned out to be a sledding hill!! While there is no definitive tagging for these, we concluded that the tagging documented for Piste Maps is probably the best fit.

Location: Fearagh, Kilteevan Electoral Division, Roscommon Municipal District, County Roscommon, Connacht, Ireland

OSM Ireland Map&Chat - May 2022

Posted by dónal on 21 July 2022 in English.

At the OSM Ireland Map&Chat in May 2022, a query arose about how to create maps for specific topics (one of the Irish contributors wanted to callout some memorial benches in their local area).

As a result of the discussion, I created a umap project to demonstrate some of the possibilities:
https://umap.openstreetmap.fr/en/map/kerry-pike-gaa-memorial-benches_765585#18/51.91009/-8.56102

The overpass api query that is plugged into the “Remote Data” url field is:

https://overpass-api.de/api/interpreter?data=[out:json][timeout:25];(node[%22amenity%22=%22bench%22](51.909338348825024,-8.562796711921692,51.9115654863502,-8.5586017370224);%20way[%22amenity%22=%22bench%22](51.909338348825024,-8.562796711921692,51.9115654863502,-8.5586017370224););out%20body;%3E;out%20skel%20qt;

Note: can actually be simplified since we know all the results will be nodes (above has nodes and ways. An exercise for the reader…

Overpass Turbo also has the ability to do something similar (limited control over display format though):

Overpass Query (with MapCSS): https://overpass-turbo.eu/s/1iBL
Interactive Overpass Turbo map: click here

For those looking to understand MapCSS a bit more, see https://wiki.openstreetmap.org/wiki/Overpass_turbo/MapCSS#Color_Coding

Location: Coolymurraghue, St. Mary's, Cork, County Cork, Munster, T23 A406, Ireland

One-upping other map providers...

Posted by dónal on 27 June 2021 in English.

Changset ID # 107053042

Cork County Council in conjunction with TII (Ireland’s authority for major roads) are in the process of building a bypass past Macroom and Ballyvourney in Co. Cork, Ireland.

Recently, I was travelling back from Killarney and noted that the road had been re-aligned. I quickly captured some GPD traces and have now updated OpenStreetMap with the new roundabout and re-aligned N22 (the bypass will assume the N22 name I presume when it opens).

This was a relatively nice edit to undertake. While I only had one trace to work with, it was good enough to get a rough idea of what needed to change. I also had to amend the N22 route relation(s) - there were two; one in each direction. I didn’t have a whole lot of experience editing these types of relations but the josm interface was intuitive.

As always, I wanted to validate my work (especially as I was editing a primary national route in Ireland). ra.osmsurround.org is a great tool for that so I checked both relations for issues (see relations 3147725 and 558006).

I expect more changes in the coming months as work progresses on this road.

Finally, I checked to see if other map providers had picked up on the change. Funnily enough it appears that Google have picked up on it (possibly using data from Google Maps’ users) but in much lower detail. I guess their algorithms detect something is up but don’t know how to deal with it properly yet. 1-0 to OpenStreetMap!

p.s. Driving round the roundabout 3-4 times was quite fun and a great way to get a good trace for it.

Images

Roundabout on OpenStreetMap Roundabout on Google Maps

Location: Carrigaphooca, Kilnamartery, Municipal District of Macroom, County Cork, Munster, Ireland

I have a few passions in life. Two of them overlap in an interesting fashion…

A number of years ago, I decided to look at how engineering (and specifically full-lifecycle analysis) could support improvements in road safety. While many road safety issues are primarily behavioural (speeding, awareness, road rage, etc), I strongly believe that infrastructure should guide road users to making better decisions over time through feedback, reinforcement of good behaviours, etc. Anyway… I decided to buy a dashcam and figure out if there was a way to collect data in a way that would provide strong arguments for change at both a local and national / international level. Due to other commitments and a lack of interest from the general public I haven’t made much progress. :(

My other rationale for getting a dashcam was to allow imagery capture to support mine and others’ OpenStreetMap activities. I drive quite a bit on my way to climb mountains and explore the south coast of Ireland (sometimes to play MMORPG Ingress). Capturing imagery and contributing it to the global community seemed like a good way to get additional “mileage” out of my travels!

But dashcams aren’t setup for sending data to Mapillary I here you say… True… but I never give up!

The OpenStreetMap Ireland community recently decided to capture imagery and authoritatively map the #WildAtlanticWay, a 2500+ km route along the west coast of Ireland. While there is some existing imagery of the route, there are some notable gaps and we wanted to fix that.

Being one of the primary contributors in Cork (Ireland’s largest county by land area), I was motivated to try and contribute more imagery in Co. Cork and Co. Kerry. So I started digging into the tools available for processing and uploading data again (to see what improvements had been made since I last tried).

Processing dashcam videos so they are usable my mapillary

I quickly came across this discussion on https://forum.mapillary.com/ which indicated some folk had make progress extracting imagery from a Viofo A119 dashcam. I had a slightly newer model (the A129) so I was hopeful that the code would work. I updated my dashcam firmware and captured some new video. When I got home, I tried processing it but got timestamp decoding issues (the data offsets for the A129 were different (see here)). Fortunately, one of the project contributors was able to decipher the offsets and provide a patch that would work for the A129. Wohooo… I tried it with some more video I had captured and my first set of JPGs were created! \o/

Uploading to Mapillary (part 1)

Next up, it was time to upload the imagery to Mapillary. The acquisition of Mapillary by Facebook has been well documented and the team have not been slow at pushing forward with their plans. This has impacted imagery contributors and the OpenStreetMap community considerably in the past couple of months as Mapillary update their API and rollout out some new processing pipelines and features. Unfortunately, this also meant that most of the opensource ecosystem was broken too. Ahhhh!

Fortunately, I found the mapillary/mapillary_tools repository (official tools maintained by Mapillary). But I was running into issues. My password needed to be changed because of the recent acquisition and after that I couldn’t log in using the python tools on linux. Was there lag with the password change replicated to auth servers? Was my account broken?

I could not determine the cause so I filed an issue.

Again, I got a response fairly quickly (not bad for a friday evening PST time!!). It wasn’t me ( \o/ )… it was a bug. API v4.0 was rolling out and the tools didn’t support it (yet). A quick commit by the developers and a new release (v 0.7.0) and authentication was working for me. Wohoo…

Uploading to Mapillary (part 2)

The moment of truth had arrived. My images were ready. My Mapillary authentication was working. It was time to upload imagery…

$ mapillary_tools process_and_upload --import_path "../random-mapillary-stuff/output" --user_name "donal"
Running user_process for 415 images, skipping 0 images.
Sub process ended                                                                                                                                                        
Running import_meta_data_process for 0 images, skipping 415 images.
No images to run import meta data process
If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun
Sub process ended                                     
Running geotag_process for 0 images, skipping 415 images.
No images to run geotag process
If the images have already been processed and not yet uploaded, they can be processed again, by passing the argument --rerun
Sub process ended                                      
Running sequence_process for 0 images, skipping 415 images.
Sub process ended
Running upload_params_process for 415 images, skipping 0 images.
Sub process ended                                                                                                                                                        
Running mapillary_image_description for 415 images, skipping 0 images.
Sub process ended                                                                                                                                                        
Process done.
Uploading 415 images with valid mapillary tags (Skipping 0)
Compressing 415 image files to /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip
Writing /home/dhunt/src/random-mapillary-stuff/output/2021_0610_194450_037F.MP4_000000.jpg to /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip/2021_0610_194450_037F.MP4_000000.jpg captured at 2021_06_10_20_44_50_000
Writing /home/dhunt/src/random-mapillary-stuff/output/2021_0610_194450_037F.MP4_000001.jpg to /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip/2021_0610_194450_037F.MP4_000001.jpg captured at 2021_06_10_20_44_50_333
Writing /home/dhunt/src/random-mapillary-stuff/output/2021_0610_194450_037F.MP4_000002.jpg to /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip/2021_0610_194450_037F.MP4_000002.jpg captured at 2021_06_10_20_44_50_666

=== snip ===

Uploading /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip (187423362 bytes) ...
Finishing uploading /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip with file handle 

=== snip ===

Cluster 2880127168972143 created
Deleting /home/dhunt/src/random-mapillary-stuff/output/sequence_b0645290-3b0a-41bc-903b-961b8b95699f.zip...
Done uploading 415 images.
No post processing action specified.

and now we wait for the imagery to be processed by Mapillary… =)

p.s. Next goal is to process the rear camera such that Mapillary knows they are pointing the opposite direction to travel. Wish me luck!

Lessons learnt

  1. Try new things… Even if they didn’t work before, maybe the ecosystem has matured and your odds are better.
  2. Don’t give up… Ask for help… File issues.
  3. Document your findings. Inspire others. Guide others in the right direction.
  4. Be grateful for those that help you. Give back in some way.

References

If you are having auth problems with the mapillary_tools toolchain, the following command is helpful for doing a complete recent.

$ mapillary_tools --advanced authenticate --force_overwrite --user_name "<username>" --user_email "<email>" --user_password "<password>"

Tech specs

  • Vehicle: Volkswagen Passat B6 2010
  • Dashcam: Viofo A129 Duo Dual Lens Dash Cam Full HD 1080P 140° Wide Angle Front and Rear Dashboard Camera w/GPS WiFi, Parking Mode, Supercapacitor, Low Light Vision G-Sensor
  • Laptop: Lenovo Thinkpad Carbon X1
Location: The Marsh, Centre B ED, Cork, County Cork, Munster, T12 WE28, Ireland

Whitechurch, Co. Cork map update

Posted by dónal on 6 December 2020 in English.

Whitechurch map before... Whitechurch map after... Changes displayed on osmcha.org















Yesterday, I finally found some time to complete some surveying in my local village.

My plan was as follows:

  • Prepare for survey of house names and numbers
  • Complete a survey of the village to capture notes
  • Add relevant data to the map.

Preparation

I’ve been aware of the excellent Field Papers project for some time but despite some earlier plans, I hadn’t managed to complete the complete flow. For those that aren’t familiar with the software tool, it allows you to print paper maps, capture notes and then scan and bring the data into your favourite editor for further processing.

For this effort, I choose a 3x2 (HxW) A4 format (six sheets) that covered the core of the village while ensuring the various estates were mostly on the same page (link). I also created a 1-page atlas for Barleyfield (link) which was just a little bit too far west.

Surveying

Normally for these types of efforts I would hop on a bicycle and survey with zero emissions but today I had our youngest with us so I opted for the car. I worked through the area methodically capturing house numbers, street names and other useful data (e.g. house outlines of terraced buildings, etc).

I did discover that many of the housing estates in Whitechurch are a work in progress! Most are half built or are missing houses. The initial layouts are there with lots of potential for infill development / completion of the planned estates. My searches online indicated that some of the developments have changed hands over time which probably explains the lack of progress.

Every now and then I stopped to engage residents regarding bits of information that were not visible (houses with missing numbers, street names). No raised eyebrows but I suspect longer conversations would have been had if I was on foot or bicycle.

The survey took about an hour to complete despite all the little cul-de-sacs and other permeability features.

Updating the map data

Once I was back home, I scanned in my completed sheets (I have a mullti-functional printer with ADF which makes this very easy). The android app allows me to upload the files (JPGs in this case) directly to Google Drive. From there I uploaded them to the Field Papers website and waited for them to be processed. Once processed I was able to import the data as raster layers in JOSM.

Top tip: If you have multiple sheets, import all the sheets and then expand the area that you have pulled OSM vector data for.

Once in josm, I started the process of creating building outlines, adding green areas, road names, driveways and missing pedestrian only ways. This was time-consuming but taking a methodical approach worked well. I found that drawing outlines and driveway ways first and then going back and tagging in batches saved some time and effort.

Top tip: The terracing plugin is really useful for semi-detached and multi-home buildings. I found that creating a rectangle closed way, splitting it and then using the extrude tool resulted in the best results. I later discovered that the tool will also add addr:housenumber and addr:street tags if you can reliably predict the numbers. Nice!

Note: I also had to move the outline for most buildings due to the imagery used.

Validation

No editing session is complete without validating you haven’t done something silly / dangerous. I fired up the validation tools in JOSM and started working through the most obvious errors.

Errors identified:

  • untagged ways. how embarrasing!
  • buildings with unknown tags. this was mostly me making mistakes when adding house numbers.
  • overlapping ways. this seemed to be mostly a result of the terracing tool. I’m not sure why some of the results ended up like this.
  • overlapping ways and buildings. This was mostly buildings overlapping with landuse=* areas. Small adjustments in most cases.
  • houses with no addr:street=* tags.
  • end of a way close to another way. This happens quite a bit with driveways and in housing estates that have roads that don’t connect. I added noexit=yes tags where I felt it was appropriate.
  • highway=* ways that had no name. The majority of these were either under-construction roads. One tertiary road has no name or ref so will have to go searching for a suitable source.

Someone along the way, my keyboard stopped working in JOSM. I’ve been getting some errors in JOSM over the past few days. Appears to be a known issue so hopefully it gets fixed soon. I was able to work around it using the mouse and menus fortunately.

Tidying up

I was ready to submit the changes. but…

NO KEYBOARD!!

How does one write a good changeset comment without a keyboard?? (hint: you don’t!).

I managed to save the session and layer I had been working on. I closed JOSM and re-opened it to find my data was all good. PHEW

Unfortunately, in the meantime there had been a small edit in the area. I walked through the conflict resolution process and fixed up the map to account for the changes.

Changeset submitted. Woo!!

A few refreshes of the map and the results were visible.

At this point I noticed a few minor errors and a few missing buildings (not in my plan but it annoyed me enough to add the building outlines).

Then is was off to bed to find one of our kids with a 38.5°C temperature. 2020 just keeps on giving!! 🤣

Lesson learnt

It was nice to find a nice block of time to do some detailed mapping. Most of my mapping is bits and pieces that I either observe on my daily travels or quality improvements based on map usage.

The effort covered:

  • Lots of terracing. I feel much more comfortable using this feature now.
  • Extruding lines. I had never really used this feature but had heard many mention it. Found it easy to use once I understood what it did.
  • Joining overlapping areas. I had never used the feature but found it quite useful for some of the more complicated building outlines and the residential landuse areas that needed to be joined up.
  • Naming of housing estates. I wasn’t really sure how this worked but the landuse=residential wiki page had good guidance and it was easy to adjust the existing landuse=residential areas to match the local names.
  • Usage of the landuse=greenfieldlanduse=brownfield
  • Saving your work locally so you can resume your session later.
  • Merging upstream changes into your local layers including dealing with conflicts.

metadata

Changesets: 95345380, 95352956, 95353202, 95353233

Imagery: Bing (majority of buildings), Maxar Premium (newer buildings missing from Bing)

Time taken: 1 hour (survey) + 6 hours (data processing)

Location: Castle Court, Whitechurch, Municipal District of Cobh, County Cork, Munster, Ireland

As businesses start re-opening, local authorities in Ireland are supporting businesses to adhere to government health advice and specifically social distancing techniques which have been shown to slow the spread of the disease.

One of the most visual supports is the pedestrianisation of streets for specific hours of the day / week so cafés and restaurants can make use of the outdoor space (risk of infection is lower if people are eating outside). Unlike continental Europe and parts of the USA, pavilions and street dining is not common here.

OpenStreetMap contributors are keeping up with the local authority efforts by updating map data as changes are made.

Most of these updates make use of the conditional access key/values outlined in https://wiki.openstreetmap.org/wiki/Conditional_restrictions

Here’s an example change in Cork, Ireland that I did earlier today: https://osmcha.org/changesets/87507009/

Note: the logic is that motor vehicles can’t access the street UNLESS it’s between 17:30 and 11:00 (i.e. the inverse of the pedestrianisation period).

Location: City Centre, Centre A ED, Cork, County Cork, Munster, T12 EK53, Ireland