Recent diary entries
This month local mappers, in collaboration with our data team at Mapbox, completed the task of adding missing buildings in the SF area. Our sprint focused on the whole of SF Peninsula covering the areas of Sunnyvale, Mountain View, Palo Alto and Menlo Park areas covering ~1290 sq.km. This is a continuation of our earlier efforts from 2014 when we addded ~114,000 buildings in San Francisco.
Based on the existing building density, we estimated a total of ~180,000 missing buildings in the project area, and finally traced ~173,447 buildings over the last two months.
Methodology and Observations
- An initial trial run was conducted to determine a rough estimate of the amount of buildings to be added. It was crucial to gather an idea to timebox the task to a particular timeframe.
- Regular reviews were conducted to determine any gaps in coverage and accordingly create new tasks.
- There were outdated imports in the area which needed to be cleaned up. Accordingly clean up tasks were created for these areas. Since these misaligned footprints aren't due to any mapping efforts a separate ticket was created for the clean up tasks in mapping repository.
- Imagery is outdated in some cases, where new buildings have cropped up. One such instance was the new Apple spaceship campus in OpenStreetMap which was absent from Bing/Mapbox imagery.
Timelapse of building footprints added in the SF peninsula over the last 3 years
This task provided us with an opportunity to streamline our mapping efforts in the open and how we engage with the local community in the form of blog and diary posts. Feel free to join us in our current effort to clean up imported buildings in Palo Alto and Cupertino and also suggest other areas that require improvement.
Do drop into our mapping repository to track our latest OpenStreetMap mapping projects and provide your inputs in how we can be doing better. Happy Mapping!
So much has changed on #MapLesotho. I first joined training provided by Fingal County council in 2013 and the main focus of the training was on ARCGIS. However, so many changes had to be put into place as it became too expensive to maintain the software licences annually.
Now there is no need for software licenses as MapLesotho has moved into another phase of opensource. It now uses JOSM, Mapillary, Hotosm software that do not require any expensive license fees.
I only started mapping last year through my colleague and mentor,Tshedy Thobei,she showed me a lot of things about OpenStreetMap.
There were a lot of challenges along the way,we would not have enough data to draw due to lack of support from our superiors, we only had one laptop to use so you can imagine how challenging it was. Today is my second day of training in OpenStreetMap and I am glad that all of the things I learnt from her are now coming up in training.THANK YOU MADAM!!!!!!!
I first used opensteetmap in February 2015. There was a #MapLesotho Validation Challenge at the beginning of this year whereby I came in the first place and I won a Samsung Galaxy Tablet
I was given a #MapLesotho printed T-shirt yesterday and I walked down to the mall with one of my colleagues. Our T-shirts drew a lot of attention from the people we met. They might be eager to know more about openstreetmap and participate in #MapLesotho.
I first started using OSM to help #MapLesotho in June 2015 in the mapathon that was held in Lesotho Maseru Avani. All thanks to my instructors from Action Ireland and my colleagues in Lesotho, Lineo Mothae and Mats'eliso Letsie who introduced me to all the necessary steps to mapping. I am now a very happy mapper.
I have just joined the Fingal County Council #MapLesotho team in Maseru for this years training sessions. Fingal are arranging two weeks of training and workshops that will give a number of members in the #MapLesotho community better tools for mapping. One of the tools for improving the detail of the now almost completed base map of Lesotho will be Mapillary. I will be holding workshops explaining how to use Mapillary and there will be mapping events.
Flying in to Lesotho was beautiful - clear sky, mountains, fields and in the end these neighbourhoods surrounding the airport.
Starting with album of unofficial bicycle routes around Malacky. It can be found at http://rudava.mypage.sk , it is of course based on OSM data (and my paper notes) and drawn by hand in UMAP.
I'm expecting to slowly add new tracks as I find them and as I get contributions from other cyclist in the region. I found a local cyclist group "SCK Záhorák Malacky" which will have knowledge about local paths.
A new version of the routing engine GraphHopper appeared!
Read more here
Continuing from the previous diary post.
Last week, the quadrant routes (routes having 4-digit route numbers prefixed with
SR) in Pennsylvania were re-tagged by replacing the
ref key with the
Total routes re-tagged: 6787
Breakdown of re-tagged routes
ref:penndot=SR ****: 35
ref:penndot=SR ****: 3151
- This was a mass-edit and have been uploaded in one go
- After the edit was done it was noticed that in 29 of these routes
SR ****was followed by a non-state route number like
Historic PA ***
- This edit was done in 7 parts, each changesest containing the following number of routes:
- 32: In this case,
ref=SR****was changed to
ref:penndot=SR ****using the TODO list plugin in JOSM. Since changing the rest 3560 routes in a similar way would have taken a very long time, from the next step only
refwas changed to
ref:penndotby mass modification and
SR****was left as is
- 32: In this case,
- Also this time it was ensured that none of the routes of the order of
SRfollowed by a non-state route number were edited
- This edit was done in 7 parts, each changesest containing the following number of routes:
- Of these 6787 re-tagged routes, 1031 routes were found with the
penndot_reftag alongside the
reftag. Following the OSM Wiki guidelines, the
penndot_reftags were deleted for these routes.
- @rickmastfan67 pointed out that 4 nodes with exit numbers were wrongly re-tagged to
ref:penndotin this process. This change was reverted back here
- 37 routes were found where
SR****is followed by a non-state route number like
Historic PA ***
- 79 routes were found where a non-state route number like
Historic PA ***is followed by
For the above 116 routes, the
ref tags were not changed to
ref:penndot as we're unsure of the correct tagging convention for these cases. We would appreciate any input on what would be the correct way to tag them and if the 29 similar routes that were unintentionally re-tagged to
ref:penndot in the second step need to be reverted back.
I am in Maseru to train 14 members of the #MapLesotho OSM community. This is the third, and final year of the training. That is explained here here
So far the community members have decided that they will form three clusters of interest, and these are:
Training and Engagement - main mission, to make Lesotho be as mapped as it possibly can be, and ensure that lots of users are recruited and make a contribution to the map
Technical Quality - ensure all that is mapped is traced and tagged in an orthodox way to increase usability of the map
Spatial Analysis - will use the data as much as possible to investigate matters relating to the physical layout of Lesotho.
I will be sad to leave, but to see the OSM community become so focussed here is a wonderful thing for me, knowing that in some small way I helped it. Right now the guys here have the basemap for the whole urban area 1% from completion and 2% away from validation.
OSM, as a VGI mapping project, inherits various challenges: naive contributors, flexible contribution mechanism, and uncertainty of spatial data. The facts that rise subjective classification problems in the resulting data. Whether a piece of land covered by grass is classified as “park”, “garden”, “meadow”, or “forest”; whether a water body is classified as “pond”, “lake”, or “reservoir”. In OSM project, such of these classification answers are likely dependent on contributors’ perceptions. However, the appropriate classification of entity is strongly related to some qualitative observations and quantitative measures.
Thus, Grass&Green is a tool that has been developed to help contributors to assign the most appropriate classification of some entities of grass-related features. The role of contributors is needed not only in adding new data, but also to revise and confirm the existence one.
• The tool has an access and contribution to users accounts
• It has very supportive and easy to use interface
On the right hand, it presents the entity under investigation with some qualitative descriptions of its context. On the left hand, it show the OSM entity combined with some recommendations and suggestion, with full flexibility to update/confirm/reject these given recommendations.
• The tool provides the user with textual and visual descriptions of the target features and similar identical features.
Example of classification improvements:
o The next was only grass with fence, the contributors with our tool makes it "Garden" which is more expressive information than grass with fence.
o The next was labeled as "Park", however its characteristics is far from being park for amusements and entertainment. Out tool correct it to the reasonable classification as "Garden"
The status of contributions: We have around 250 participants from more over that 30 counties. They checked around 3000 entities in about 3 months. However, we are seeking for more confirmation and rejections. The data need to be check and we adapt the methodology of crowd-sourcing revision.
They are mostly agree or partial agree with our recommendations and working to improve the classification quality of these features.
So, please feel free to contribute and follow us on Twitter and Facebook. Send us your feedbacks. Every small contribution even for one entity would be appreciated.
There are some issues JOSM users experience which are not the fault of JOSM developers because Java/OpenJDK causes it.
You use Ubuntu or Debian an experience strange GUI issues at JOSM's tag editor and at comboboxes at JOSM presets? They look like this:
There is even a video of it.
There is a workaround. Thank you, Don-vip. Remove the line
/etc/java-8-openjdk/accessibility.properties (the file might be located at another directory if you use OpenJDK 7)
How I discovered the bug
I have been using Arch Linux for the last 18 months and started using Ubuntu (better: Xubuntu) on my new laptop. First I thought it were a KDE-related problem because I tried Kubuntu. A switch to XFCE did not fix it. Searching JOSM's issue tracker for open issues did not help because the issue was closed (that's ok, it's no JOSM problem). I tried an older version of JOSM but did not help. I tried OpenJDK 7 and 8, both behave the same. I asked at #osm-de IRC and malenki pointed me to the issue and the workaround.
In search for a dedicated car-GPS, 15€ for a used Garmin nüvi 205t seemed like a steal.
There is a lot of documentation how to get map data running on Garmin devices, but the maps made by Computerteddy and frikart.no were not playing too nicely with this device.
From the beginning:
First concern: Will it read a 32GB microSD. Oddly enough, it worked.
What is next: We get some well established OSM based map data, and good to go. In theory...
Let's see what isn't going so smoothly.
Say, we want to search for an adress, how about Lothstraße in München? After all, it is in the map data:
Turns out: no results when searching for it. Why? I can't tell you, really, but we can solve it.
Let's convert some maps in a way that works, tested specifically for the nüvi 205t:
- Get a map, for example germany-latest.osm.pbf from Geofabrik
- Convert the .osm.pbf to .o5m using osmconvert
osmconvert germany-latest.osm.pbf -o=germany-latest.o5m
- Filter with osmfilter to keep only boundaries in this new o5m file
osmfilter germany-latest.o5m --keep-nodes= --keep-ways-relations="boundary=administrative =postal_code postal_code=" -o=germany-latest-bnd.o5m
- Generate boundary files with mkgmap. We need these, because without them, there would be countless duplicates of the same city and the streets would be split into all these different "virtual" cities on the device, making it uncomfortable to search adresses
java -cp ./mkgmap/mkgmap.jar uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreprocessor germany-latest-bnd.o5m bnd
- Now we need to split the map with splitter, so it can be processed for the device
java -jar ./splitter/splitter.jar --output-dir=./splitmap/ germany-latest.osm.pbf
- And in the end, we convert the map for the device, using mkgmap again (The options --tdbfile and --make-poi-index might very well be useless, but this is the exact commandline i tested with success)
mkgmap --gmapsupp --country-name=Deutschland --country-abbr=Deu --bounds=bnd --drive-on=right --process-destination --process-exits --housenumbers --latin1 --index --location-autofill=is_in --make-poi-index --poi-address --add-pois-to-lines --add-pois-to-areas --tdbfile --verbose --route ./splitmap/*osm.pbf
- Copy the resulting gmapsupp.img and the osm* files to a folder called "Garmin" on a FAT32 formatted microSD
There were still two duplicates of some cities, but the one using the format "Cityname, State" contained >99% of the streets, and the one using "Cityname, Country" almost none, so it is easy to use the correct one when searching for adresses.
I am stil confused why there were no other reports about problems with the maps made by computerteddy and frikart.no yet, but if they show up, mkgmap is your friend.
Getting ready for Garmin Vbri camera.
Have a look at bing images I see five circle segments that can be footpaths on the ground. Any idea of what it can be? Directional antennae?
Aww — looks like Google Map Maker is editable again in Canada. It's got some restrictions now, including one that The Great Unwashed can't edit polygons any more (which, in my 'hood at least, means no more polygons nicked wholesale from OSM). But the main new feature is: “Top mappers in your country are now empowered to moderate your edits”. These Regional Leads bless your edits … once they get around to it.
Seems that most of the comments on the Canada forum are of the form “I added my business X months ago, why hasn't it shown up?”
Now, if only there were a thoughtfully moderated alternative to Map Maker out there … ☺
Trying to improve the commitment of new mappers and to help them overcome the obvious beginners problems when trying to map, the Dutch community (after discussion in the user-forum) started to welcome new mappers as soon as they had made their first edit (in the Netherlands) on the map. To find out who the new mappers were, I used this rss-feed, provided by Pascal Neis.
This welcome program started on the 1st of August of 2015 and continues to this day. It is run by me and as such is a one-man task.
During this process I became curious to the mapping behaviour of the mappers and started to collect some data about their activity:
- when did they start their user account?
- when did they start to map?
- how many edits did they do?
- and much more
Soon I realized that I needed more data (over a longer time span) to get a better insight and so I contacted Pascal Neis and asked him to provide me with the relevant data, dating from some years back. After some startup problems with the data - not all the mappers seemed to be present in the data - I started my research with a dataset that contained the following data:
- date of registration
- date of first edit in the Netherlands
- date of their latest edit
- number of changesets
The dataset I have used for my research contained 3205 mappers that have done a first edit in the Netherlands between 1-1-2014 and 29-1-2016.
On first inspection of the data, it surprised me to see that some mappers did their first edit 7 years after they had created an account! This, then, was the first thing to investigate: how many days (after registration) pass before the first changeset is created?
Next I investigated how many days passed before the mapper did his latest (and very often his last) edit.
We see that most mappers (77%) create an account and start to map immediately, but 4% of the mappers waited more than 3 years before they did a first edit. But it is striking to see that for almost all of those mappers (68%) this first edit is also their last! So called "hit-and-run" mappers.
"Last edit" is of course hard to tell, because they might return some day in the future and do another edit, but experience so far doesn't prove that.
Of course it is difficult to draw conclusions based on a rather small dataset, but it nevertheless looks not to far from truth to conclude that OSM mapping is basically in the hands of a small group of dedicated persons.
When are you a regular mapper?
If I look at my own status, I have got the label: "a crazy mapper", whatever that means, but once every three days (on average) I'm mapping: adding new things, fixing errors, searching for errors etc. But even if you add/change/correct things once every three months, you're a regular mapper.
The number of days since your last edit is a good measure of your status. See the next table:
This table shows that 138 mappers (4%) did edit something but did not return for a period of more than 730 days (2 years) after this edit. This is the maximum my dataset can reveal (because it spans 2 years and one month), and it is possible that some of those mappers will return in the future, but it is not very likely.
1411 mappers (44%) did their latest edit more than 1 year ago and still another 25% of the mappers did not return to mapping within (at least) 6 months.
One might say that for the majority of the mappers it is a one-time-only affair. Probably fixing something in their own area (missing names, shops, houses etc) and then never return.
A good measure of your mapping activity is the number of changesets you have done, and that is what is in the next table.
(Showing # changesets, # mappers in numbers and as %, sum of group left to it.)
This table shows:
1225 mappers did create 1 changeset
10 mappers did create (each!) more than 1000 changesets
And 82% of the mappers created between 1-9 changesets. From the graph it is obvious that this is almost a perfect example of an exponential curve.
In the Netherlands we have an active (albeit small) community of mappers and there is no indication that this community is different (statistically) from the complete set (2 000 000+) of OSM mappers (see links below), but it is also clear that the results that we get from the different datasets are not always easy to understand and only after at least one more year we might get some results that show us if the welcome program that we run in the Netherlands has improved the participation of the Dutch mappers!
In the wake of a serious act of driver brutality against a cyclist, in which the police were unable to specifically prosecute, I feel that it is not safe to cycle in the UK today, because the infrastructure that is required to enable safe cycling without a helmet or any of that extraneous gear is either not there, or it is at best too piecemeal to be considered "safe".
Having surveyed London streets for OSM, it is unsurprising that the London Cycle Network and its signage is far too piecemeal to enable safe cycling without a helmet or any of that extraneous gear: there cycling in the UK is not user-friendly but instead like playing Russian Roulette. It is unsurprising that Netherlands is so ahead of us in the provision of cycling infrastructure, to a point where helmets and other extraneous gear are a mere sports thing over there.
Therefore, I think that in the UK, the car and the bus is still the king. With all the aggression against cyclists in the UK (just look at all those videos on YouTube!), this is why I feel that it is not safe to cycle in Britain at all, and non-drivers may be better of with public transport, until the government provides the right infrastructure so that cycling is no longer like playing Russian Roulette.