Recent diary entries
Improving the OSM map - why don't we? (13)
Why so many people are not using OSM.
Do you recognize the renderer that was used for the above screenshot? I'm pretty sure you can't. Because it wasn't rendered but printed in the Times Comprehensive Atlas of the World.
Looking at this map, it is clear (at least to people who are familiar with "paper" maps ) what we see:
* A number of Islands that have a name as a group (Canary Islands) that are part of mainland Spain
* Each Island has its own name (printed in italics or bold italics)
* Each Island has a capital (printed in bold, but this is not true for all the Canary Islands)
* A number of towns is printed in normal type
Now let's see how OSM based maps and renderers show this to the world.
The same Islands, with three different ways of rendering (Humanitarian, Mapquest and Mapnik).
The most striking omission (to me) is, that none of the islands is shown with their name and Mapnik shows every Island as "España".
Even when zooming in, the names of the Islands never (and I mean NEVER) show up (I'm supposed to see Tenerife now somewhere on the map):
Now, what kind of a map is that?? Not showing what is most important!?!
Is there a road (top picture) running from Santa Cruz de Tenerife to España?
When I use a (printed) map or atlas, I can see at the large scale maps (1:500.000) what I need to find my way. At that scale I'm not interested if there is a paved/unpaved way ahead of me. And even less I do care about the traffic signs that I might see once I'm there.
I know that everything I'm looking for (and much, much more) is in the OSM database, but why is it shown to me at the wrong moments (if at all) and at the wrong zoom levels?
BTW, Google maps is not doing much better than OSM, showing (some) Island names at high zoom levels.
Do I use OSM myself? Yes!! All the time, and because I have learned to ignore all the crap it is giving me (like showing me the map in Chinese when viewing China, even if English is the language I have installed as my basic language), and because I know the strength it has with the right tools, to me it is the perfect map.
But to a lot of people who are used to a regular printed map or to Google, OSM is just a funny experiment that you can't even use decently on a mobile phone.
Of course, there are tools and apps that use the OSM data in a much more user-friendly way (especially on mobile devices), but why can't openstreetmap.org be a bit more user friendly?
One more example of the incompleteness of OSM.
In the part of the map I'm showing you here, we are supposed to see the capitals of the UK (London), France (Paris), Belgium (Brussels), Luxemburg (Luxemburg) and Holland (Amsterdam). Can you spot them?
Even worse, at this zoomlevel the only capitals shown on the map are London, Dublin and Budapest!!
Madrid, Paris, Brussels, Amsterdam, Luxemburg, Rome, Vienna, Berlin, Oslo, Kopenhagen, Stockholm? What?? Where?? Are they gone??
Friends I'm trying to move over to use OSM (by pointing them to my own openpoimap, I often tell, they call it a "slippy-map", but you better consider it a "shitty-map".
I hope that the developers who are working on the way the basic map is being presented to the users, read this and try to create a map that users recognize instead of being puzzled.
I have said it before, at this moment OSM is a map for mappers, not for users.
 Of course there are people who have never seen or used a printed map before, and for those OSM is maybe a great tool, but I doubt it.
I did it !!!! I finished tracing Carei town ! Please check out my upload !!
When it comes to visual map styles, there are always some people, who worship paper maps. In case of United States, it's about USGS topo maps, in the UK it's about Ordnance Survey, in case of Russia - about Soviet military topo map (slang name "Genshtab", which means "General Command"). These people always think, that symbology of these maps is something absolutely superior, because professional cartographers were working on it for many years to make it effective. Which is true, but barely relevant to digital cartography.
Here are several reasons why this extrapolation is wrong. (For some people it's an obvious thing, but I just like to have it here in written form to be able to refer to this diary entry in case of discussions and arguments instead of typing it every time I need it.)
Any professionally developed map style is always based on several factors: main use of specific map type, map scale (level of details), media (paper or screen).
Exactly by this reason, there are many different professionally developed standards for civilian topographic maps, hydrographic maps, military maps, aeronautical charts (and they are different, depending on altitude), marine nautical charts. In certain cases, for example - in case of nautical charts - IHO standards are universal (see IHO download page, especially - S-4 document), since it's considered safer to have uniform styles.
Map media played huge role in case of printed maps. It was too expensive and hard to use high-resolution full-color printing before. Low resolution full-color printing is completely unacceptable for maps, since large raster dots distort small objects easily. That's why paper maps were (and still are in some cases) printed with so called "spot colors" - inks of predefined colors (like black, brown, blue, green, orange, red, violet). And that's why paper map styles were developed having this and only this technology in mind. At certain grade, it was true for the first digital maps too, because transflective color LCD screens were far from ideal in color reproduction and their real usable color palette was limited by less than sixteen colors. Everybody who tried to make own styles or maps for those very first Garmin or Magellan GPS devices with color LCD screens should remember that perfectly.
But modern display screen is a high contrast full-color output device, which is at least two times better in terms of contrast than common printed matter. Therefore, it is easy to use more colors and shades for map style without compromising readability. There is no risk of bad color registration (when outlines of different colors are shifted relatively to each other due to bad printing machine setup), so we can use really thin lines, if we want to. Since high quality printing became more available, technical limits of printing process are also way wider now than it was back in fifties, when majority of paper map standards (which are still in use) were initially developed. You can check out Spesifikasjon for skjermkartografi - specification for Norwegian maps, issued by national mapping agency. It's perfect example of more or less full usage of color for different map features, which allows to put more information on map.
Purpose of any map style is the most important factor for its development. It is more obvious for people, when we comparing topographic and nautical charts, but somehow it's not that obvious when it comes to military and hiking maps. Army has own tasks we all know about. That's why Soviet military maps have different symbols for buildings made of wood and stone, because stone houses are fireproof. Same thing goes to roads - they are classified by several factors such as surface and width to give officers an idea, if particular road can be used by tanks and what will happen to it after they pass there. Is it enough for hiker or bicycle rider? Most likely, no. But it's acceptable in case if there is nothing better (like in case of Russia).
To get some idea about maps style, designed for outdoor use by pedestrian, anyone should check out International Specification for Orienteering Maps and some orienteering maps available online. Orienteering maps are perfectly readable, but in the same time, they giving you huge amount of information about landmarks, terrain, vegetation and other things in very dense "package". They even showing features like "wood, passable in one direction" (dense planted forest), "crossable and uncrossable pipelines" and many others. Is there anything like that on maps, intended for tanks? Obviously, no.
Sure, anyone is free to use whatever map one likes and to reproduce its style in MapCSS, SLD or QML, but let's avoid doing it blindly or based on limited knowledge.
I attended the CubaConf 2016 - the first international free software conference of Cuba, which took place in Havana. It was amazing to be part of this event and share knowledge about OpenStreetMap with Cuban people and with attendees that came from 17 countries around the world.
Our OpenStreetMap activities started two days before the CubaConf. On the Saturday, April 23, PB Echeverría, a experienced Cuban mapper and one of the organizers of CubaConf, gave a talk about OpenStreetMap in the FLISOL (a free software event that happens every year in a lot of cities in all Latin America).
In the following day, we had a Bike Mapathon. PB, Mauricio Miranda (OSM Argentina), Julio May (OSM Costa Rica), me and some other Cubans and Colombians took bicycles to ride around the El Vedado to map points of interest and make photos for Mapillary.
Monday was the first day of CubaConf, although we didn't have activities about OSM, I could meet Laura Barroso. She is the translator of WeeklyOSM to Spanish and is a software developer as well. I also could meet Orkut Murat, a Turkish OSM mapper. Orkut gave a talk about GNU Milion, a drupal extension he is developing to manage GIS data in Drupal.
In the second day of the conference, we had three talks about OSM. I started the session presenting an introduction to OpenStreetMap and showed some projects of the Latin America communities. Ivan Terceros showed the efforts to map Ecuador after the Earthquake. Finally, Julio May talked about how OSM is being used in Costa Rica to map areas that are in risk of flooding.
To finish our OSM activities, we had a Mapping Workshop in the last day. Due to the restrictions in the internet access in Cuba, we didn't have internet in the place of the Event, so it was an offline workshop, but it worked very well. I prepared a lot of screenshots showing how to map using Maps.Me and OsmAnd. I also had the source code of iD editor in my computer, so I could run it offline and show the begginers guide that is included in iD. Furthermore we showed how to start mapping using JOSM: the main shortkeys, how to load photos, GPX files, etc. And we talked about FieldPapers as well!
You can see more photos of CubaConf on flickr.
Digismartek provides you digitization Services like document management system, Electronic management system,document scanning services. We provide the best dezitization services.
I got involved with OSM back in January. Now its been 4 months and I think its the right time to make an introductory post.
I started looking for interesting projects, and iD caught my attention. The first bug that I fixed was pretty simple, It required me to fix a faulty regex. D3 was new to me, as I come from the fancy React/Angular land which obscures most of the internal working. My mentor Bryan was more than happy to help me out. After my first week I discussed with him about the possibility of an OSM GSoC on #dev channel. Which brought up a lot of heated conversation regarding the pros and cons of GSoC for the organization. The channel did admit that overall GSoC 2015 was a success. Which got me pumped up to spend my summer working on iD.
While this was happening I also started going through OSM wiki/diaries to learn mapping. I planned on mapping my college. My college and its surrounding had a spotty coverage. Since, I was a newbie I tried to map things I was most confident with. In India we call dormitories hostels, so I started marking my college buildings with hostel tags. When I showed my work to Bryan he was to quick to point out that might not be the case everywhere. He suggested me to tag all the buildings with
University Building tag.
This is what my college and surrounding area looks like now.
One of the best things that contributed to iD was the multiselect feature, which I implemented with the help of Bryan. This feature took a lot of days for me to implement and gave me a good understanding of how iD works.
Another exciting feature which helped me build confidence with the iD code was the imagery offset. The earlier implementation of imagery offset was unintuitive and required a fix. This feature request required me to fix the UI and to make it easy to use. Know more about this feature, click here.
I will continue with my investigation on the lane editor for GSoC. And learn more about D3 to fix some of the pending bugs of iD. I plan on writing about my progress weekly from now on, so that the community knows where I stand and give me inputs on how to improve upon my work.
For years I've been meaning to try out Mapillary properly, but my phone is broken and can't connect to wifi, so no upload.
But this was another thing I got to do while on holiday in Brazil. I persuaded my wife to install mapillary on her phone, and we mapillarised while we were driving around her home city of Guarulhos. She's pretty bored of all this mapping stuff, but likes to keep me entertained while we're in Brazil, and maybe she likes the idea of mapping her home town too. So we got lots of photos around these colourful Brazilian streets.
Also while going on a trip to the seaside, but sadly I don't think I got much of a seaside view in any mapillary photos. Some are quite nice and jungly though
And on the way there my wife wasn't amused when we got massively detoured in a drive around the city of Mogi das Cruzes. It was only semi-deliberate :-)
On the site you can see what proportion of streets have coverage (green) on a map display like this. I guess Mapillary coverage is expanding, but there's a long way to go. Hopelessly incomplete... but maybe if enough people join in, we'll have street-level photos of every street in the world. It feels like exploration. Conquering the territory, or laying the first tracks to form a skeleton that others will build upon. It feels like the early days of OpenStreetMap.
I was impressed by the ease of photo data gathering with the app. It's a very passive process but the app gives satisfying flashing counter as it gets photos. It detects when you stop moving and stops taking photos. The messages makes it nice and clear what's going on.
We were driving. I'm not sure how well it works with pedestrian tracing. By fortuitous coincidence, I was gathering up freebies at SmarterTravel Live conference, a few days before heading to Brazil, and one of the freebies (courtesy of dat mobility. Thanks!) was this simple windscreen mounting sucker thing. Perfect for mapillary tracing!
I was also impressed by the ease of upload. (Given a wifi connection) it's very simple to upload this quite hefty amount of data to the mapillary servers. There's then a delay while mapillary crunches the data before you see your green line on the map. I notice there's then a further delay before the images get stitched together with the animated zoomey effect. Understandably because it's a huge amount of processing, which no doubt gets queued up along with everyone else's collections of hundreds of photos.
But what about using Mapillary for contributing to OpenStreetMap? When pondering OpenStreetMap workflows, I see a sliding scale of more or less passive data collection. Mapillary is hugely passive, which is great news when it comes to quick effortless surveying, but the flip-side of that is, you leave a lot of work to do later on when you're back at the computer. I did a bit of this last night. Looking through the places we'd been driving around Guarulhos, clicking around with the mapillary JOSM plugin. Click next photo -> next photo -> next photo, which is essentially like driving along the road all over again, but this time I'm scrutinising the photos for anything I can add to the map. Probably a lot slower than the drive itself (depending on how much stuff you see which needs adding). In other words, after all that very simple passive surveying, I've left a lot of work to do later. A hopeless amount really
Being in Brazil my photos are all super-sharp in the harsh sunshine, however I found it pretty hard to identify Brazilian highstreet shops. In the more shabby areas the shops are not clearly branded or labelled (akin to the shabby north London shops all around me), and I often looked at translating a shop sign which turned out to be a billboard/product advert. Slow going, and I probably won't get around to examining all my Brazil mapillary traces for data to add to OSM.
But being shared with everybody, a mapillary trace is a resource which has the potential to be super-useful for other OpenStreetMappers. I've already used other people's mapillary traces in London on a few occasions e.g. puzzling over particular data problems from notes or other QA tools. But mostly I've found myself wishing there was more mapillary coverage. When we get to the point where we can back-up all of our data fixing with both aerial imagery and street-level photos, Mapillary will be quite something! ...might have to borrow my wife's phone!
Q: Who are you ?
I am Pete Masters (OSM: pedrito1414 - a hangover from my time living in Mexico). I live in Glasgow and work for MSF, running the Missing Maps project. I am a fairly rubbish climber, a regular footballer and get out to the lochs and mountains as much as possible.
Q: When and how did you discover OpenStreetMap ?
As a user. I have travelled extensively with my various jobs and OSM and associated apps are indispensable. I can never understand why someone heading abroad wouldn't have OsmAnd or similar on their phone!
Q: What do you map ? Is there any difference with your early days ?
As a contributor, I guess I am a relative newbie. My mapping started with HOT mapping and then Missing Maps of course. The biggest difference? My eyes. It continues to amaze me the level to which we can train our eyes to pick out detail in satellite imagery....
Q: How do you map ?
In the field for MSF, I use OpenDataKit, OsmAnd, field papers and occasionally OSM Tracker, OpenMapKit. I do a lot of tracing, too in JOSM. At home, I do a little bit of surveying - mostly alterations and corrections I notice while navigating or leisure mapping while on holiday....
Q: Where do you map ? Locally, HOT ?
A little bit locally, but mostly in the field.
Q: What is your biggest achievement as mapper ?
I have mapped in Congo (DRC), Central African Republic, Bangladesh, amongst others, but the experience of Bangladesh was pretty special. Working with Jorieke Vyncke and incredible OSM Bangladesh community was an absolute pleasure. We worked hard, we had a lot of fun and we did the job that MSF needed us to do. Very proud of that. Also, massively proud of the heights that OSM Bangladesh have since reached - I'm a big fan ;) With the uni of Lubumbashi guys in Congo
Q: Why do yo map ? What motivates you ?
I have worked for MSF for a long time now and love the organisation. That I can practically contribute to their mission is extremely satisfying and motivating. I also find helping responders through the HOT activations following disasters very rewarding. I guess this is also the first time I have been a part of such a talented, committed and interesting community of people, too. Remotely, through conferences and meetups, and through the London and Glasgow mapathons, I have met brilliant individuals and made some good friends....
Q: What is the most difficult part of mapping ?
For me, it's not the mapping - it's the organising. Trying to make sure that data quality stays consistently high, while at the same time bringing in new people to OSM and HOT is a huge challenge. We are getting better, but there is a long way to go and I welcome ideas that anyone might have....
Q: What are your mapping plans for the near future ?
Q: Do you have contact with other mappers ?
All the time! I go to at least a couple of mapping parties a month in various cities. And, I hope you don't mind, but I'd like to mention Nick Allen and Ralph Aytoun by name. They may be surprised, but I see them as my mentors in all of this - knowledgeable, incredibly generous with their time and always available and patient for my (never-ending, sometimes-foolish) questions ;)
Q: Do you use OpenStreetMap yourself ? How ?
As above... In the field, both for personal travel and work.
Q: Can you tell us a bit more about "Missing Maps" ?
The Missing Maps is a project made up of numerous organisations and thousands of individuals. Each one has different objectives, resources, skill sets and each brings something different to the table. The openness of that table is massively important and we try to make it as welcoming as possible to people who want to pull up a chair. We unite around a common humanitarian desire and that's the fuel in the fire.
The project was born out of a need. A lack of geo-data in the places where MSF teams work - places that rarely make the news. The volunteers who engage and contribute are helping to meet that need and directly supporting life-saving medical work in some of the most dangerous and vulnerable places in the world.
While the Missing Maps project wasn't my idea - I was there at the start. I named it ;)
Q: Do you do anything else than mapping that is related to OpenStreetMap ?
I still love to use paper Ordnance Survey maps while hiking.
Q: To conclude, is there anything else you want to mention ?
Just to say that this last couple of year's immersion in OSM and HOT has been an education, an experience and a pleasure. Long may it continue!
Thanks a lot Peter for taking the time for this interview.
The previous interviews can be found on the wiki
Another round of Ordnance Survey data has been released and processed by ITO! to produce a very useful comparison against OSM. Once again, their very usable web slippy map, and JOSM layer this has shown up a new building site I didn't know existed ready for survey.
As well as missing developments, the comparison also showed up a fat-fingered typo from one of my past surveys - quickly fixed!
As usual, don't assume OSM mappers are any better or worse than the professionals - OS data feeds from local authorities don't always match the 'ground truth' signage and so don't accept the OS Locator data has to be correct without a check particularly if OSM is marked source=survey.
If there's no signage out there to contradict or correct, then an attributed name from OS is better than none (source:name=OS Locator). If Locator is wrong, use the tag 'not:name=' to show the issue to comparison sites like ITO, and also to OS so they can check themselves.
So once again, raise a glass to OS Open data, ITO for the service, and the local groups of fellow mappers slowly increasing the comparison UK Percentage Complete beyond the current 97.60%.
Help turn all UK counties UK cyan on the overview map - a 100% match!
On many occasions in the past I've said "it's really easy to create your own Garmin maps" but I don't think I've ever documented the procedure. This is an attempt to do that.
You've got some sort of Garmin device with an SD card in it with some maps that you've downloaded from somewhere on it.
You want to replace those with a map you've created yourself.
You've got a PC with around enough memory in it. The one I used for the examples below had 4Gb; it's possible that less may be needed for a small file such as the one used in the examples below.
All the path examples below for for Windows (actually tested on Windows 7), but it should be relatively straightforward to adapt them to work on other operating systems.
What you need to do
The easiest way to create Garmin maps from OSM data is to use "mkgmap". That's available from:
The download page for the latest mkgmap is http://www.mkgmap.org.uk/download/mkgmap.html and the current version is "3676".
Unzip the zip file and move the folders intact to a known location. In my case I moved it so that "mkgmap.jar" file was in: C:\Utils\mkgmap-r3676
I didn't use a path with a space in it or a path that Windows recognises as "special" to avoid any possible confusion later.
"Splitter" is a separate program used to split a large OSM data file into portions small enough for mkgmap to work with (the resultant Garmin map can still route between portions).
The latest Splitter is here: http://www.mkgmap.org.uk/download/splitter.html and the current version is "437".
Unzip the zip file and move the folders intact to a known location. In my case I moved it so that splitter.jar was in: C:\Utils\splitter-r437
Next, we need to download an OSM extract to use to create maps from. As an example I'll use Romania, and extract for which can be downloaded from here:
That page currently says:
romania-latest.osm.pbf, suitable for Osmium, Osmosis, imposm, osm2pgsql, mkgmap, and others. This file was last modified 7 hours ago and contains all OSM data up to 2016-05-13T19:45:03Z. File size: 146 MB; MD5 sum: b0554ae9632cc6c2be1c1e3525b5dcb5.
After downloading, check that the file is intact:
md5sum romania-latest.osm.pbf b0554ae9632cc6c2be1c1e3525b5dcb5 *romania-latest.osm.pbf
(see links from https://en.wikipedia.org/wiki/Md5sum for where to get an "md5sum" program from if you don't already have one)
That matches so we can continue.
Create a directory to work in - in my case I created "D:\doc\gps\romania" and copied romania-latest.osm.pbf to it.
From a command prompt (on Windows), change the current directory to be where the downloaded .pbf file is.
d: cd \doc\gps\romania
See what version of "java" you have installed. Type "java -version" at the command prompt to find out. In my case that says:
java version "1.8.0_91" Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
so Java is already installed. If you don't already have Java installed download and install from somewhere like: http://java.com/en/download/windows_xpi.jsp
Next, run splitter. The "-Xmx1200m" tells it how much memory to allocate; the "--max-nodes" how big to make each split piece. The examples for the parameters given here should "just work":
java -Xmx1200m -jar c:\utils\splitter-r437\splitter.jar romania-latest.osm.pbf --max-nodes=800000 --output=xml
That should create lots of ".osm.gz" files. Then run mkgmap. Here "--style" determines the style of map to create - "default" is actually a folder containing several files that make up that style. Unsurprisingly the style called "default" is the default mkgmap style - not particularly optimised for foot, bicycle or car use.
java -Xmx1200M -jar C:\Utils\mkgmap-r3676\mkgmap.jar --style-file=C:\Utils\mkgmap-r3676\examples\styles\default --route --gmapsupp *.osm.gz
When that finishes it should say something like "Total time taken: 621133ms" (i.e. about 10 minutes on a fairly slow PC for a country that is relatively small in OSM terms).
d:\doc\gps\romania: -rw-rw-rw- 1 A.Townsend None 123359232 05-14 13:31 gmapsupp.img -rw-rw-rw- 1 A.Townsend None 4270 05-14 13:31 osmmap.tdb -rw-rw-rw- 1 A.Townsend None 73728 05-14 13:31 osmmap.img
Create a subdirectory and move those files into there for safekeeping:
d:\doc\gps\romania\default: -rw-rw-rw- 1 A.Townsend None 123359232 05-14 13:31 gmapsupp.img -rw-rw-rw- 1 A.Townsend None 73728 05-14 13:31 osmmap.img -rw-rw-rw- 1 A.Townsend None 4270 05-14 13:31 osmmap.tdb
Copy those files to the GPS. The method to do this varies by device. I'd expect that people reading this will already do this with files they've downloaded from elsewhere; if not there are lots of examples on places such as http://help.openstreetmap.org.
When it's copied across, turn on the GPS and check that the new map is visible.
Next, how to change the style slightly?
In C:\Utils\mkgmap-r3676\examples\styles, take a copy of the "default" folder and call it something like "mystyle". We'll leave "default" as it was and make a simple change to "mystyle". As a test we're going to make tracks appear like unclassified roads.
Edit c:\Utils\mkgmap-r3676\examples\styles\mystyle\lines in a text editor. The "lines" file is the one that deals with linear items - things such as roads.
Here's the line in that file that deals with unclassified roads:
highway=unclassified [0x06 road_class=0 road_speed=3 resolution 21]
It's pretty straightforward what is happening - the OSM feature at the left ("highway=unclassified") is to be transformed into the Garmin feature at the right ("0x06").
Here's the line that deals with tracks:
highway=track [0x0a road_class=0 road_speed=1 resolution 22]
change the latter line to:
highway=track [0x06 road_class=0 road_speed=3 resolution 21]
(i.e. so that "highway=track" is treated as if it was "highway=unclassified")
Rerun mkgmap with the new style:
java -Xmx1200M -jar C:\Utils\mkgmap-r3676\mkgmap.jar --style-file=C:\Utils\mkgmap-r3676\examples\styles\mystyle --route --gmapsupp *.osm.gz
Create somewhere safe to store the new files, and put them there:
d:\doc\gps\romania\mystyle: -rw-rw-rw- 1 A.Townsend None 123338752 05-14 14:23 gmapsupp.img -rw-rw-rw- 1 A.Townsend None 73728 05-14 14:23 osmmap.img -rw-rw-rw- 1 A.Townsend None 4270 05-14 14:23 osmmap.tdb
Copy to the GPS again, and check that tracks do indeed appear as roads.
... and that's it.
Obviously there are many more changes that can be made. There's quite a lot of information on the OSM wiki at places such as http://wiki.openstreetmap.org/wiki/Mkgmap/help .
The resulting example "default" files can be found at this link:
The resulting example "mystyle" files can be found at this link:
The Hadjer Lamis area is very poor, and there is an unusual burden of disease and malnutrition amongst the population which contributes to high mortality in children under five years old. In order to better understand, assess, and respond to this, we need to know more about the population.
We are mapping villages and taking their names on the ground, but identifying all of the inhabited areas and counting the structures is much more efficient from aerial photos. Knowing where all of the villages are scattered through the savanna helps us to map them, and counting the buildings within each village gives us a quick and fairly accurate method to estimate population (important to understand the spread of disease and identify areas of highest need). Perhaps surprisingly, structure counts are often more accurate than asking how many people live in the villages.
The tasks at hand are:
Find all of the villages in the area, and draw an area around them, tagging each as Land use, Residential.
Map the structures in each village. This is done by tracing all buildings as polygons and tagging them as Building Features=Building (in iD editor) or building=yes (in JOSM) , or by simply counting them and adding a tag Structures with the appropriate number.
Settlements in this part of the world are generally organised into extended family compounds, each containing a number of small shelters and often a few crops. You can see the outlines of the compounds as dark lines dividing the village in to smaller sections. The villages are generally more or less circular.
This village (Absoufa) is quite typical of the villages in the area. Here's what a little part of it looks like on the ground.
Villages much like this are scattered all over the landscape in Hadjer Lamis. It is difficult to find them all on the ground, as even the local people do not necessarily know all of the locations of all of the villages! If they are already traced, it is much easier for our field teams to find the villages to tag them with the appropriate local names and other information.
Try to trace around all of the compounds, not just the houses; the compounds often extend well past the inner circle of houses but are definitely part of the village.
There are two basic types of houses here: round huts, often called Tukuls, and rectangular buildings. All are made of some combination of mud and straw (in some cases the straw has been pre-processed by donkeys or cows, making it stickier).
The tukuls are easy to spot from the aerial imagery. They are round, with a distinctive central point where the roof peaks. They look a little bit like, well, um, a rounded mound with a darker-coloured point at the top centre... you know, like a mushroom cap.
Please simply tag all structures as "building". There is a distinction between "house" and other types of building, but it's incredibly difficult to tell the difference from an aerial photo, so best to simply leave them as "building" for now.
The rectangular houses may have a thatched roof or a corrugated metal roof. A metal roof is usually quite obvious from the aerial photo as it reflects a lot of light. In general, the metal-roofed buildings are more likely to be important buildings such as a mosque, chief's home/office, or health clinic. If you see an obvious metal roof, you can add a tag for roof:material - metal.
Some of the buildings may be for storage of grains or animals (i.e. chicken coops). You can't tell from the aerial photo, so don't worry about it! Just trace all of the standing buildings you can see. The primary goal of the building tracing is to estimate population, and we can do this simply by multiplying the number of structures or roof area by a constant determined by surveys we do on the ground.
It is fairly common to see a faint ring or rectangle where a building once existed (mud and thatch homes do not last forever). It is best to count only currently standing structure, as this is most likely to correspond to the actual population. However, in the event that you can't tell whether the building is still there or not, please trace it! When in doubt, add the feature. It's easier to remove it than to find a missed one.
Unlike areas further south in Africa, where villages usually consist of a collection of homes without much internal separation, villages in Hadjer Lamis are basically organised by family compounds. Each compound is surrounded by a fence of mud, wood, or brush, and contains several buildings (sometimes including grain storage buildings as well as houses). Many compounds also include some cropland, in what is effectively a small farm or large garden.
These fences really define the internal structure of the villages.
Sometimes buildings are integrated into the compound wall; often you'll see that a rectangular building shares an outside wall with the compound perimeter.
There is no existing default tag for "compound" in OpenStreetMap. Therefore, when they are mapped as an Area and simply tagged "compound", they do not appear in the OSM map (though they are present in the data). For this reason, until we are able to resolve this, we are not tracing the compounds in the current task. However, please learn to recognise the compounds as they are part of the village; if you only trace around the houses the village perimeter will appear smaller than it actually is.
It is almost 8 weeks since I began mapping (21 March 2016). In my first Diary Mapping Thorneywood Mount I wrote:
One thing that keeps becoming clear is that I need Cards to id myself & give to others, plus, perhaps some literature to give to save myself 5 or 10 minutes explaining each time what the hell OSM is.
Thanks to a comment from LivingWithDragons I was able to get some funky little hand-outs from Andy Allan (thanks Andy), but still no personal ID. I was reminded of this following an unpleasant interaction with a little sh*t at Nightingale House, City Heights (the survey was made last Wednesday, 11 March 2016; I've consciously let 2 days pass to cool down, and to try to make sure that I didn't swear whilst I wrote this diary; as you can see, I haven't fully succeeded).
City Heights is built within the grounds of the old Nottingham Borough Lunatic Asylum (see the diary entry Nottingham Suburban Railway, Part 2 for more info) and has both new build houses in the lower parts and flats built within the original buildings erected for Nottingham's lunatics.
This is Nightingale House (built 1880, this is just the front; there is also a long, long section behind & then another wing beyond that):
I think that you may be able to guess that some men, if put in charge of such a building, may begin to over-estimate their own importance. Even so, the cardinal error was mine and – as I do not wish to have to repeat it – have decided to make a public spectacle of myself by revealing it to you.
I'm still not very certain of the best way to map these multiple-entrance buildings, and would welcome links to how to best do it (explicitly, how to map the flat numbers). I took account of the Wiki & ended up using “Man Made | Man Made | Entrance” with “addr:flats”, each number separated by commas. Normally, I do belt ‘n’ braces & add a relation with various “addr:*” added (postcode, etc.) but I still haven't discovered how to edit Relations, and therefore adding flat numbers to an associatedStreet relation was out of the question.
My current practice is to photo the entrance lodges to save having to write down each number. Here's an example:-
I took a photo on my mobile of two, and then made my mistake:- I rang the bell for the Admin Office as to ask if they had a list of all flat numbers (and hopefully also locations) to save time – it is an enormous estate (125 acres) (0.5 km²) with 6 individual buildings.
I wasn't thinking straight; on almost every occasion in which you ask an administrator for help they will say “No”. If you are especially unfortunate (as happened here), they will also attempt to deny you permission for other things (even if they do not have the authority to do so), which then puts you in a tricky position. Sigh.
The boss administrator turned out to be quite short, which on this occasion was a warning sign. His first statement was to tell me off for not wearing any ID on a lanyard (do I need to point out that he wasn't wearing any himself?), then said that I should have seen the signs that said that this was private property (he was lying at this point - there arn't any) and that therefore I'm not allowed to photograph the entrance lodges (also erroneous). I was on a hiding to nothing here, so made my excuses & left.
I've recorded & entered details for all flats in all buildings, and that is probably the best middle-finger that I could give.
I'm sorry Government of India, but it appears that OpenStreetMap will not adjust the borders just to please one country.
We've already got the tools that will help you create the map to your liking. You can find the instructions here.
The Geospatial Information Regulation Bill, 2016, basically bans all civilians from using or creating any kind of map.
The data team at Mapbox have built tools that allow detecting and cleaning these issues as they happen on the map.
Detecting issues with OSMlint
Each validator produces a GeoJSON output of the OpenStreetMap geometries that are invalid and needs review.
Fixing issues with To-fix
To-fix is a micro tasking tool that helps us create tasks of issues to fix on OpenStreetMap. The issues collected with OSMlint is loaded into to-fix, each of which is then reviewed by a member of the mapping community or the Mapbox data team. Over 500,000 issues have been fixed so far using to-fix.
To-fix also allows marking issues as false positives. These are later investigated and used to improve the validator algorithm in OSMlint.
The linting pipeline
We've implemented an architecture that automates the detection of issues using OSMlint and loading them into to-fix on a daily basis.
This helps keep the map free of basic data errors that would otherwise cause connectivity issues during routing and navigation. You can currently access the output of the following validators in to-fix for review:
- Crossing major highways
- Crossing minor highways
- Islands major highways
- Islands minor highways
- Kinks major highways
- Kinks minor highways
- Overlapping major highways
- Overlapping minor highways
- Unconnected major highways
- Unconnected minor highways
- Impossible one-ways major highways
- Impossible one-ways minor highways
- Highway intersects water - major
- Highway intersects water - minor
As always, we are constantly looking for issues in our systems and how it can better serve the needs of the OpenStreetMap community to create a truly open map of the highest quality. You can contribute by creating new validators, improving our current ones or just reviewing the issues on to-fix. Feel free to hit me on twitter or OpenStreetMap if you have ideas to share.
There has been a lot of chat on the HOT list about validation and a lot of that has focused around tools that new mappers use. It is obviously tiresome for validators to be squaring buildings endlessly or dealing with validation issues that would have been picked up by the original mapper at the touch of a button in JOSM, but that are invisible for those using iD.
In response, we have decided to start trying to work out better and quicker ways of moving people to JOSM at the London Missing Maps mapathons.
The first try at this happened in May. Using the eventbrite system to register participants to certain categories of ability has been our modus operandi for a while, so we simply upped the percentage of places available to those who wanted to learn JOSM and reduced the iD places. We also emailed ahead to our JOSM-experienced mappers to ask if they would mind making an effort to sit with, and support, the newbies.
Robert led an impressive, three hour walk through that mappers could dip in and out of and answered specific questions as he went.
From chatting to people there, not as much actual mapping was done. Despite asking all people to download and install the software before they came, there was still a fair amount of troubleshooting to do in this respect. We will see how the validating goes on the task we tackled. Feedback from validators looking at this task more than welcome!
I'd also love to hear from others organising mapathons where JOSM is the primary tool and from new mappers who have learnt to edit OSM with JOSM, or a combination of JOSM and iD. How did it work for you? What kind of training or resources worked best? What elements of the tool is it important to learn first? Any other comment?
Looking forward to taking this further...
Imagine you are driving down a highway with your car guidance giving you the instruction "In two miles, take exit number 164A and follow the signs towards Dearborn Street" using OpenStreetMap data. This will soon be possible with upcoming improvements to the OSRM guidance engine that will use
destination tags more intelligently from the map data.
Over the last week, the Mapbox data team reviewed freeways in 30 cities across the United States to find gaps, in exit number and destination information that could be improved using Mapillary images and official documents. We want to share our findings here and welcome your feedback on our approach, use of tools, and workflow.
We stuck to reviewing motorways and motorway junctions, as we did during mapping destination signs for 9 US States, and covered 328 motorways in total. As part of the review, we used Mapillary imagery, official Department of Transport data, Crosscountryroads, Aaroads, and Wikipedia for reference. For identification of exit numbers and destination tags, we relied on Checkautopista2 by k1wi.
Quickly find motorways with missing exit numbers and destination tags using Checkautopista2
Here is how we went about tagging:
ref=*representing exit numbers.
noref=yesfor junctions with no exit numbers.
destinationfor city, neighbourhood.
destination:reffor reference-number of next motorway.
destination:streetfor streets and road.
In total, we added 761 tags on the 328 motorways we reviewed. Here is a full breakdown of the number of destination related tags that were already present on OpenStreetMap before 1st April in 30 cities and what was added during this project:
Note There were already 2653 nodes with
exit_to tag in these cities, which is an older tagging scheme for destination mapping (read more). These features were not modified.
As we worked on this project, we got a ton of help from OpenStreetMap contributors who provided feedback, caught mistakes, and helped us improve our process. As remote mappers, its important our edits are validated by local mappers, feel free to review the data in any of the cities that were worked on and post your feedback on our project tracker. You can look forward to more updates to the map in these cities as we work towards a more navigation ready OpenStreetMap.
Data quality is as important a function for any contributor to know as is adding large quantities of detailed data to OpenStreetMap. If we were talking one or two individuals editing, quality assurance would be straightforward, but what about with 2.4+ million registered users adding 2,000,000+ nodes a day?
Luckily, the tireless vigilence of active contributors is a source of some consolation but nearly not sufficient by itself. The important question then becomes, how can we detect and remedy mistakes and acts of vandalism as and when they happen? In trying to find answers we've put together a "Validating OpenStreetMap Guide" which aims share our learnings to working with OpenStretMap. The guide covers the basics of handling data validation in OpenStreetMap by defining problematic data, tools for detection, methods for investigation, and avenues for resolution for individual and community mapping efforts.
History panel showing different details about users and versions
By sharing this we'd love to get your feedback on if we're on the right track, and suggestions on how this guide can be improved to help bring an unparalleled level of data completeness that OSM users worldwide can benefit from.
Still working on the analysis, arcgis doesn't work well with big datasets...
Results coming soon!
Finally, I got a chance to map Bacoor and Imus, with two trips where I am able to survey those areas, the first being from NAIA Terminal 1 towards home, and the second from home to SM City Bacoor via Molino, Mambog, and Palico, SM City Bacoor towards S&R Membership Shopping in Imus, and S&R Membership Shopping towards homme.
On the first trip, my survey found a split, new traffic lights, and pedestrian overpass on Aguinaldo Boulevard, new traffic lights and marked motorcycle lane (marked by a blue line on the right lane) on Molino Boulevard, and new traffic signs, installed by the Department of Public Works and Highways Cavite District Engineering Office, showing destinations of several intersections on Molino Road. I already mapped the new traffic lights and destinations shown on the newly installed signs, and yet to map the split, traffic lights, and traffic lights on Aguinaldo Boulevard and the motorcycle lane on Molino Boulevard.
On the second trip, my survey found several missing business, missing business names, location of barangay boundaries on the Mambog area, a traffic light on the intersection on Aguinaldo Highway and Tirona Highway, an Alfamart branch near SM City Bacoor, a no left turn sign on the Aguinaldo Exit of SM City Bacoor,, a missing church along the Aguinaldo Highway, a welcome marker on the Bacoor-Imus boundary, several missing kilometer stones, a split and new traffic lights on Aguinaldo Highway near Tanzang Luma, missing barangays of Imus (like Bayang Luma, Patindig Araw, Anabu I-B, Anabu I-D, Anabu II-B, and Pasong Buaya 1), a missing name of an industrial area in Anabu, Imus, a S&R New York Style Pizza store inside S&R Membership Shopping in Imus, a missing Kawasaki motorcycle store, and a lot more. Most changes have been done,after that trip but a few is to be done soon.
I preferably enjoy mapping areas while on the go, like on trips, where I sometimes bring a notebook, a pencil, and a sharpener (in case the pencil tip goes dull) to create a survey map that I use to collect information before adding them to OpenStreetMap. But, when I don't have a notebook and pencil, I memorize the names and locations of places I found on the survey before adding them to OSM, yet, a few fades away from my memory and a second survey is needed to retrieve them back. Now, with a new tablet, I am able to take pictures with lat-long information to determine the location of a certain place.
My mapping activity on Bacoor and Imus is still few, but with my personal knowledge of those areas, because I have relatives living in Imus, I am able to add a few places that I know of without needing a survey.