Recent diary entries
Why am I against wholesale import of administrative boundaries from any 3rd party source for the PhilippinesPosted by maning on 11 August 2015 in English (English)
Disclaimer: This is specific to the Philippines, not a general OSM issue.
One of the most difficult data to collect in OSM are administrative boundaries (
admin_level=*). It defies the on-the-ground rule. One cannot just go out and start surveying admin boundaries with a GPS. On the other hand, we see the importance of having admin boundaries in our database. We can define town/city limits. It improves geocoding. The maps looks nice. Humanitarians need them because they can plan and allocate resources according to administrative jurisdictions during a crisis. The only logical way to have this in OSM is to get them from various sources and do an import.
The most comprehensive source we found for the Philippines is from the freely available GADM. This website has a comprehensive collection administrative boundary data for free down to the smallest administrative units for many countries including the Philippines. Over the years, I tried to track down the provenance of GADM’s PH data. My geo-forensic skills lead me to people saying that the PH dataset originated from our national mapping agency. So, it seems very authoritative, why don’t we just import them? I say NO and here’s why (again, I am pertaining here to the Ph situation, GADM data maybe good in other countries and I have the utmost respect to the maintainers of the site sharing this to the public).
Even if the license is compatible, the data quality is REALLY bad. Again, from Eugene’s mail to the list, here’s screenshot comparing OSM and GADM boundaries in Quezon City.
If the above arguments does not dissuade you, consider this story.
Administrative boundaries imply a level of authoritativeness in the data. Ordinary users of the data may take it as exact and definitive. For example, during the immediate relief operations after Typhoon Haiyan in the Philippines, OSM volunteers across the world participated in the remote mapping response. We generated a tremendous amount of building, roads, and landuse data which were used by many international response agencies. Since we lack and can’t remotely map admin boundaries in the affected areas, response agencies resorted to using 3rd party admin boundaries (GADM is one of them) as an overlay to OSM derived basemaps. These maps were used by humanitarian agencies to organize relief operations. When I went to the affected areas months after the Typhoon, local authorities are complaining that in some instances, international agencies are insisting in their own maps particularly the village boundaries as a basis of relief supplies allocation. Local authorities would insist that these boundaries are wrong and does not exist in reality. Of course, response and relief planning should be dependent on other factors and not just based on a map, but, a wrong map exacerbates an already confusing situation.
So am I saying that we shouldn’t be adding admin boundaries in the Philippines? No, what I’m saying is that any 3rd party admin boundaries that covers the whole country you stumble upon either from the internet or directly from various agencies are wrong. No comprehensive data exist in the Ph.
What can we do now?
Talk to your local government authorities. You may find good data from specific local government units, if ever you get a hold of such data, let us know and if it is good enough, we can start the process of adding them in OSM.
Go out and survey! Map places as nodes. Oftentimes, a place node (
place=town,city,village) is enough. For finding places in OSM, would you want a geocoder to give you a wrong boundary/polygon or a node/point that tells you exactly that this place exist in reality? In many cases, this is what I’m trying to do. Here’s a screenshot of village nodes we mapped in a remote town in Leyte. Compare that to the inaccurate (by 3 kms south) imported boundary of the town. Unless we get a better source to replace this boundary, I would rely more on the place node we surveyed with the local community.
I know, sometimes, it is very frustrating, OSM is heralded as one of the best freely available geographic data, yet, we lack the most basic admin boundaries over many areas. Imports might be the immediate solution, but, I ask PH mappers no to do this. I’m fine with missing boundary data for now, in time we can improve this the OSM-community-way, by importing, we maybe propagating more errors rather than helping improve our map.
Last July 24-25, we had the third leg of the series of crowdmapping events co-organized by The Asia Foundation (TAF) and its local partners in Butuan City. The first was in lloilo and the second in Tagbilaran. Similar to the previous events, the objective of this mapping party is to increase awareness of the potential of using OpenStreetMap to complement the various mapping initiatives of the TAF’s local partners in the region by inviting mapper volunteers to participate.
Butuan is a special place for me. I have never been to this part of Mindanao, but, this is where the journey of my relatives may have started when my grandfather and his family migrated from Luzon and started a new life to what was then called the “new frontier” of the Philippines 4 decades ago.
Butuan is also a significant trading port during the pre-colonial times. Within the alluvial plains of Agusan river, archeologists discovered remains of pre-colonial settlements serving as a major trading and inter-cultural exchange between the upland tribes of Mindanao to Indonesia and mainland Asia.
In the late 1970s, remains of balangay boats within a dried-up river channel near the city centre were discovered.
“The balangay was the first wooden watercraft excavated in Southeast Asia and is evidence of early Filipino craftsmanship and their seamanship skills during pre-colonial times.” Wikipedia.
A proof that early Filipinos are excellent sea navigators and have sustained trading relations with the rest of Asia even during pre-colonial period. The word balangay is where the word
barangay originated. It is the smallest administrative division of the Philippines (
Moving on with the mapping, I arrive Butuan early morning of Friday. The orientation to participants will start later in the afternoon. With nothing else to do, I explored the city on foot with my GPS and camera taking several geotagged photos for my own mapping including a mapillary sequence of the mighty Agusan River.
Later in the afternoon, we started the orientation to the local cycling group and instructed them to download OsmAnd which they will use for the field mapping exercise the next day.
We started early morning of Saturday to the usual meeting place for bikers, after a few minutes of basic OsmAnd use, the teams proceeded to their assigned patch to map POIs focusing mainly on 24/7 and cycling related facilities.
We met again for lunch and uploaded all our collected data. We mapped around pois: 435, lines: 40, polygons: 27 during the day and I’ve seen a few mappers continue adding more POIs a week after the event.
The cyclists really love OsmAnd because it is not only a data collecting app but more importantly, a complete offline navigation app they can use for their regular rides. One thing I noticed is that with the ever increasing size of data download for the PH maps in OsmAnd, it is starting to be very difficult to download for areas with very slow connection especially in remote areas and even in major cities in Mindanao and Visayas. Expensive data plans were maxed out just to download the worldwide overview map. I’ve already posted an GH issue hoping the OsmAnd devs would consider splitting the PH map into subregions just like other countries.
As usual, notes, photos and map updates in this page: http://maning.github.io/taf_crowdmapping/butuan.html
Once again, we had another mapping event. This time, in Tagbilaran City, Bohol. This is part of the series of crowdmapping events co-organized by The Asia Foundation and its local partners. The first was in Iloilo City last May 2015.
During the event, Bohol Governor Edgardo Chatto welcomed all the mappers and expressed his support to implement this initiative for the whole province of Bohol.
Bohol's major income is tourism. However, when the 7.2 earthquake hit the island in 2013, the tourism industry was heavily affected. On the other hand, judging by the number of tourists (local and foreign) who was with me during the flight, I think tourism is now recovering. What a better way to help by providing a good map not only for the tourists but also for the local community!
The local partner of The Asia Foundation is the local chamber of commerce. Naturally, we focused on collecting business and tourism related map data during the field exercise.
Using phones powered by OSMAnd, we collected and updated pois: 451, lines: 94, polygons: 168.
I joined one of the field teams and also took the opportunity to kickstart crowdsourcing streetviews using Mapillary.
Other than several hiccups with the internet, the organizers did a very good job in managing the event. I hope the map data we collected will help in expanding the mapping community and help the province in revitalizing its tourism industry.
More notes/photos and map updates in this page: http://maning.github.io/taf_crowdmapping/tagbilaran.html
Last May 2015, I joined a mapping party in Iloilo co-organized by The Asia Foundation and several bike groups (IPAD Xtr, ICYC, iFOLD, Augustinian Cyclists) in the city. Originally, we designed to have an editing session using iD. But since most of them did not bring laptops, (they went to the meetup venue mostly on bikes of course!), we focused mapping using SmartPhones installed with OSMAnd.
Thanks to Erwin for updating the adding a new section on OSMAnd2 in learnOSM which I had to review early morning before the event. ;) The good thing with OSMAnd is that you have a lot of options for mapping, the bad thing with OSmAnd is that you have a lot of options for mapping. ;).
Anyway, we focused on using OSM plugin to map bike related POIs. There were some crashes on one device and we were not successful getting the plugin to work on iPhones/iOS. All in all, I think OSMAnd is perfect for groups like this. Even if they don't use any of the editor, they can continue contributing by just uploading POIs. There were some issues along the way such as some POIs in the middle of the sea and other with only name tags. We are fixing it now.
Here's my presentation notes/videos, mapper contributions and photos.
This post was originally shared in the OSM-PH mailinglist.
One of the on-going initiatives by the local OSM Philippine volunteers is to go to local communities to assist in training the local population to update and use OSM for disaster risk reduction.
One partnership we are nurturing with is the DRR mapping work by the Philippine and Swiss Red Cross in small island communities in Busuanga, Palawan. Last April 2015, mappers GOwin, feyeandal and dichapabe, went to Busuanga to start the mapping community with the local government and Red Cross volunteers. After the training the online mapping community lead by GOwin continues to assist the Busuanga mappers in updating the maps.
This work was featured in an article in Channel News Asia.
By combining local knowledge and OSM tools, we hope to continue building the local mapping capacity and data that will empower communities to respond to any crisis.
In reply to RichardF's call for stories, I shared a few of my mapping stories. Marking the link here, so I get back to it later: http://www.openstreetmap.org/user/Richard/diary/35107#comment30756
The first time I saw this in the map, I've always thought it was an editing mistake. This is Catarman Airport in Northern Samar, Philippines. What looks to me as a bug is the secondary road intersecting with the airport's runway! Surely an editing mistake. But since I haven't been there and the satellite image is too coarse, I cannot verify if this is indeed the case.
Yesterday, I had a chance to talk to locals familiar in the area and they indeed verified that this is correct. The airport services one or two flights a day. Vehicles are allowed to cross the runway in between flights similar to a railway level crossing. Aerial shots from wikipedia and from another website confirms this as well.
My question is, how do I tag this? Surely, there are similar situations in other areas.
(I posted this in the HOT list. Board election is upcoming and voting members were asked to communicate what HOT's board should focus on in the future. Here's my appeal)
Let me kickstart this. First, no, I’m not running for the board, but here’s what I want HOT to aspire for in the future.
As many have said, we’ve come a long way since we started with the Haiti earthquake response. We have better tools, more capable people, better systems/organization and worldwide recognition. For many areas in the developing world, we are the default map. This is especially true for us in the Philippines where more and more users are utilizing our map across a diverse type of organization (international, national agencies, local government). Having said that, I also see a lot of improvements we can look into within HOT and the larger OSM community. Below are “wishlist” for the HOT community to consider. Note that this is my own perspective having been involved in several humanitarian mapping work(both as a remote mapper and deployments on the ground) in my own country. This is not in the order of priority.
From data consumers to data contributors. Many international organizations are using our data, but, I often wonder, do they contribute back? I know a couple of organizations are doing this (IFRC/ICRC/ARC/BRC, MSF, MapAction to name a few). I think we should consciously encourage these consumers to give back. We are not just a source of free geospatial data, we are a community and they are part of it.
Building local community capacity. We are very good at responding to crisis. For a very short period, we can provide highly accurate data (street and building level detail) to any area in the world, but at the end of every response, have we considered how will the local community (if there is one) curate and continue maintaining the data? I think for every activation we respond to, we should always consider building local capacity. Some countries might not have any local community, but in areas where there is, we should strive to engage with them no matter how small this community maybe. Because ultimately, it will be local community who will maintain what we kickstarted.
Focus more on preparedness over response. MissingMaps, MapLesotho, HOT-Id (and other HOT technical assistance) are doing this already. And I think this is what we should be do more. For areas in the Philippines where we integrate participatory/community-driven mapping for disaster risk reduction, the simple exercise of mapping is a powerful tool to increase awareness on the local hazards and to engage local stakeholders (affected communities, DRR managers, local governments) in a discussion for better preparedness and response. We’ve witnessed instances where pre-mapping as part of the DRR activities allowed better response during a typhoon last year.
Better tools under challenging environment. As I said above, we have better tools now. But offline/very limited connectivity remains a big concern for most of the areas we are responding to. Better and simple tools under this condition should be what we should aim for.
Regional exchange and “mentoring”. I’ve learned a lot when collaborating with other mapping groups under a similar context. Fo example, we had several exchanges with OSM-Indonesia (through HOT’s project), being in a similar hazard context and, in a way, a closely-related culture, I find it that many of the techniques they have developed are applicable to our own condition. Would it be possible for HOT to facilitate this? For example, can MapKibera lead regional mentoring in Africa? Or OSM-Haiti within its region? Oftentimes, mentoring is from the “North” to the “South” or from “West” to “East”. This is also very valuable but lets also consider that there maybe local community experiences which can be shared within the region having the same socio-cultural context that can be more effective and adapted to the local condition.
Hoping the Board and the HOT community can work on some these wishes.
Short post. Testing mapillary around my area.
Crowdsourcing geotagged photos is not a novel idea, but, one thing that attracted me to mapillary is the friendliness to OSM.
Maybe a longer post on my rig and experience later.
(Re-posted from the talk-ph list.)
I discovered OSM around Jan 2006 (user# 1417) while trying to look for PH vector data I need for a research. Free geographic data in the PH back then is very limited. The idea of building it from scratch got me interested. However, I wasn't able to edit right away because I can't get the then java-based web editor to run. ~10 months later I stumbled upon JOSM in the wiki and created my first node.
The oldest rendering I was able to save was this. As far as remember, it was Mike Collinson who made those edits. After several borrowed, hacked, broken, lost GPS since, Marikina is still not complete. So, I'm still here mapping.
Sinusubukan ko ngayong isalin sa wikang Filipino/Tagalog ang iD. Bagamat ito ang wikang aking nakagisnan, nakakahiyang sabihin na medyo nahirapan ako sa pagsasalin. Sanay kasi ako sa paggamit ng Ingles sa mga technical na bagay gaya ng computer.
Halos lahat ng nasa "core" ng iD ay naisalin na (maliban sa "Walkthrough").
Unang pasada pa lang ito. Susubukan ko pang ayusin ang ilang mga
Sa mga ibig tumulong, madali lang naman magsalin gamit ang transifex. Puntahan mo lang yung section sa Filipino.
Para sa pauna kong tangka, hindi ko sinunod ang tuwiran o literal na pagsasalin. Kung tutuusin, parang "taglish" yung ginawa ko. Sa aking karanasan kasi, mas mahirap unawain ang tuwirang salin gaya ng pagsasalin sa tagalog ng OpenStreetMap website:
require_cookies: cookies_needed: Tila mayroon kang hindi pinagaganang mga otap - mangyaring paganahin ang mga otap sa loob ng pantingin-tingin mo bago magpatuloy.
Sa aking palagay, may mga ilang technical na salitang mas mainam gamitin sa wikang Ingles kaysa isalin ng tuwiran.
Kung tutuusin karamihan sa mga Pinay/Pinoy na contributor sa OSM ay sanay na sa paggamit ng Ingles. At gaya ko, may mga pagkakataong mas komportable ang paggamit ng Ingles lalo na sa usapin sa computer.
May bahid ng pagiging makabayan ang dahilan. ;) Bagama't pwede namang gamitin ang wikang Ingles, pinagmamalaki ko ang pagiging Filipino kaakibat nito ang aking wikang kinagisnan (mother tongue). Isa pa, hindi natin alam, baka may gustong gumamit ng wikang Filipino/Tagalog habang nag-eedit sa OSM gamit ang iD, mainam nang mayroon silang mapagpipilian.
For the past several months, OSM Philippine data is getting noticed by a good number of people and organizations. This is in part because of our crisis mapping in response to Typhoon Yolanda/Haiyan but, I also think the data is now in a usable state in many areas thanks to the dedication of local mappers.
As I look around the map, there is obviously a lot of inconsistency in data coverage. Some are mapped up to the individual buildings while others just a
place=town node and nothing else.
Two things I consider to be a factor of this situation:
- Most mapping are based on satellite imagery. In the case of my country, while imagery is available over large areas, many areas still don't have hi-res imagery. This makes mapping "stop" at the edges of the imagery. The image below shows roads digitized during Yolanda mapping (map).
- Most local mappers are too focused on improving their own patch. This isn't really bad since you should really map areas you are familiar with but, this leaves behind adjacent areas looking bare and the coverage uneven. Even I am a victim of this micro detail mapping. Just look at Marikina (my mapping patch) on the left and Cainta on the right (map).
I still think we should map in detail our own local area, but we should also keep in mind to look beyond our patch to produce a good level of consistency and evenness of coverage.
Eugene outlined this level of detail concept a few years back. In the document, he proposed 7 LODs based on the following parameters:
- Scale / zoom
- Spatial accuracy
- Completion unit
Re-reading this document, I feel that the current Philippine data isn't even on the first level. So this year, I plan to incrementally work on reaching LOD 1 (while continuously improving my own patch of course!) focusing on the following:
- Check whether all cities and municipalities have at least a
place=town or citynode located within the town/city center;
- Check whether all national roads are mapped and connected to the cities and municipalities (in OSM tagging, the
highway=motorway, trunk, primaryroads);
- Move previously imported
place=islandnodes to each way/polygon (working on this now);
- Map major rivers (
waterway=river) and other large water bodies;
- Fix relations of existing provincial boundaries.
Definitely a big task, but we can always work together.
Recently, I've been trying to move tags on island nodes to its way (natural=coastline) following the best practice of One feature, one OSM element. The original place=island nodes were from an GNS import. Most of these islands now have a digitized coastline so it makes sense to add the place=island and all its tags to the ways.
To do this, I ran an overpass query to get all place=island nodes within a given boundingbox. Code is below:
<osm-script output="xml" timeout="25"><!-- fixed by auto repair --> <union> <!-- query part for: “place=island” --> <query type="node"> <has-kv k="place" v="island"/> <bbox-query s="4.061535597066106" w="111.57714843749999" n="21.166483858206583" e="127.11181640625"/> </query> </union> <!-- print results --> <print mode="meta"/><!-- fixed by auto repair --> <recurse type="down"/> <print mode="meta" order="quadtile"/><!-- fixed by auto repair --> </osm-script>
The result will show on the map like this:
Then, I clicked the "Export" link to download the data into JOSM.
Slowly, working my way on each island node, I transferred the tags to the coastlines. If there is no natural=coastline, I trace them using Bing's high-res imagery and then add the tags in the island nodes.
Right now, I'm focusing my efforts on smaller islands where the natural=coastline is on a closed way. I will start working on the islands relations once I finish the closed ways. Might take a while though (current count is ~2,500 nodes).
My wishlist for the changeset/history view, ala GitHub map diffs.
(images from GitHub)
Catching up on the slides and tweets of the successful SOTM 2013 in Birmingham, I found this very interesting talk by Alyssa Wright (@alyssapwright). The slides discussed the general trend of a male-gender-biased-tagging of features in OSM (see slides #72 to #79).
This reminded me of a resource mapping and assessment we did for an Indigenous Peoples in the Philippines. The research covers a protected area where several IPs communities (Batak and Tagbanua) are living. Part of the research is to conduct participatory mapping workshops with several IPs villages. We used a physical 3D model (very similar to this approach) to allow community members to identify key resources and other geographic features.
During the series of mapping workshops, I insisted that as an initial mapping activity, we divide the group into men and women. Both groups will have its own 3D model and they were instructed to identify important geographic features within their community. The final map will be an integration of both workshop output.
The map showed very interesting results.
The men group covered a larger extent of the area, common features they identified are:
- names of all major rivers and streams;
- location of hunting grounds including accurate position of where they hunted the largest wild pig, snake, or eel;
- important trees for gathering resins and wild honey;
- approximate boundary of forest cover types.
The women group on the other hand covered a smaller area mostly within the established settlements of the tribe, common features they identified were:
- location of community structures such as schools, place of worship, community halls for gatherings;
- sources of clean water (wells and springs);
- a stream that regularly overflows limiting access to children going to school;
- patches in the forest to gather medicinal plants and other wild vegetables;
- patches of swidden farmlots.
The community mapping exercise provided a rich source of information for the resource mapping and assessment. Moreover, conducting a separate mapping workshop to each gender group in the community encouraged greater participation of women.
Both maps shows very different priorities and perspectives but not one more important than the other.
OSM-Philippines have a new logo. Eugene (seav) designed this logo using the original OSM logo as its inspiration. He added the basic outline of the country's archipelago and the three stars and a sun elements of the Philippine flag.
The community love it! Thanks to Eugene for designing our new logo!
Available here as svg and png: http://wiki.openstreetmap.org/wiki/File:OSMPH_Logo.svg
SteveC recently started a Kickstarter campaign for a GPS Art Poster. Since my country is not included, I decided to check the tracks around my OSM patch.
The image below shows all the publicly available traces in OSM (not all of them are mine but most are) and also some I didn't uploaded over Marikina City. I probably have more lying around SDcards and in my GPS internal memory.
The image does gives you a personal story of your mapping expeditions.
The "low-res" tracks along the circular roads (northeast) were one of my earliest mapping for OSM. Back then, I was using an old eTrex without an SDcard slot. It would take around 2 hours of walking to fill the internal memory. To download the data, I hacked an old serial cable mouse and a plastic card (very similar to this rig) because buying the Garmin serial cable is almost half the price of the device.
The tracks made by my students (6 years ago) for an end of term mapping project is still very visible and were never replaced by a much better trace.
Someone (not me) must have forgotten to turn his/her GPS off and then uploaded the point cloud without any post clean-ups.
The lonely trace in Provident Village was when I visited a fellow mapper and delivered one of our GPStogo unit he is planning to use for a mapping expedition.
The large blank patches (center west) are private subdivisions where you are not allowed to enter unless you show an ID or you have an official car sticker. I never bothered cycling inside those posh subdivisions.
What's your GPS track story?
Testing the rendering of landuse and natural multipolygons using JOSM's Mapnik Style. Of course, it's not in the main OSM db!. :-)
Will @MapBox allow imagery tracing for OSM with their new satellite tiles? http://mapbox.com/blog/digital-globe-partnership/
I've been passing along doing housenumbers for years, partly because I find it too micro and partly because I am scared getting addicted :-).
I tried it before using my usual workflow, bike and a GPSr. It usually takes half an hour to collect some numbers, another hour to download the trace, edit in JOSM and, finally upload. Months of work got me this result.
After that month, my drive to do it died.
Recently, I stumbled upon KeypadMapper. The wiki promised a "rapid data collection" approach to address mapping. So, I tried it, and it delivered its promise!
Again, with a bike and this time a 'droid phone, I was able to collect housenumbers in a fraction of time than with my old workflow. I like the simple interface (you can collect a housenumber in less than 3 phone taps). It creates a .osm file and with just a few adjustments, you can upload them via JOSM. The map below took me less than a week to finish.
There are a few tips I find useful using this app.
- You need to be on a consistent speed and direction. The app considers your movement direction to approximate which side of road (left or right) the housenumber is located.
- An accuracy of <6 meters is probably the best.
- Slow walking will produce low quality GPS trace with jagged lines which influences which side of road the housenumber will be located.
- It is better to focus on one side of the road for one trip and then the other side on another trip. Editing will be easier since you are sure all your housenumbers are on one side and you can adjust them accordingly in JOSM.
Any other advice for doing address mapping? Please share them below.