Recent diary entries
*Edit: big thanks to Charlotte Wolter for having kindly corrected and improved the English!
The use of OSM in humanitarian and development projects is growing, and more and more stakeholders are interested in this approach, impressed by the results it has shown since the earthquake in Haiti four years ago. OSM has, remotely, grown a significant community of volunteers who are mapping affected areas, especially with the Tasking Manager tool and with field projects led by the Humanitarian OpenStreetMap Team and several partners, stakeholders and individuals. The past criticism by some stakeholders (against openness or about potential quality issues) has toned down, and OpenStreetMap has become a key element in humanitarian relief and Disaster Risk Reduction Preparedness (as shown in the Open Data for Resilience Initiative: Field Guide). HOT has played a leading role in making this happen, creating the link between the OSM community, and humanitarian and development stakeholders. HOT has become a kind of OSM chapter for these efforts and a structure able to run successful field projects, sometimes in tough conditions. These projects have demonstrated the capabilities of OSM in these contexts.
What should be the role and strategy of HOT regarding this (exciting) situation? Should it be an internal growth? In 2013, HOT had 6 medium or big projects. So it could expect to double this in the coming years and have a few full-time staff and make HOT a small/medium size NGO. I think it is definitely important for HOT to run projects, in order to grow capacities. These include internal capacities, such as expertise and skills within the deployed OSM specialists, as well as communautary tools, like in the past for the Tasking Manager or the HOT Exports. Also important are external capacities, such as creating or supporting local communities. Also each new project enhances knowledge as lessons are learned.
Nonetheless, there are limits, if this is the primary strategy. First, any individual close to HOT activities can thus be seen as having a conflict of interest as soon as she or he engages in a paid activity where OSM is involved (training session in an event where all trainers are paid, field support to a local project, etc.) or is involved in initiatives with external organizations. “Conflict of interest” sounds like being guilty of something that should be avoided, Consequently initiatives may tend to be self-restricted by these individuals. And will this allow HOT to be the only organization using OSM on the humanitarian and development fields? No, and it is actually not even the case. There are already many organizations in this field that have run or are running their own mapping projects based on OSM, such as World Bank GFDRR, American Red Cross, MapBox, Architecture for Humanity, etc. And others will very likely do the same in the near future. OSM will be used more and more as a platform for mapping projects or as a component in projects that are not only mapping based.
Should these initiatives from individuals and external organizations be considered as endangering or competing with HOT? I don’t think so. First, I think there are donors who target those who have initiated a new methodology and/or technology. So HOT will be identified as such an initiator and will continue to make the difference for such funders. I also think that, these initiatives represent great opportunities to expand what should be the main aim of HOT, like any other community-based OSM organization: increasing the use of OSM and enriching its data.
What should then be HOT’s primary strategy? IMHO, it is to foster, advise and support any project that wants to use OSM in humanitarian or development contexts, as long as it is respectful of OSM and HOT ethics that are agreed within the whole community of OSM contributors. (Do we have any ethics that everyone agrees on?)
This calls for the definition of a HOT Project with a HOT Charter and HOT Commons that any individual or organization could concur with and even officially join and/or fund.
The HOT Charter would contain good and fair practices that should be embedded in any OSM project. Building local capacities should be one of the major concepts. * when the time allows, train every mapper in the whole work flow, not just field surveyors or editors, which is very common in classic field-collection projects * train the future trainers to make possible the rise of a local community * ensure the conditions (space, equipment, Internet) that will allow this community to continue the mapping are initiated during the project
The HOT Commons would provide: * a set of tools to create, access and analyze OSM data easily, according to the needs and ideas expressed by those using them, i.e. international and national organizations or NGOs, local communities, local authorities, etc. This exists, but could be improved * a set of technical guides, describing, not only how to create OSM data or products based on it (what LearnOSM covers), but also work flows and methodologies to train new mappers and organize data collection, from the lessons learned from the remote activations and the field projects * neutral support for external projects following the terms of the HOT Charter, such as projects started by individuals or local OSM communities that request it, in order to create local capacities, including in Crisis Preparedness and Response
This approach would be a virtuous circle: * the HOT Charter would ensure the mapping projects based on OSM led by other organizations would follow the fair practices identified by the community. It also would ensure that those organizations would not use OSM as a platform to raise money without building any local capacity. If they do not endorse the charter at least it will be clear for everybody that they decided not to follow fair practices. Of course, the compliance with the fair practices by those who endorse the HOT Charter would be verified by a neutral committee from the community * the HOT Commons could get financial support from donors interested in supporting one or all of its aims and would strengthened the creation of tools and guides for all and the support to local OSM projects
I am interested in any comment on these topics!
I have recently been honored to be elected within the HOT Board, but I had not presented yet my contributions to HOT and OSM during the past year, despite having been deeply involved in HOT and OSM activities all along 2013. My mistake, due to the fact I intensively contributed to a debate of ideas after and during the election, therefore did not find the time to write this feedback. Here it is. Rather than organizing it thematically, I chose to do it chronologically, because I think it reflects better how things slot together.
2013 was actually the second year I dedicated my whole professional time and large part of my free time to HOT and OSM, mixing outreach, training, crisis response and project management. Sometimes exhausting or stressing, but definitely thrilling.
In January I was continuing the management of the EUROSHA Project that had started late September, 2012 with the training and the deployment of the volunteers and a HOT field support of three weeks in each of the four countries (Kenya, Central African Republic, Burundi and Chad). It was then the project mid-term and a necessary review and discussion within the EUROSHA consortium of 9 NGOs for the second phase. Supporting the volunteers teams (whose blog posts can be found there) through emails and Skype calls was also one of my duties, as well as preparing the second field support missions, as planned in the project frame. In the same time, I was discussing with the HIU, as the area around Molo in Kenya, where the volunteers were hosted, was not covered by high resolution Bing imagery. An official request for a delivery through their “Imagery to the Crowd” program has been done and the imagery kindly delivered a bit after. Another EUROSHA related activity was starting a HOT Monitoring in Central African Republic on January 5, as the country had been invaded by the Seleka Rebellion in late December, what made necessary the evacuation of the EUROSHA volunteers deployed there. WIth them, we also set an agreement with UNICEF for an import in OSM of their database about health facilities, schools and drinkable water points over half the country. Otherwise, I also had a global Skype chat with all the French Speaking OCHA GIS/Information Management Officers, that was organized by Andrej Verity, in order to make them know more about OSM and how to use it in the field. In February, after a blog post about EUROSHA in Kenya (see here), back to the field, starting with Burundi for three weeks of field support for the EUROSHA volunteers (their facebook here), for outreach, internal and external training on OSM and QGIS, as related in this other blog post. A camp mapping in partnership with UNHCR in Gasorwe finally occured just after my leaving (see a video). Once again, I had great times with the volunteers, but also the Geography professors from the University of Bujumbura. Late March, the team from Kenya (their Facebook here left the country before the Presidential election and joined Bujumbura with Stéphane, their HOT field support, and we all had interesting meetings and joint work. Early March I directly went to N’Djamena in Chad to support the EUROSHA volunteers during their last weeks of deployment. Outreach and training again (here are some pictures and a video), as well as the results conference at the National Library during the Open-source Softwares Days. Back to Burundi for a couple of days, then back to Europe.
After a few days break, I went to the EU Joint Research Center in Ispra, Italy to talk about OSM and HOT, and other interesting topics, as related in this blog post before going back to home. In April, I have started leading a HOT Activation for Central African Republic (see blog post here as Bangui, the capital city, had just been taken over by the Rebellion. The aims are mapping the most affected cities, getting a consolidated road network and documenting the UNICEF data to be imported, in order to get an authorization from the OSM import list. I hoped the settlements layer from OCHA COD (Common Operational Datasets) could be also imported, but unfortunately, the license status of this data remaining uncertain, so this could not be done. Late April, I went to FOSS4G Buenos Aires followed by SOTM-Ar. I made a presentation of HOT (my first one in Spanish, or rather portuñol) then participated to an OSM workshop/training during the SOTM day. Then I prepared my deployment in Northern Eastern Haiti to join CAP103, the OTI funded project in partnership with Limonade University that had already started for one month and a half, involving a lot of HOT members and contributors (Nicolas, Brian, Jaakko, Pierre, Fred, Yohan and Will), 14 experimented Haitian OSM trainers and 60 new local mappers. These beneficiaries have been be trained to OSM techniques and organized in 6 effective teams to map the area between Cap-Haitien and Fort-Liberté, and along the project, have been supported to create their own local organization with the help of 2 Community Mobilizers (Delphine and Emilie). The project also has been the opportunity to set a thinner HDM preset and a Humanitarian OSM rendering. During two months, my duties encompassed organizing the mapping planning and review, meeting local authorities and involving them in the mapping (especially for the helmet names and remote places), daily reporting, as well as training the young mappers to QGIS (in my average Creole). Was quite exhausting, but really great time with everybody. Once the project was done, I stayed voluntary for extra training and support, a visit to COSMHA-STM (the OSM Organization in Southern Gonaives created last year after the HOT Project in Saint-Marc with OTI) and a few meetings in Port-au-Prince, where, as ususl, I set up home in Haiti Comunitere, a HOT partner for years in the country and a great place to be.
In July, before visiting my family in France, I taught OSM at Federal University in Salvador da Bahia, Brazil before going to Geneva for ECOSOC where I held a HOT booth. Was a good opportunity to meet people from other Virtual Technical Communities that are also part of the Digital Humanitarian Network like SBTF, Translators Without Borders, MapAction or Humanity Road. A few weeks later, I joined SBTF in order to find ways and opportunities for SBTF and HOT to work jointly during Activations. I spent the rest of July and August in France; the HOT project in Mongolia was supposed to start during the summer, but has been postponed. Therefore, apart final reporting on the EUROSHA project, I had Summer holidays and on my free time, I continued working on the UNICEF data to be imported in OSM. Once clear with the Import talk list, I modified the raw data according to OSM tags and Ben Abelshausen designed specific Tasking Manager jobs cutting the dataset with the extent of each tile. I wrote a detailed workflow and started the import. In the same time, along with a few mappers, we improved a lot the road network in the country and had exchanges with MSF Spain deployed in Kabo that mapped the town and its outskirts on OSM. In the meantime, I also applied for a voluntary UNSPIDER Technical Advisory Mission in Malawi in mid-October and I have ben selected. Still in August, large floods affected the Niles in Sudan, especially around Khartoum, and a HOT Activation has been launched (see its Wikipage with interesting exchanges with UNOSAT, HIU, and local relief volunteers. One challenge was to define AOIs (Areas of Interest) and we used this uMap to compile various information (special thanks to Brendan!). The Response of the OSM community has been great and imagery from Charter Activation has been released and made accessible for OSM mapping. Thanks Guilhem, the raw imagery has been georeferenced and hosted and I used the great offset_db JOSM plugin to add reference points based on Bing imagery.
Early September I went to Birmingham by car with people from OSM France to attend SoTM. Opportunity for me to also meet HOT members or contributors I had never met before or for a long time (Haiti 2010 after the Earthquake). Most of us then moved for the HOT House in Clumber Park during one week to discuss about various topics and review documents. Then Kate, Rafael and I went to MapAction base camp during one of their - impressive - exercises to discuss how to interact more with HOT - they were preparing a deployment in Karthoum. During these 2 days, we also discussed with Dale and Robert from American Red Cross regarding the design of an OSM mapping project they wanted to set in Northern Haiti. Back to London for a couple of days, kindly hosted by Harry, before going back to France and prepare the HOT project in Mongolia with Russell. The Mongolia project in Ulaanbaatar lasted 7 weeks between Early October and mid November and was a challenge regarding the language for us (now there is a Google Translate for Mongolian that did not exist in 2013), sometimes chilly but basically great time, with motivated students in Ulaanbaatar, but also under gers in the countryside during two week-ends. In the middle of this mission, that was supposed to happen earlier, I went to Malawi for one week as one of the Technical Advisers for the UNSPIDER mission. Likely the most crazy trip back and forth in my life so far, but also a great experience and the opportunity to advocate for OSM (see here). Once the project ended in Mongolia (see the Facebook group here), I directly joined Nairobi, Kenya, to attend ICCM, the International Conference for Crisis Mappers. Interesting discussions, in and off, like the one with Heather Leson from GISCorps, about working jointly in the future. During the third day, dedicated to Self-organized sessions, I participated to one about crisis in CAR and set one about the initiatives covering more than just Crisis response (Crisis response is not the only step involved in a disaster: preparedness allows to reduce its impacts, and once the emergency time is gone, recovery and reconstruction are also critical. Are there other initiatives than OSM that aim to this large response? How could they work together?). In the afternoon, Heather and I organized a Mapping Party in partnership with Joshua and Benson from HIU and a team from Map Kibera Trust. At least half of the participants had never mapped with OSM and raised many questions about its use in humanitarian contexts. Those who mapped contributed to the Haiyan Response. About this topic, I must admit I did not participate, except for a few edits, because of the huge number of committed people, making my contribution not essential, while it would have forced me to neglect the Activation for Central African Republic, also a United Nations-declared Level 3 humanitarian emergency (the highest one). The day after ICCM ended, I joined the DHN Summit in where all the participants (OCHA, DHN Members, representatives of local organizations) discussed about how to improve the DHN mechanisms and response (see the HackPad here. I left before the end to catch a flight for Senegal to join, as an individual, a sprint organized by the International Organization of Francophonie (= French Speaking Countries) to translate the Intermediate and Advanced Chapters of LearnOSM from English into French. It was the opportunity to work again with Nicolas and Pierre after the time passed together in Haiti, with former EUROSHA volunteers and to meet for the first time mappers from Senegal, Burkina Faso and Togo who have been mostly commiting in OSM since the Senegal project and the EOF projet. The Sprint week was intense but fruitful. Once done, many of us remained voluntarily a couple of weeks more, in order to provide mutual training on various OSM techniques, and organize training to students in Dakar and Saint-Louis. On December 19, I went back to Europe to join my family for Christmas greetings. Unfortunately, a political crisis arose in South Sudan and led to a HOT Monitoring. A new opportunity to work with HIU, but also GISCorps (whose one team mapped the city of Malakal) and start using uMap systematically to quickly show the mapping response over the country. An improved version for Central African Republic would follow soon after. But this is already 2014.
PS: if anyone quoted would like not to be or differently (eg with the link of his OSM profile or any other profile), please tell me. Likewise, if I forgot someone I should have quoted or a topic I should have talked about, please tell me too.
OSM mapping at regional or national scales: aiming at creating consolidated reference layers / La cartographie OSM à échelles régionales et nationales : l'objectif de créer des couches de référence consolidéesPosted by sev_hotosm on 15 February 2014 in French (Français)
Most often, OSM mapping is focused on areas of interest: a neighbourhood, a slum, a town or a city, less frequently a region, in which the aim is to get a multi-thematic map, more or less detailed, but generally more than the ones that are or could be available through an official way or a commercial product. Nevertheless, from this approach results a “gap map”, a kind of patchwork where highly detailed areas stand alongside others where the the strictly minimum is not even fulfilled. This inconsistency has consequences: * it is one obstacle for the official national agencies to adopt OSM or an opportunity if they want to criticize the project. These agencies aim to produce complete reference layers and do not operate like the OSM community through mapping parties or focused edits on Areas of Interests. So whatever OSM is very detailed in some places: if they take a bit of time to browse the map they may quickly reply that yes, but some regional capitals are missing and that the primary and secondary road network is quite incomplete, contrary to their reference layers. * it is also a limitation for the humanitarian workers to potentially adopt OSM. One easy incentive is the OSM data can used for routing in many devices and such data is frequently absent in developing countries. But if the road network lacks continuity and connectivity... It is all the more a pity that within the next decade, smartphone will likely expand as the GSM phones did the last ten years, and the local population could use them whenever to get around and find POIs.
This is why any OSM country project should encompass national scale reference mapping. It would be both useful and actually also quite motivating to to achieve various completeness steps. It does not mean at all replacing the usual focus on AOIs, that is definitely useful (HOT participates to it through the Tasking Manager jobs during a Crisis Response), but is a complementary to get a map consistent enough to have the minimum everywhere.
Of course ways to reach the aim have to be discussed. What follows is a first reflection. It might be better not to start a lot of reference in the same time over a same territory, but to move forward step by step, from the most primary reference features to those more secondary, the first ones providing the backbone for the latter and being often less numerous, therefore shorter in time to achieve. Checking the main administrative centers, adding the missing ones and apply the Admin level tags decided for the country will allow to determine what are the primary road connections and the OSM gaps. This done, the same process could be done for a sub-level, by checking administrative sub-centers and secondary/tertiary road connections. Once traced, it might be interesting to add the residential area for the villages and towns crossed by these roads and give them a name, from an existing POI or an external ODbL compatible source like the Geographic Names for Geopolitical Areas from NGA, even if this one is often considered as quite outdated.
The means to do this mapping efficiently is also an obvious topic. Tasking Manager v2 should allow to import geojson file with polygon, as it already exists for tools like Mapcraft, that limits their size to under 500Kb. For the road network, I aim at creating buffer layers from opendata road network. Each buffer would go from a town to another and represent a task within the TM job. Statistics applied to such jobs would provide useful information regarding the progress of OSM compared to these reference layers.
Texte en français
Souvent le mapping sous OSM est réalisé par zones d'intérêt : un quartier, un bidon-ville, une ville, rarement une région où l'on cherche à obtenir une carte multi-thématique plus ou moins fouillée. Cela permet de produire des cartes ou de la donnée là où il n'y en a pas d’officielles ou d'accessibles. L'inconvénient de cette seule approche est d'obtenir une "carte à trous", une sorte de patchwork sur lequel des zones extrêmement détaillées en côtoient d'autres même pas munies du minimum d'information. Cette carte manquant de cohérence a des conséquences : * c'est un des freins à l'adoption d'OSM par les instances cartographiques nationales. Celles-ci fonctionnent par logique de référentiels et non d'événementiel (dans le sens: on décide de faire la carte de telle zones), par thématique. Vous pouvez leur expliquer et démontrer que vous avez plus de rues dans la capitale ou telle autre cité, si elles prennent le temps de regarder la carte OSM et veulent répliquer, elles vous diront à raison que certes, mais au moins elles ont un réseau routier primaire complet sur le pays, ou que telle capitale régionale ou préfecture ne figure pas sur la carte * c'est aussi une limite par rapport à l'adoption d'OSM par les humanitaires. L'un des vecteurs faciles, c'est la navigabilité de la donnée OSM. Or les données navigables sont absentes dans les pays en développement. OSM en possède, mais elles sont fragmentaires. C'est d'autant plus dommage que d'ici 5 ans, les smartphones auront remplacé les téléphones et la population locale pourrait bénéficier de la navigabilité des donnés OSM pour se déplacer et trouver des POI.
C'est pourquoi il faut lancer des projets de référentiels à l'échelle nationale, étape par étape en allant du plus élémentaire (et moins coûteux en terme de temps consacré) au plus complexe. Cette approche ne vise pas du tout à remplacer celle par quartiers, qui a montré toute son utilité, mais constitue son complément naturel pour obtenir une carte qui contient le minimum pour être cohérente globalement tout en comportant des zones où le mapping a été particulièrement poussé, que cela soit lié à une nécessité (urgence, aménagement) ou le désir d'une communauté.
Evidemment les moyens pour atteindre cet objectif doivent être discutés. Ce qui suit est une première réflexion. Il est sans doute préférable de ne pas démarrer un grand nombre de référence en même temps sur un même territoire, mais de progresser étape par étape, depuis les objets les plus importants vers ceux plus secondaires, les premiers fournissant l’épine dorsale aux seconds et ayant également l'avantage d’être plus rapides à achever compte tenu d’un nombre d’objets souvent moins important. Vérifier l’existence des chefs-lieux administratifs, ajouter les manquants et appliquer les tags Admin level qui ont été décidés pour chaque pays permettra de déterminer les principales connexions routières et ce qui manque dans OSM. Ceci fait, le même processus pourra être réalisé à un niveau hiérarchique inférieur, en vérifiant les chefs-lieux de moindre rang et les connexions secondaires ou tertiaires qui les relient. Une fois ceci tracé, il pourrait être intéressant d’ajouter les zones résidentielles pour les petites villes et villages traversés par ces routes et ajouter leurs noms, à partir de POI existants ou de source externes compatibles avec OBbL, à l’instar de Geographic Names for Geopolitical Areas de la NGA, même si celle-ci est souvent considérée comme obsolète.
Les moyens pour réaliser cette cartographie de manière efficaces est également un sujet important. La v2 du Gestionnaire des Tâches devrait permettre d’importer des fichiers geojson de polygones, comme cela se fait déjà avec des outils comme Mapcraft, qui les limite cependant à une taille de moins de 500 Ko. Pour les réseaux routiers, j’ai dans l’idée de créer des buffers à partir de données ouvertes existantes et ouvertes (mais pas forcément d’une grande précision). Chaque buffer irait d’une ville à une autre et représenter une tâche dans un job du Tasking Manager. Des statistiques sur de tels jobs fourniraient un indicateur précieux de l’état de progression de la donnée OSM par rapport aux données de référence.