Recent diary entries
Talking of the OSM Contributors, we often see the Big Numbers. In this Diary, my objective is to focus on the OSM Contributor profiles, to try to measure the impact of various groups on the OSM Edit Contributions.
Since 2005, there has been an explosive growth of new OSM Registered members from 500,000 in 2012 to 1 million in 2013 and 4.2 millions at the end of september 2017.
Pascal Neis and Alexander Zipf study in 2012 showed that only 38% of the registered members at the end of 2011 had started editing the database and that only 5% (24,000) of all members actively contributed to the project in a more productive way.
Activityworkshop.ne published in july 2013 an interesting analysis of contributors «Joining and leaving» as participants. It shows the volatily of OSM contributors with a high volume of contributors starting and stopping contribution shortly after. As we will see below, a high percentage of people that start to contribute stop the first day or after a short period.
There are also various studies that show the contribution inequality with most of the data produced by a minority (see Anran Yang, Hongchao Fan, Alexander Zipf, 2016 and Ding Ma, Mats Sandberg and Bin Jiang, 2015). Statistics and analysis presented by Pascal Neis and Simon Poole over the last years did also show various aspects of the contributions, with the concentration of Contributions by a minority and the volatility of contributors participation.
The OSM Changesets Dump File contains metadata about each changeset edition. Like others, our analysis comes from this file. If we dig in and analyze the OSM changesets database, this shows that for the 13 years from 2004 to end of september 2017, 953,200 contributors edited at least one object and 108,800 edited more then 1,000 objects (ie. node, ways or relations). This is an indication that there are massive inflows of new participants that contribute minimally. The analysis below will confirm this hypothesis.
The Pulse of OpenStreetMap Contributors
Cohort analysis let’s break a dataset into related groups that share common characteristics or experiences. For OSM, we can group contributors by the year they started to contribute and compare the various cohorts to see patterns of contribution.
The graph 1 reveals what I call the «Pulse of OpenStreetMap Contributors». Rodolphe Quiedeville OSMPulse website did also illustrate the beat of contributions. While his real-time graphs (Last update in 2014) did focus on the number of objects edited minutely, we focus on the contributors with the same year of experience. For each calendar year, this is like if we did organize a marathon will all the contributors aligned on the same start line, looking at their progression month by month. We could also follow them for even longer periods and compare their long term behavior.
Graph 1 Note that this graph do not show a long timeserie. These are individual graphs for each yearly cohort. For each year, we follow for 12 months the new contributors that start editing.
Here for each calendar year, we follow OSM new contributors and participation from their month 1 to month 12 of contribution (it does not matter if one started in january, feb. etc). With such cohort analysis, we can see all the new entries for the year. This reveals what I call the Pulse of «Discovery contributors» with the great majority that do not participate more then 1 month. The high rate of departure at month 1 confirms the volatility of contributors participation. It shows what is called the lower tail of Contribution with a high number of Contributors with a minimal impact on the OSM edits. There is a lot more to say from such analysis and I will come back in an other Diary with more profile analysis from the cohort trajectory statistics.
Simon Poole published in his OSM Diary various examples that show the variability of inflows of new contributors and how it is not always related to significative edits. The sudden increase of contributors in early 2016 that we can observe on graph 2 below comes from Maps.Me editors where many of them did map personal infos. At the end of 2016, thousand of faked accounts were created in USA by SEO companies. OSMstat for 2017-09-30 shows also indications of various profiles with 5,413 active contributors and 3,301 with node edits > 15.
Monthly Statistics – Let’s color with Contributors Profiles
Lets’ now add Days profiles to the monthly statistics and help better see the heterogeneity between the Contributors and the Contributions. Pascal Neis OSMstat website and Simon Poole stats on the OSM wiki OSM wiki let us observe monthly statistics of contributions. We observe since the beginning of 2016 an average of 25,000 to 50,000 active contributors per month.
Graph 2 combines the monthly statistics of Contributors (ie. have edited in the month) and Contributions (ie. number of objects edited node, way or relation) from the OSM stats wiki. We color these charts with the Contributors profiles based on cumulated days of participation since the first edit to OSM (Pascal Neis classification). The comparison of the two graphs let’s observe the concentration of Contributors in the first two classes and the concentration of the Contribution in the last class.
Profiles of Contributors and Contributions by month for 2017 up to september on Graph 3 let’s measure the respective percentage of each class based the cumulatives days of contribution. For 2017-08, the first two classes «Discover 1-2 days» (19,186 contributors) and «Rarely Active 3-14 days» (12,845 contributors) represent 66% of the share of Contributors. In comparison, their share of Contributions (13%) is relatively minimal. The «Discover» class with 3.5% of Contributors corresponds more or less to the «Pulse» we observe on the cohort analysis.
The other tail of distribution is represented by the 4,000 contributors that are part of the «Mega Active» class (271 days and more). They represent 8% of Contributors and 37.6% of Contributions.
Pascal Neis Contributions of the yearly cohorts graph on his 2016 yearly Statistic Blog, shows the respective importance of each yearly cohort on the level of monthly Contributors. In this case, the cohorts are not aligned from month 1 but colors let's see stratas of contributors by the year they started to edit. With this representation, the peak of the yearly new contributors is less acute, being spread in the month they started editing. The top of Graph 4 reproduces Pascal chart. Every year, we observe the jump in the number of contributors, and their relative importance that reduce gradually in the next years. Again, we see the rise of Contributors from 2016.
The Contributions Profile at the bottom of the Graph (ie.Percentage of Contributions by months) reveals that the first year of participation, the yearly cohort of new contributors represents nearly 40% of contributions, that share reducing in the following years. With the rise of Contributors in 2016 and 2017, we observe also a rise in the share of Contributions. Simon has measured that the rise of Maps.Me Contributors had a minimal impact on the share of Contributions. More analysis will be necessary to explain which categories are responsible of this jump.
I hope that this different angle on the Contributors data will hep to better understand the various contributions to OSM. Do not hesitate to comment. And I plan to continue such analysis in other Diaries.
HOT successful growth over the last five years is recognized by the various medias and humanitarian organizations. HOT was successful to develop projects in partnership with various organizations and to mobilize developpers and mappers volunteering or contracting in supporting different programs and actions. There were Projects in Indonesia, Haiti, Africa, and Local community development actions. To support these activities, Tutorials, Software development were other dimensions of this action. The Tasking Manager is the example of a tool developped with the support of volunteers and contractors, experimenting with various partners. The Activations and other Programs have contributed to develop workflows to better respond to various problematics in the humanitarian sector. This was possible with both the support of skilled volunteers, staff and the synergy with the partner organizations. As we progress to develop projects with the partners, it is essential to assure to maintain a good coordination with the HOT community, A great part of the dynamism and success of HOT comes from this ecosystem with these highly skilled mappers and developpers that develop tools, orient projects, interface with humanitarian organizations to adapt to their needs and find ways to collaborate.
OSM is now the DeFacto map for international humanitarian responses. But for HOT to grow successfully, we should surely not count only on dedicated Activation leaders, Developpers or Staff. The same with the Board of directors or the Partners. We need to work together. The Board of directors represents the membership and assures the management of the organization with the support of the staff. They cannot surely report all actions or detail of contracts and staff management. And they have to delegate some responsabilities to the staff.
But the Board cannot drive such an humanitarian organization like a private company not accounting to anyone. But Accountability is an essential element of the Governance of such humanitarian NGO. It is essential to report to our shareholders, the membership, to discuss the major orientations, to use the internal ressources to better define our projects and orientations. The Governance is an important aspect of this election. We have to assure that the Board of directors is accountable to the membership and operates with openess and respect.
In the technological and very dynamic sector where we operate, Partnership is an important aspect. Partners are playing a great role collaborating at the design of projects. sometimes involved in conjoint FundRaising actions.
But we need to reinforce this member-based organization, assure that the membership can play his role and that the major orientations and FundRaising are not controlled by partners that could have conflicted interests.
I am retired after a carreer working for government agencies in Canada. My contributions over the last years clearly show my support and expertise for the development of HOT, my engagement for this organization, my support of the local communities, my volunty to progress with field work in the context of various projects and the various humanitarian emergencies. My previous diary (http://www.openstreetmap.org/user/PierZen/diary/34570) shows how as a leader of Activations I contributed to professionnalize our responses and play a major role contributing with the UN Agencies and international organizations. OSM is now the DeFacto Basemap for such responses. I have lead activations for DR Congo, Mali, Haiyan/Philippines, West Africa Ebola, Vanuatu and Nepal. The West Africa Ebola OSM Response was the opportunity to develop with MSF-Switzerland and CartONG an innovative approach for interventions in the context of epidemies, collaborating with their field teams to adapt constantly to this epidemy.
Thanks for your support and involvment in HOT.
My journey at OSM / HOT
The Haiti Earthquake in january 2010 let me discover what it was possible to do remotely to support humanitarian actions. I jumped in rapidly and used my various skills to support this first HOT activation. I remember these updates where we saw first Schuyler and Tom Buckley and then Nicolas Chavent and Robert Soden in action, the interviews in the tents, the HOT kits, the workflows built with the UN agencies.
On the steps of these initiators of HOT Activations, I started at the end of 2012 to coordinate a serie of major activations where the core coordinators we supported remotely the international community. With the rebellion in the Kivus, north of the Democratic republic of Congo, hundred of thousands of people were fleeing on the roads. Nicolas who had worked previously for the UN Agencies (WFP and UNJLC/Log-Cluster at inter-agency level) organized the contacts with OCHA and asked Claire to assure the liaison from Kinshasa.
We had the objective to professionalize our activations, to gain confidence of the various international organizations and convince them to collaborate with OSM and HOT, to make OSM the Reference map in the context of disasters.
In parallel of the Congo Activation, I started The Mali activation two months later. Andrew Buck then joined in and Severin started activations for the Central Africa Republic (CAR) and South Sudan. At the same time, Jorieke and other Eurosha volunteers in CAR had to move out rapidly because of the insecurity.
With the Activations, the support of local communities and discussions, collaborations with the Red Cross, MSF, Eurosha, Espace Francophone projects, a place of innovation and experimentation was gradually implemented. We also proposed Design improvements of the Task Manager to adapt it to the context of Activations, offered various Daily exports, looked at using mobile devices for data collect, contributed to software experiments, design, development.
The core coordinators, we volunteered to develop gradually this model of activation that showed such success for the Ebola Outbreak response. We documented the wiki project page for each activation, advertised the OSM maps & products, assured daily updates of various exports, documented the Task Manager Jobs and negotiated for constant adaptations of this tools that is central for supporting these remote activations through internet. Jean-Guilhem Cailton and Fred Moine were also very supportive and reactive about imagery processing.
We assured the remote response mapping, obtaining the imagery from various Imagery providers, coordinating with the international organizations, convincing sometimes the developers to adapt their tools to the particular context of our activations. I bloged and emailed regularly on the activations on the hot website and the mailing list and participated to international conferences making outreach about our activities.
In parallel with the activations, we also worked to innovate in data collection and technical remote support of local communities. We also established contacts with the Red Cross and MSF, discussed about ways to progress. This was later beneficial for the Haiyan/Yolanda Activation in november 2013.
For the Mali Activation in early 2013, we implemented an Imagery Crowdsourcing application to have the capacity to spot villages spread in either flooded or desertic areas and difficult to view. Pierre Giraud adapted the Tasking Manager to import the polygons of the various residential areas. This helped focus around the villages, locate and trace the highways connecting to these villages.
The Haiyan typhoon that hit Tacloban city in november 2013 was very destructive and cut all communications and electricity. Our Red Cross partners were waiting to deploy in Tacloban, not knowing what was the exact situation on the ground. The international community was rapidly mobilized, satellites re-oriented to obtain post-disaster imagery. OSM, Copernicus and UNOSAT did Post-disaster evaluation to facilitate the logistic of the organization.
For that Activation, we coordinated with Maning Sambale (HOT/OSM Philippines) who managed discussions with the local governments about Opendata and interacted with the government and the international agencies and mobilized the OSM-Philippines. We had a very effective coordination with the American Red Cross, the UN agencies and the international organizations.
The first 10 days of the activation were impressive with the OSM developers adapting the Humanitarian style to show damages to structures and adding this style to various tools. This was quite a satisfaction 10 days later for all of us to see the Poster size maps delivered to the OIM at Tacloban airport. This is crisis response, with a lot of adaptation, listening, innovation.
In 2013, participating to a training mission in north of Haiti and to the Espace Francophone LearnOSM translation sprint, was the opportunity to be confronted with the field reality, the cultural context and the technical challenges we are faced in such missions. Very satisfactory experience with great friends. This is the type of experience we should give the opportunity to the more engaged volunteers of our organization, to help them progress and adapt to the reality of the various countries where we operate.
My 2014 HOT Journey
The Ebola outbreak in Guinea spread rapidly in march 2014 and for the first time in urban area and large territories. In the context of epidemics, the contact tracing is a crucial operation. MSF epidemiologists were faced with blank maps. Village names were an other problem with duplicates or variants. Looking at the Haiyan experience, they thought that GIS support and OSM maps would be beneficial to this response. CartONG GIS specialists worked in the field with the epidemiologists and had coordinating with HOT. The first imageries were bought by MSF and the OSM response was simply fantastic, three cities being mapped in one day. Rapidly the Red Cross joined in, HIU, MapBox and Airbus Space & Defense also provided free imagery. Volunteering over all of the last year for this Ebola response, was for me and surely for others both challenging, sad with the spreading of the epidemic, these continuous deaths with this very complex epidemic. But at the same time this was rewarding looking at what we can accomplish in such circumstances, how we can progress to coordinate with other organizations and make the difference. After Haiti, the West Africa Ebola activation is a reference, not only for the more then 15 millions objects edited and the 3,000 contributors from more then 100 countries, but also for the efficient of collaboration we implemented with UN Agencies and international organizations.
People that want to know more about me can read my HOT updates and my conference presentations on SlideShare.
My vision of HOT
This is a year of transition in the HOT organization with the majority of the Board of directors not renewing their term and the executive director announcing this week her departure. Thanks to all for their contribution to this organization.
Many of this year Board Candidates have significant experience and engagement with HOT, either on the Board mounting projects, interfacing with the partners. While we discuss for this election, I am convinced that the previous Board, the Executive director and the Board candidates can sit together and plan a smooth transition, assuring that the essential informations are transferred to the new Board and that we maintain the momentum of HOT.
Now that the nomination period is closed, the HOT membership will continue privately the discussions about orientations of HOT. I hope that this will be done with respect and openness. These annual meetings are also to review our processes, to question and assure that we adapt smoothly, that we assure to stay dynamic, motivated and innovative. Accountability, Openness, Respect are for me essential values in an organization like HOT with a mix of passionate and engaged volunteers, contractors and paid staff from which we benefit expertise. We have to be careful to offer a place of discussion where members can recognize themselves.
To meet once a year to select a Board is not enough. The new Board should look at how the information circulate better, the expertise of the membership is more effective to orient the projects. We should interrogate ourselves how the Working groups or other participation channels can facilitate the HOT operations with more connections between the volunteers, the contractors, the ED and the Board.
With the complexity of projects running, the share of expertise among members, the development of internal capacity in organizations like HOT is an essential factor of success. HOT is an organization with technical people having expertise in various sectors. We interconnect with various UN agencies and international organizations both to develop softwares, projects, collect field data, discuss about building information systems, collecting data from various governments and organizations. Organizations like HOT with both paid staff, contractors and volunteers should not be thought only as a hierarchical structure with the Board, the executive, and the contractors. There are surely confidential matters that should be dealt only by the Board and the ED, but we need to bridge from the various projects. It is essential to assure skill transfers and revise the actual rules of governance. We should take care to have the mechanisms that assure the relay of information from one to the other, and the capacity to revise our projects if necessary and assure a constant progress in this context of innovation and complex organisations sharing work with the the international organizations.
Let me take the example of an activity I was quite involved over the years with HOT. Since the start in 2010, the Activations offered our community the possibility to identify a clear space where we can collaborate with international organizations, innovate and have a significant impact mapping to support their deployment. As one of the core Activation coordinators since 2012, I am proud of the model we developed coordinating with UN Agencies and international organizations. We need to consolidate this action, by adapting the softwares to the needs of the activators and progress with our partners in field data collection. Tools like ODK can favor more exchanges between the humanitarian organizations and OSM. We can say that as volunteers for the OSM Ebola response in 2014, we impacted positively the organization. This opened doors for more collaboration and facilitated access to resources such as the 200,000 USD donation by the Hewlett Foundation, all of these allowing to deepen the impact of remote HOT crisis activations for both Humanitarian and Development actors as well as local communities.
I agree with Jorieke saying that the heart of HOT is this community and that the Board actions should not be isolated from the community and with other candidates that say we should do more community support and preparedness.
I also agree with Severin and Nicolas vision about how to reinforce this member-based organization with a more transparent organization.
We need also to identify clear objectives how the various activities may globally make the difference with common objectives about obtaining OpenData necessary to have more accurate maps. They should be integrated with a vision of which informations are the more crucial in the context of disaster and for economic development in general. Territory management by local authorities or by the international organizations supporting in case of disaster need solid Opendata about infrastructures and administrative limits. Obtaining such Opendata from various governments is often uneasy. But more we develop interelations with OSM local communities, universities, local government, more we will progress to answer these questions. For the Haiyan Typhoon response in november 2013, I saw how useful was the OSM community with Maning Sambale coordinating with the remote coordinators and interfacing locally with the international agencies and local governments.
HOT is an innovative organization that has multitude interactions with other organizations. The Board, the Executive, the contractors and the volunteers all have significant expertise and assure the progress of HOT in interaction with the UN Agencies and international organizations. We should assure a synergy of these various parts of HOT, reinforce the participation of the volunteers and contractors to project design, orientations, discussions with partners, evaluations. This to assure that we progress both with innovative products and projects, this in complex environments in interaction with our various partners.
We need Board directors willing to work with this community in a less hierarchial organization, discuss orientations, the progress of the projects, and assure the development of the organization.
We do not need to change fundamentally the structure of the organization, but to assure more bridges between the Board, the Executive and the membership, be more accountable, more open to discussions. If we can have more possibilities for the engaged members to participate to some field missions, this should also be quite a motivator. These are conditions that should favor the membership participation, more exchanges and assure constant innovations in the organization.
How I can play a role on this Board
I have previous experience as a Board director and managed the major activations since 2012 interacting with various organizations.
My grounding to the various realities of HOT action, my involvement both in action, organization, innovation, development, the support to both the humanitarian organizations and local communities, gives me the capacity to evaluate, orient projects, discuss with partners and the community. My experience in the various sectors of activity of HOT and dealing with partners is an asset for the organization.
In this year of transition, we need experienced people in the field of activities of HOT with a vision of the organization progress, pragmatic, able to sit both with the partners and the community, to continue in the action like we ever did.
HOT is a techy volunteer organization I am proud of and that I want to assure it will progress well.