OpenStreetMap
On January 16 of this year, the Peruvian company Guiacalles requested to delete private data that was added in OpenStreetMap without considering the copyright.
This copyright infringement was due to an import of data by TELCOM IP that was carried out in Peru 7 years ago. Since it was impossible to verify the original license of the imported data, the DWG did a redaction to delete the original data. This affected 28 cities of the country, removing everything from main streets to pedestrian streets, and loosing all the edits of the OSM community that were made on top of that data in the last 7 years.
Post redaction view - JOSM editor
Our workflow during and after the redaction
The #osmPe channel on telegram was the main communication channel for coordination of the whole redaction process, as well as tickets in the repository of osm-peru-redaction.
As soon the redaction finished, DWG posted a task with the affected areas. Based on this, the community documented a workflow that allowed us to coordinate the mapping and restore the map to its old state.
Task to review and map the removed highways from Peru
In addition to restoring the geometries of the roads, the road names were added using a data source from the government which was incorporated into JOSM as a WMS layer.
Mapping workflow to restore and add highways name
What should we keep on mind?
OpenStreetMap is a collaborative project that any user can contribute to, but we should always make sure that all the added data has a license compatible with the Open Databases License. These reversions are detrimental to all those who contribute and use OpenStreetMap. Many data was lost which leaves our maps in bad condition.
On a more positive note, it does speak to the strength of OpenStreetMap that we were able to recover from this redaction in such a short time. The community came together to add and modify 96,000 highways throughout the country, leaving the map in a better place than it was before.
Discussion
Comment from Omnific on 2 May 2018 at 22:04
Shocking loss of work. There has to be a better way to revert than just deleting the work of everyone.
Not to diminish the work you all did, just horrible that DWG couldn’t save any of it, such as at least the tags that have been added over the years (not in the original import). There has to be a less ham-fisted approach to reverting data.
Thanks for the effort, all issues notwithstanding.
Comment from Chetan_Gowda on 3 May 2018 at 07:14
This is bad and amazing at the same time. @karito awesome summary.
Comment from Dalkeith on 3 May 2018 at 11:03
Congratulations on the inspiring work to get back to a good position.
It shows several things input of data is getting better and more automated with cross referencing to other sources proving very powerful. At some point we will probably be able to automate 99% of the physical data from things like Bing Map. Not sure how far away that is.
Comment from LivingWithDragons on 3 May 2018 at 12:26
Wow this is quite an interesting topic of trouble + amazing response.
It would be great to have it talked about at the State of the Map conference. We’ve already selected talks, but there will be some slots to sign-up for 5 minute lightning talks at the conference. If anyone involved in this is going to SotM, then perhaps they want to prepare a brief talk about it?
Comment from pnorman on 3 May 2018 at 17:46
The redaction code saved as much as was possible. The problem is that there wasn’t much that could be saved.
Comment from mikelmaron on 4 May 2018 at 10:15
Incredible work @karitotp and OSM Peru.
I agree both that there was not much more DWG could directly do, and that more could have been done. The redaction code is well designed, and did what was needed, and needs to be run in this kind of situation. More could be done to prepare for the repair before the redaction happens, but the DWG does not have extra time or resources to do that.
Some communities do – and I recommend we allow a bit more time and find ways to be more collaborative to prepare before the redaction starts. Things like extracting tags that are not compromised by private data, and building a service to refer to while repairing the map; Set up metrics to track how close the repair is to prior data coverage; develop tasks and workflows.
I guess it could also be argued that the community wouldn’t have responded so fast unless the data was just deleted. I think with a deadline for preparation of 2-4 weeks, and good documentation on how to prepare, communities could be even more ready for situations like this.
Comment from PlaneMad on 4 May 2018 at 13:10
Fantastic and inspiring how the Peru community got together to repair the broken map!
Comment from muzirian on 5 May 2018 at 08:00
Awesome work getting it back and nice write up :)