Zaatari, in northern Jordan, is facing huge humanitarian pressure from the Syrian conflict, considered the second largest refugee camp in the world. To aid the response and improve coordination of geographic data in the camp, a remarkable cooperation is underway there among UN agencies, the OSM community, and potentially, the refugees themselves.
As the Syrian crisis deepened, HOT was prompted to activate in January. It's naturally proven difficult to coordinate remote mapping inside Syria, though there has been significant and interesting activity over the past two years.
We've known Edouard Legoupil from UNHCR for a while, through even pre-Crisis Mappers days. We personally overlapped in Nairobi at the start of Map Kibera, and since, we've discussed ways of bringing open approaches to mapping and engagement in camps. Likewise, Lars Bromley from UNOSAT has been a colleague since we collaborated together in Darfur Google Earth layers project, and while he was at AAAS, supplied detailed historic satellite imagery of Kibera for our analysis of spatial extents of post-election violence. Both have innovated, very close to where things are built, and have created space within their organizations to work collaboratively. With the HOT activation, we've been able to come together and do practical work on those ideas for Zaatari.
The unhrc-jordan OSM user has imported roads, toilets, water points, health facilities, mosques, schools, offices, even restaurants. Another partner organization, UNICEF Jordan also uploaded child specific infrastructure (though this was later undone, and re-uploaded by UNHCR for some technical reason). Now, UNHCR is working with REACH to map on the ground in Zaatari, both confirming data and including "informal" infrastructure built by refugees; all the normal services of a city are growing in this place. In the future, there may be opportunity to engage refugees themselves in the processes. In traditional practice, data is shared through colleagues. Through OSM, they become openly collaboratively shared.
UNOSAT is tasked with another piece of the geographic puzzle in Zaatari. They analyse satellite imagery to identify ALL structures in Zaatari, currently at 28174 structures. Lars confirmed that this data could be imported into OSM as well, and that I could help with that process through the UNOSAT OSM User.
Working off the July data, the first challenge of the import was transforming the ESRI Geodatabase released by UNOSAT into a format I could work with. Luckily, I was at OSFAC's office in Kinshasa where I had access to ArcGIS, and could do the conversion to Shapefile. Then, to my surprise, JOSM was able to read the Shapefile directly! In JOSM, I used the under-appreciated filters feature, to select only features with Shelter_St=1 (meaning the structure currently exists, the data contains shelters that no longer exist). Then, mapped the Structure_ column to either shelter (for 1) or administrative (for 2). Also stored the acquisition date, event code, and source, for reference (within a unosat: tag space). And also, the objectid, which could be used for applying updates later on. Removed the other tags. And because of the large size of the upload, used a script rather than JOSM. Here's an example structure from the import.
So right before I was ready to do that import, Lars notified me that UNOSAT had released a new analysis. I could just redo the process above with the new data, or use the opportunity to test if it works to update based on unosat:objectid. Yea, did that!
Again, stuck with a GDB, until I was fortunately introduced to the simple and strong GDB Flee. From there, I went about writing a conflation script, which reads in the shapefile and OSM data, detects actions (create, update, or delete, either way), and then outputs OSM XML for examination in JOSM. There might be other tools to do this (please let me know in comments if so), but was useful for me to get my head around the process, a first step at abstracting conflation, and dig into some geo mangling code.
It's not fully tested against all cases, just the situations in this conflation, so putting together proper tests for this later is on the agenda. In this case, after the deletes and creates, the remaining number of structures added up precisely to what UNOSAT reported, so looks fine.
The thing to grapple with here are unexpected situations. The script deals with this by outputting OSM features with a NOTICE tag. Then in JOSM, can use filters to only show those features, and inspect and make decisions manually. For this import, there were about a dozen features that were supposedly collected in the previous analysis, but hadn't already been imported. This was strange, but didn't investigate further and since the numbers added up, decided to include them in the import.
Interesting results immediately came out from comparing multiple datasets, and putting many eyes on the data. Those dozen "missing" features were highlighting in JOSM Validation as Nodes at same position. The precisely same lat/lon, but different objectid tags. I confirmed they were definitely present in the current set of features, and that the total number of featues matched what's reported by UNOSAT. So perhaps I can conclude there may be duplicates in the source analysis, and hopefully this community view on the data can help improve the authoritative view.
Also interesting to look at how the UNHCR and UNOSAT data interact. Administrative buildings or compounds are polygons in UNHCR, points in UNOSAT. Some compounds contain multiple administrative structures; it might be interesting to associate them via a relation or tagging. Some shelter structures show up with administrative areas; do those structures need to be reinterpretted, or have the areas changed? Now that these are together in a single database, we can script or use GIS to analyse those kind of issues and prioritize further investigation.
One thing that might help is for OSM users to access the imagery directly. In fact, the most recent analysis come from State Dept HIU supplied imagery of Zaatari, so we can perhaps request access through Imagery to the Crowd. We'd want to have some good analysis first, and a clear idea of how mappers could help.
Finally, there's potential to do custom rendering which highlights all the data, and camp specific information. UNHCR is already rendering Zaatari on their data site. It would be good to also show the node buildings in an unobtrusive way, to show how the admin structures and shelter distributions interact.