OpenStreetMap

Imports

Posted by TheDude05 on 18 August 2017 in English.

I recently received notice that a changeset I had made had been reverted due to not following the protocols in place for importing data. First the facts of the matter. My family was going to vacation at Folly Beach in South Carolina and I thought it an excellent opportunity to do some mapping while in the area to verify local businesses and add information about them to the map. When viewing the area in OSM I saw that only a few buildings had been mapped and that the majority of POIs were nodes. I decided that I would see if I could find a building footprint file already created from the county so that I could add the buildings to the map and then replace the node information with tags connected to the structures where a single occupant resided within the buildings. I find this to be cleaner when it is able to be done rather than have closed ways and nodes trying to describe the very same thing. Now I will admit that I had some confusion and very much a lack of knowledge of what constituted an import and the process that was in place to regulate them. I found a file with the county for building footprints, exported the buildings for Folly Beach using Qgis, cleaned up the attribute data so that when bringing them into JOSM they would have the tag building = yes and proceeded to copy them into the main layer. After this was done I scoured the area for duplicates and resolved any there were, removed nodes that had POI information and instead attributed that information the the closed ways, and used the error checker to insure that I had not introduced any errors. I believe it was necessary to move some roads that ran through buildings due to geographical error but other than that it was nearly a blank canvas. I posted the changeset and readied myself for my vacation.
On arrival and with the mind to do some mapping in the wild I loaded up Vespucci and began downloading the data so that I could get to it. I was amazed to find that the buildings I had placed and properly attributed had been removed. When I arrived back at home I checked my email and found that a user had redacted my changes and rolled everything back to as it was before I began editing, citing a lack of a wiki page or discussion within the import mailing list. I searched out the import page to see if I had been in violation and it would seem that I had been. Mea culpa, mea culpa.

Now based on the directions at the import site I can see that if a group was to be doing very large and numerous imports the instructions make complete sense, especially in regards to large geographic areas. The need to insure that data is duplicated or error is introduced has been addressed and I see that there were lessons learned the hard way. Based on the instructions included it seems that the idea is about very large imports along a county, province, state, country, or large city where the potential for error is far greater and much data may already exist. Folly Beach is, according to Wikipedia, 18.9 square miles, the data that existed were a few nodes, the beaches, the wetlands, the roads, and a couple of buildings. The chances of error were minuscule and the import, which I acknowledge failed the guidelines, was checked over for error. My problem is that the wholesale redaction of the changeset did not actually look at whether there was an improvement to the map (I believe throughout the OSM community it is generally found desirable to have buildings represented) nor if any error had been introduced. It would seem rather that a prominent individual within the community has created an account that is solely dedicated to a vigilante style role of cleaning up what he describes as vandalism and what he deems to be bad imports and mechanical edits. Now while this role may be a valid and needed one I do find it intriguing that while there is so much oversight in the import process, one individual can decide for himself without discussion or debate that a changeset is either vandalism or an illegal import or is damaging to the map. Be that as it may I submit myself to that editing as I had seemed to violate the protocols in place.

As to the protocols themselves I find that while it does seem to make sense to prevent error or duplication, it has a chilling effect on mappers and on entities that may wish to upload their data to map. In the United States there are 50 states, 3,007 counties, and 19,354 incorporated places that are, for the most part, independent of one another. This also does not take into account that each of these entities would have multiple commissions or departments that could have interest in uploading data to the map. What the import guidelines would mean for these groups, should they wish to share data with the map, is that they would each create a wiki page for each import they wish to do, they each create an account specific for the import, and that they would submit their import to a mailing list for approval. Having worked in the public sector for small government I can verify that the majority would not want to go through this hassle every single time they wished to share data with the community at large, nor would they have the time at their disposal to do such. This leads to less adoption of the map and a loss of great data for the map. The question would be what to do about this.

We could hope that the many different entities would work in cooperation to limit the amount of import wikis and requests by combining their efforts or even forming a committee to combine their efforts thus making the requests less numerous (presupposing mass adoption of the map). This could happen and it would be great, but I do not see it in the near future as many have no idea about the map or about contributing to it. The second proposal that I would humbly submit is this, allow a trusted partner status to communities and groups much in the same way that Google has done with their map. Communities have a great desire to see their communities represented accurately in online maps. In two of the local governments I worked for we had this partnership with Google, which did not exclude us from scrutiny but rather gave us the benefit of the doubt that we knew what we were talking about. With this process an entity would need only submit once for the right to import, committees that accept the applications can make a stress of what imports are allowed and the requirements the group must follow, verify that the group is legitimate, and then approve them to maintain their AOI on the map. I think this process could lead to greater adoption and more accuracy for the map in the long run.

Whatever may be decided on the future of imports I still acknowledge my error and humbly offer my apologies for breaking protocol in my efforts to strive for the bettering of the map.

Location: Folly Beach, Charleston County, South Carolina, 29439, United States

Discussion

Comment from maxerickson on 18 August 2017 at 22:47

I’ve come to look at the import process as an up front quality control step. Of course quality assurance could be done by finding problems with imports and then handling the problems (either by removing and redoing or by fixing the data directly), but that pushes the QA work from the people who believe the new data will improve the map off onto whoever happens to look for and notice the problems.

In this case, one thing the import process probably would have raised is that the county has parcel data with addresses and property use information (the property use is shown in the “Class Code” field at http://ccgisapps.charlestoncounty.org/public_search/ ). For a lot of data users, the address information is more interesting than the building footprints, and some effort to set a more specific value than building=yes is always nice.

Comment from TheDude05 on 18 August 2017 at 23:08

My goal was to match the buildings with use while in town, the majority of them being residential houses excepting the downtown area where an on the ground evaluation would be appropriate. As for the import process I don’t disagree something has to be in place but for consistently good contributors a stream lined process may be needed. If a city does a large import through the process they would have to go back through it anytime they wished to maintain it. An example would be new subdivisions within a rapidly growing city. Frisco Texas has often ranked high in lists of fastest growing cities in the U.S. and if they wished to upload new addresses for a new subdivision they would need to use this process even if it were for only 30 items. Same would be said for placing the building footprints. If they have already created these items in ArcGIS why would they want to have to do it a second time to upload it to OSM, especially when they had new subdivisions coming on every couple of months? That is where is see an issue.

Comment from Glassman on 18 August 2017 at 23:16

Sorry you discovered our convoluted import process in this manner. As [maxerickson](https://osm.org/user:maxerickson} noted, the process is designed to improve the quality of imports and to insure that the data being imported is compatible with the ODbL license. The process at times helps the quality of the import by improving the tags, reducing nodes, improving the geometry and as point out, adds suggested data like addresses, that can also be imported.

I would encourage you to join the [OSM US]{https://osmus-slack.herokuapp.com/) Slack import channel. We can help you through the steps next time you want to import.

FYI - You could have opened up the building shapefile as a layer in JOSM and just copied in the data. More work, but not an import. Licensing is still and issue.

Clifford

Comment from TheDude05 on 20 August 2017 at 23:12

Clifford, I did open the shapefile in JOSM and copy it in, the person who redacted it called that an import.

Comment from amapanda ᚛ᚐᚋᚐᚅᚇᚐ᚜ 🏳️‍🌈 on 21 August 2017 at 14:51

Hi, sorry you had a bad experience.

Just a few thoughts, for some people, they want there to be a chilling effect, they don’t want imports to be as easy and streamlined. I’m not one of those people, but I’d lean close to that way.

It’s not up each city/county/state to import the data, an independent mapper can do it. If you’ve done a few of them, it gets easier and you know what to do. For an experienced mapper, the import guidelines aren’t too strenuous.

You could even combine many county datasets into one large dataset and import that! There’s no requirement that there is only one source dataset. That is harder because then there are many copyright and data quality issues to confirm.

If a city has 30 new addresses, surely it’s easier for someone to manually add them to OSM, then you don’t need an import!

It would seem rather that a prominent individual within the community has created an account that is solely dedicated to a vigilante style role of cleaning up what he describes as vandalism and what he deems to be bad imports and mechanical edits.

Yes, people make separate accounts for data clean up work like that. “vigilante” is a little strong, OSM is kinda anarchistic, everyone’s a vigilante!

Log in to leave a comment