OpenStreetMap
I imported all the Mono County building footprints from the county GIS file, one of a handful of counties that have given us permission to use their data. Buildings are always good to have, and I wanted to see how they would look. I think San Diego county is the only other area in California with building imports. Hopefully we can get Santa Clara County buildings and files soon. I saw CAPIL had county zoning and parcel files, but I couldn't find any way to download them.
Unfortunately the upload crashed about half way through so there are a ton of extra points as with Santa Cruz. I will go in and clean them up with JOSM and validator as with Santa Cruz. It just takes forever. Also not all the addresses were in the shapefile, so many of the buildings have a 0 on them. And each of the buildings in a complex has a name, instead of one name for the whole area.
I think I need to increase the amount of memory with JOSM. Also I need to figure out a way to have the outer relations show up in the OSM files. I originally thought it might be a problem with the 2,000 node limit. In the rules file, I just define the outer shapes. I think I saw somewhere the inner relations weren't defined. I think there is a multipolygon. Should I try that?
Discussion
Comment from balrog-kun on 9 October 2009 at 01:07
Hi, before you start removing the fallout from the crashed upload by hand, try simply reverting the changeset that had failed, if no edits were made on top of it, it should be pretty easy. Or you can tell me which changes need reverting and I'll do that as I have the all gear set up for this.
Also instead of restarting the upload you could either edit your upload script to continue from where it had left off (that's what I normally do although I realise each script is different) or download the state after the crash from OSM and use the existing nodes instead of creating new ones by matching coordinates.
I recommend splitting long ways at 200-300 nodes and not coming near the 2k limit, makes working with the data easier.
Comment from nmixter on 9 October 2009 at 06:02
Thanks for the comments. The changeset is 2767175. Looks like the whole thing failed. I don't know how to revert it, so if it doesn't take too much time, I would appreciate it if you want to give it a shot. What script do you use? Do you the python or java shp2osm? I'm using the java version. What do you add to the script to have it pick up where it left off? Thanks.
Comment from nmixter on 9 October 2009 at 06:04
Thanks for the comments. The changeset is 2767175. Looks like the whole thing failed. I don't know how to revert it, so if it doesn't take too much time, I would appreciate it if you want to give it a shot. What script do you use? Do you the python or java shp2osm? I'm using the java version. What do you add to the script to have it pick up where it left off? Thanks.
Comment from balrog-kun on 9 October 2009 at 12:08
Looks like the revert worked! Hopefully nothing useful was removed :p
The imports I've done so far were not from .shp files so I had to use a custom script to convert (in Python, even though I can't say I know python). Then I upload using another python script (similar to bulk_upload.py) and that's where I can tell it to start from a given point instead of from the beginning.
Some more comments on the buildings imported:
* They look really cool, I think that's going to be useful especially with the elevation and storeys info -- one could potentially build some 3d models now.
* There are buildings with addr:housenumber=0, ele=0.0 or storeys=0.0, I think these tags can be safely removed where they're zero, apart from the names issue you mentioned.
Comment from balrog-kun on 9 October 2009 at 12:43
Oh another thing I noticed is that where buildings touch each other (share an edge) they would normally reuse the same nodes but in this data they have separate sets of nodes.
(The revert changeset is 2792386)
Comment from nmixter on 10 October 2009 at 03:52
Thanks for your help!! Looks good. Wish I would have known about that with other imports. Do you know if that process is documented on the wiki? Also, are you in California by any chance?
Comment from nmixter on 10 October 2009 at 04:04
I wonder if there is any way to automatically remove 0 values from tags. There's got to be a script somewhere.
Comment from balrog-kun on 10 October 2009 at 09:01
Assuming the .osm files you get from shp2osm have the xml formatted nicely, like what JOSM produces, you could remove zero tags using grep (that's why it needs to be formatted nicely so each tag is on a separate line), so you would grep for something like "key='ele' value='0.0'":
cat converted-shp-file-before-upload.osm | grep -v "key='ele' value='0\\.0'" > file-stripped.osm
Before upload because otherwise you would need to also add the action="modified" tags to every object modified. The -v inverts the matching so that matching lines are skipped instead of non-matching lines.
You can also do such things with JOSM and its search function, but scripting it seems easier.
(I'm in east Europe.)