A complete map
Posted by Minh Nguyen on 24 August 2015 in English (English). Last updated on 6 May 2017.I saved my 10,000th changeset yesterday, as part of a months-long surveying and mapping spree in San José, California, where I currently live.
I never intended to map the Bay Area. Instead, I typically spend my free time helping to map my hometown of Cincinnati and tame TIGER deserts elsewhere in Ohio from the comfort of my (armless) chair. I always assumed that the middle of Silicon Valley would be full of tech enthusiasts who occupy their time by micromapping every last bench and bush. The map sure looked complete, with lots of highway=primarys and highway=secondarys, landuse areas covering every square inch, and plenty of rail and bike infrastructure.
But then, in April, I zoomed in. I had recently joined Mapbox to work on iOS map software, and the Show My Location function went right to my unmapped doorstep. Around me was an endless parade of outdated street configurations, missing landmarks, test edits, proposed BART stations tagged as the real thing, and GNIS-imported hospitals that had been closed for years. Most of the map hadn’t been touched in six years. In terms of POIs like shops and restaurants, central San José in 2015 was as blank as Cincinnati was in 2008. (San José is the country’s tenth-largest city, with a population 3½ times that of Cincinnati.)
Zoom in all the way to the spot marked San José, and this is what you would’ve found earlier this year.
As I added in pent-up local knowledge, I couldn’t help but notice some unfortunate tagging practices. The Bay Area is (ahem) liberal in its use of highway=primary and highway=secondary. It wasn’t difficult to find quiet residential roads with speed bumps, Child at Play signs, or unsignalized crosswalks being tagged as secondary, the same tag often used for heavily-used roads in other cities or 55 mph state highways in rural areas.
Most of the giant landuse areas that blanket the city need to be redrawn. Many landuse=residential areas conflate distinct neighborhoods or include tree-lined business districts (which look like residential areas from the air). Meanwhile, many industrial areas are being converted into residential areas due to a local housing boom. As much as possible, I’m replacing these generic landuse areas with more specific ones that correspond to individually named subdivisions, office parks, and retail complexes.
A typical landuse=residential area in San José spans multiple highway=primary roads. Either these aren’t really primary roads or this isn’t really one coherent residential area.
I suspect that the highway classifications and generic landuse areas, combined with decent rail data, made the map look a lot more complete than it really was. To a newcomer, the total absence of restaurants, buildings, and non-armchair-mappable information might’ve looked like a limitation of the project rather than a blank slate waiting to be edited. And again, there should be no shortage of visitors from San José, because this is Silicon Valley, where people talk about things like OSM. I’m sure the original mappers were doing their best at the time; unfortunately, six years ago, none of us knew as much about mapping ago as we do now.
San José is looking a lot better after an intense few months of surveying. There are plenty of POIs downtown – too many to fit onto the map at z19, in fact – as well as invisible attributes like speed, weight, height, and turn restrictions. I’m having particular fun mapping the many ethnic enclaves around town, which are very poorly represented on commercial map services.
This popular ethnic strip mall is now fully mapped in OSM (seen here in iD at z20). Apple and Google make a mess of things.
The San José place=city POI incorrectly sat 12 blocks away from where it should’ve been, at the site of this church, which incidentally is missing from Apple, Google, and HERE.
Meanwhile, OSM now includes that church, as well as its full Amharic Ethiopian name. (Deciphering the Amharic signage was a challenge in itself.)
Still, that’s only one city. We’ve always known TIGER deserts are a problem, but are other cities similarly languishing after an initial burst of detail, flying under the radar because we all think they’re being taken care of? Maybe we can prevent that from happening in the future by making the map look only as complete as it really is.
Comment from joost schouppe on 24 August 2015 at 13:25
Well, some friends here have been thinking about creating a map like the nice heatmaps you often see, but where the heat comes from avarage age of the nodes in the tile. And maybe also one where the ‘heat’ comes from number of different contirbutors. These maps might help to identify the kind of places you mention: a lot of data, but little movement.
Comment from smsm1 on 24 August 2015 at 13:32
You have to be careful with heat maps as it may encourage people to simply touch OSM elements and not properly map, even so there’s no real change needed. Instead, there needs to be some form of “I’ve checked this, and it’s up to date”, without causing a lot of churn in the data.
Comment from Omnific on 24 August 2015 at 14:21
Take a look at LA. Again, similar story, pretty much empty. This is the second largest metro area in the US, and also almost completely empty. This is especially sad since buildings and addresses could easily be imported based on LA’s current data license.
Comment from rayKiddy on 25 August 2015 at 20:13
I find Sunnyvale to be fairly completely mapped. But I may be missing problems with the data.
I often wish there was a way to look at a map and filter on the type of POI. Perhaps I want to look at all restaurants or all schools. There is no way to do this, yes? I could probably do it in Overpass, but my experience with relational db and tools does not make OP’s API intuitive.
In re the place tag, I just added “place=state” to the relation that is California. I am really not sure why that had not been done.
I am also interested in adding relations for geo entities that tile the map, such as school districts. Any suggestions or pointers would be appreciated. I found out how the County Registrar gets the boundary info on school districts. It is amusing. Going to diary it. Basically, the data is in TIGER but they do not use that. Instead, districts send them narratives like “this street, west side”, “that street, north side” so on. Very ouch!
Comment from Minh Nguyen on 26 August 2015 at 04:03
smsm1: I’ve been mapping in areas that have enough work left to do that even removing
tiger:reviewed=notags is too tedious. However, it does sound like really well-mapped communities will eventually want some way to periodically mark features as reverified without making perfunctory edits.rayKiddy: I was writing about San José proper. Some of the Peninsula cities are reasonably well-mapped, as you’ve observed. I wonder why such a gap in coverage developed between San José and its suburbs.
The issue of boundaries comes up on the talk-us mailing list fairly regularly. Some mappers have advocated mapping special-purpose districts (like school districts), while others have argued that boundaries generally should not be in OpenStreetMap. The disagreement arises because you can’t always spot a border on the ground to verify it, unlike much of the data in OSM. In the case of city limits, you can almost always find a marker along the road; by contrast, school district boundaries are rarely marked. (Fire districts sometimes are, depending on the jurisdiction.) Personally, I wouldn’t really mind seeing school districts mapped as
boundary=schoolrelations, but expect a lively discussion when you are ready to formally propose importing this data.Comment from Minh Nguyen on 26 August 2015 at 04:15
ITO Map has domain-specific maps for this kind of thing. If you’re interested in accessing the raw data, overpass turbo has a wizard that generates basic queries for you and allows you to visualize the results.
Comment from joost schouppe on 26 August 2015 at 06:08
@rayKiddy: a POI focused map that lets you filter just certain types of POI’s? Here are my favourites: http://www.openlinkmap.org/ http://openpoimap.org/ http://thenextis.com/
Also, http://www.Overpass-Turbo.eu . Zoom to your are of interest, hit the wizard, type “amenity=school”, hit run. Ready. When you do run into trouble, I have found that it is indeed hard if you don’t have a background. But the guys at http://help.openstreetmap.org are, well, very helpful.
Comment from n76 on 8 September 2015 at 21:56
Just noticed this from an email on the talk-us list. A couple of items:
Comment from rayKiddy on 25 August 2015 at 20:13 “I find Sunnyvale to be fairly completely mapped. But I may be missing problems with the data.”
Thank you! Most of the area from Central Expressway to the southern border of Sunnyvale and from Bernardo to a bit east of Sunnyvale-Saratoga I did by walking about. Many missing businesses as my focus was on getting house numbers, verifying speed limits, etc. matthieun has been covering much of the northern part of Sunnyvale and on over into some of Mountain View. I haven’t met him in person but we have exchanged messages and he seems to be doing a great job. Between the two of us, I think Sunnyvale is in much better shape than many other areas in Silicon Valley.
Back to the topic at hand: I think this area is/was a “Tiger Desert” but the fact was hidden by the imports like that of land use (I think mostly by SteveA). It seems like every street I looked at has needed at least a little tweaking. And when I visit some area I typically check OSM to see what the data looks like and find that updates are needed. Just yesterday I was in the west side of Santa Clara and all the roads in that area were listed as unreviewed, had some alignment issues and were devoid of such niceties as maxspeed tags.
Regarding highway tagging, my rule of thumb is to look for painted center lines. If it has one then it is at least a tertiary but if it is lacking then it is most likely a residential. I have no clue if that matches the federal highway feeder classifications or not, just what it looks like on the ground to me compared to what I read in the OSM Wiki.
Within a few months I will be moving out of the area and the city I will be moving to is also basically a Tiger desert. More fun and starting from scratch with plenty of more streets to walk gathering information!