Recent diary entries
We've completed work on the San Francisco building footprint dataset. We added or modified over 150,000 buildings in about 5 months of tracing with a team of three. My colleague Ruben just posted stats on the Mapbox blog. Here's an animation of all changes.
We're updating attribution for OpenStreetMap-based Mapbox maps thanks to feedback on attribution conventions here on the diary and on mailing lists. The new convention on Mapbox maps is to expand attribution by default: collapsed attribution should only be used when attribution becomes unusually long, or screen space is limited. Expect us to roll out these changes over the next couple of weeks, but here is a preview right away.
The entire goal of the Mapbox team's work with OpenStreetMap is to help make OpenStreetMap the best map, everywhere in the world. We will only be able to achieve this as a community and with open data. Linking maps back to OpenStreetMap is at the heart of growing OpenStreetMap by helping turn map consumers into map contributors. Our goal with these new attribution conventions is only to further improve the connection of the many million users who view Mapbox maps every day to OpenStreetMap.
Here are the new attribution recommendations for all Mapbox maps that are based on OpenStreetMap data.
While collapsed attribution wrapped in an info - ⓘ - symbol, works well on small screens, we are now recommending to expand attribution whereever possible. The full attribution line is "© Mapbox © OpenStreetMap" and next to it we recommend an "Improve this map" link leading a user to editing on OpenStreetMap. Another change is that now "© OpenStreetMap" links directly to http://www.openstreetmap.org/copyright, "© Mapbox" continues to link to http://mapbox.com/about/maps listing the full roster of map data we're using including OpenStreetMap.
Recommended attribution on Mapbox maps. Click to explore.
Collapsed for small maps
We're recommending this form of attribution for small slippy maps. Here's an example:
Recommended attribution on small slippy Mapbox maps. Click to explore.
Use these attributions now
Until these attribution recommendations are rolled out on Mapbox.com, here are links to code snippets that already work today:
Updated attribution recommendations for Mapbox maps: http://www.openstreetmap.org/user/lxbarth/diary/21847
Showing how OpenStreetMap is a living map, and making it easy to start mapping is the first step to turn someone from passively looking at a map into improving the map. It's part of spreading the word and building our community. At Mapbox we power OpenStreetMap based maps to hundreds of millions of people, and this gives us a unique opportunity to connect them to OpenStreetMap and turn people from being passive map consumers into active map contributors. Driving contributors to OpenStreetMap is a key goal we pursue not only with attribution but also in our aggressive launch communications around prominent new customers.
Our goal is to feature OpenStreetMap to help grow the community - attribution plays a key role in this.
Attributing OpenStreetMap based Mapbox maps
For the web, at Mapbox we recommend the following two variations for attributing OpenStreetMap:
Attribution in collapsible info control
Same attribution as above but expanded
In both cases
(c) Mapbox (c) OpenStreetMap links to https://www.mapbox.com/about/maps with a full listing of all sources.
Improve this map links to a map feedback page that explains how the map viewed is based on OpenStreetMap and how OpenStreetMap can be improved by anybody. The map feedback page is smart and shows a) the exact map you came from and b) places you into OpenStreetMap exactly where you left the map so you know where to start mapping. It has an option to skip the map-feedback page the next time you click
Improve this map and take you directly to OpenStreetMap.
Map feedback page
Maps made of many sources
Mapbox maps are made up from a multitude of sources, here are some of our main sources:
- Digital Globe
- NASA MODIS, Landsat, SRTM
- USDA NAIP
- l'Institut national de l'information géographique et forestière
- Canadian government
- The National Land Survey of Finland Topographic Database
- Norwegian Mapping Authority
- Ordnance Survey data
- DHM / Terrain
- The National Dynamic Land Cover Dataset
- Custom data added to map
This list is only growing as the source composition of our maps gets more complex. So the string
(c) Mapbox (c) OpenStreetMap is crediting the map engine and design (Mapbox) and one of the most prominent data provider (OpenStreetMap) but it is also functioning as a placeholder that basically says "Attribution". This is why we link this string to https://www.mapbox.com/about/maps that contains the full list of all data. For related reasons, I also typically recommend using the collapsible info control over the expanded string on the map as it allows us in the future to add additional attributions into the map as needed without turning people's maps into NASCARs. This is a good compromise between visibility, legal requirements and the need for screen space to grow.
Mapbox can be used with any kind of map library. So, ultimately we do not have control over a given maps attribution, but if you use Mapbox with our recommended libraries, attribution will show up as explained above, otherwise it is up to the developer to ensure appropriate attribution.
We're working to make this even better, and are planning to improve:
- More granular attribution based on data actually in use on data (right now it's one size fits all and we show this attribution as soon as you use any of streets/satellite/terrain data). A lot of Mapbox maps do not use OpenStreetMap but still want to associate proper attribution.
- Allow third party users to sign into OpenStreetMap with the account they're using on the map (think of signing into OpenStreetMap with your Foursquare account). We need to make it easier to let communities that start using OpenStreetMap become part of our community. This will have huge network effects. This will also take some work on the OSM.org side.
- Map feedback also has an option to submit feedback as email, and have our team run point on edits, fully respecting privacy.
- Share map feedback where it makes sense.
Here are two typical Mapbox powered maps with attribution (click to explore).
Mapbox Outdoors: OpenStreetMap, Ordnance Survey data, l'Institut national de l'information géographique et forestière, NASA SRTM, The National Dynamic Land Cover Dataset plus more.
InfoAmazonia maps: OpenStreetMap, NASA Modis, Landsat, Digital Globe, IBGE, InfoAmazonia.
This weekend, the quarterly US #editathon takes place in 10 US cities - read all about it on the OpenStreetMap US blog.
The #editathons are not just a great excuse to meet up with other OpenStreetMappers to push on projects, but also an opportunity to learn more about OpenStreetMap. In DC we'll be hosting the #editathon in the Mapbox garage. It's going to be great weather so expect some people to go outside and survey too. Read up on the Mapbox blog on how to find the Mapbox garage. Here's a photo from last year's event there:
Hal Hudson from New Scientist wrote a great article on how OpenStreetMap helps Médicins Sans Frontières (MSF) fight Ebola in Guinea:
WHEN doctors working for Médecins Sans Frontières (MSF) arrived in the West African nation of Guinea last month to combat an outbreak of the deadly Ebola haemorrhagic fever, they found themselves working in an information vacuum.
MSF enlisted the help of the Humanitarian OpenStreetMap team (HOT) and within a few days, a huge number of mappers flocked to OpenStreetMap, putting the affected areas on the map. Where existing Bing imagery was not sufficient, Astrium and DigitalGlobe provided fresh takes.
Even if this crisis is not in all the medias, the contribution from the OSM contributors is fantastic. In 8.5 days, 302 contributors, 1.2 million objects, 114,000 buildings, 5,000 places and 6,100 landuse polygons.
The New Scientist article explains how OpenStreetMap helps fight the virus:
Mathieu Soupart, who leads technical support for MSF operations, says his organisation started using the maps right away to pinpoint where infected people were coming from and work out how the virus, which had killed 95 people in Guinea when New Scientist went to press, is spreading. "Having very detailed maps with most of the buildings is very important, especially when working door to door, house by house," he says. The maps also let MSF chase down rumours of infection in surrounding hamlets, allowing them to find their way through unfamiliar terrain.
Since the response to the Haiti earthquake we are now seeing time and again how OpenStreetMap is facilitating incredibly mapping of badly needed geo data, helping first line emergency responders do their work.
You can't do this with any other map but OpenStreetMap.
This type of massive mapping effort is only possible because of OpenStreetMap allowing direct editing of data to anyone and the availability of OpenStreetMap as raw and open data. The former allows anyone to get involved in helping respond to a crisis, the latter gives full power to responding parties over how exactly maps should look like or access to raw data for analysis. No other map offers this level of openness at a global scale.
Cross posted to talk list
Effective immediately the Mapbox Satellite option in iD and JOSM is 100% open for tracing in OpenStreetMap, including all our high resolution DigitalGlobe imagery. This is full coverage down to zoom level 19 imagery in the US + Western Europe and world wide to zoom level 17.
To use this imagery select "Mapbox Satellite" from the imagery menu in iD on the web or in JOSM. Mapbox Satellite is open for tracing in OpenStreetMap in general and not tied to a specific editor, so if you would like to add Mapbox Satellite to another OpenStreetMap editor you are welcome to do so.
This is a big affirmation of DigitalGlobe's commitment to provide imagery for OpenStreetMap (also Bing imagery contains to a very large degree DigitalGlobe material). Props to Kevin Bullock and our friends at DigitalGlobe - it's fantastic working with good people who see wins of working with OpenStreetMap.
Editing in Washington DC with the Mapbox Satellite layer
PS - on an existing installation of JOSM you'll have to refresh your imagery menu like so: http://cl.ly/image/383O2L0t431s
OpenStreetMap is published under a share-alike license, the so called Open Database License (ODbL). The license says that if raw OpenStreetMap data is mingled with raw third party data, and the result is used publicly, you are required to release the result under the same ODbL. This is, in short, the share-alike principle under which OpenStreetMap data is available today - under certain circumstances, it extends the license of OpenStreetMap data to data sets it's mixed into.
Sounds like a great idea at first, right? You're promoting the idea of opening data by making sure anyone who uses your data opens their data too. Well, there's a big gotcha: we wind up more often with OpenStreetMap not being used rather than with previously closed data opened up. This in turn hurts the project which thrives on increased adoption.
Photo: Alan Levine
Organizations or individuals who want to mix OpenStreetMap data with third party data often can't because they aren't in a position to make licensing decisions on that third party data. The reality is that opening data under a specific license is usually too slow or plain not possible.
Often times confusion about what's allowed and what is not allowed under the ODbL is just as bad. Ever seen advice opening with "I'm not a lawyer, but..."? That's what I'm talking about. Ever tried to get an actual lawyer to provide guidance on the ODbL? That's what I'm talking about. Tried to use the OpenStreetMap Wiki to learn about how the ODbL is interpreted by the licensor, the OpenStreetMap Foundation? That's what I'm talking about.
The result is that OpenStreetMap is not being used in situations where it should be used, which undermines a project whose success depends on increased adoption.
Not only is OpenStreetMap not being used as much as it could, the assumption that share-alike encourages contribution is a myth. I have yet to meet the individual, company, non profit or government agency who contributes because that's what the license calls for. And I have yet to witness the troves of data opened under the ODbL in compliance with the license. OpenStreetMap gains no extra benefit from share-alike. The reality is that OpenStreetMap is only used extensively in situations where the share-alike license does not apply, for instance, map rendering.
Here are examples of what should be possible with OpenStreetMap but is not because of share alike:
The Wheelmap community manages wheelchair accessibility information for over 400,000 thousand places in OpenStreetMap. Ideally Wheelmap would be able to syndicate this data into any other map - think Nokia, Google, Apple. Today they can't because of share-alike limitations of the ODbL. Woulnd't people using this data on Google maps mean more people with an interest to maintain and improve it on OpenStreetMap since they would know that adding data to OpenStreetMap means adding it to all the maps in the world?
Currently, New York City building and address data is being imported into OpenStreetMap (disclaimer: I'm involved). Ideally the government of New York City would just copy changes from OpenStreetMap to help maintain their own datasets - but they can't. Many datasets managed by government behind closed doors today should just be managed by the same maintainers on OpenStreetMap tomorrow - with gains for everyone. Think of the US Census Bureau whose TIGER data we're all benefiting from. This vision of citizens and government collaborating around OpenStreetMap is severely cut short by the ODbL. Governments will never use OpenStreetMap in an extensive way until they can make it part of their workflow, and as long as the ODbL taints any data that touches it, it can't. Look at the United States - many government datasets are public domain, government can't use OpenStreetMap directly because the ODbL is not compatible with it.
And what about exchanging data with our big sister project Wikipedia? We should be copying a lot more data back and forth between OpenStreetMap and Wikipedia. OpenStreetMap could be Wikipedia's geocoder and gazetteer. And yes, if it wasn't for Wikipedia's own share-alike license, we could mine Wikipedia for addresses, phone numbers, home pages, and populations without a bad conscience. Wikipedia can't use OpenStreetMap because OpenStreetMap is not truly open, and OpenStreetMap can't use Wikipedia becuase it is not truly open. What better examples of two sucessful open data projects are there than Wikipedia and OpenStreetMap - but we are not open enough for our data to touch? This makes no sense.
If we dropped share-alike, nothing would stop players like Google or Apple from mixing OpenStreetMap data extensively into their mobile maps. And this is a good thing. OpenStreetMap's opportunity is not to compete and win against the Google Maps of the world, but to say what's on their maps. With adoption on established mapping platforms OpenStreetMap would instantly reach many millions of users with its data, drastically increasing the project's impact and playing a bigger role than stale backfill. OpenStreetMap's current licensing is stunting our growth - and diminishing the impact of all of the amazing data that we have.
Under the current license, these example cases are either outright impossible, or require time, good lawyers and programmers to avoid share-alike to infect third party data with the ODbL. The ODbL imposes unnecessarily onerous hurdles at no gain for the project. Worst of all, just the license's ambiguities kill adoption.
If OpenStreetMap is to turn into the data set that makes geo data a true public good we have to drop share-alike. Let's make OpenStreetMap data actually open.
OpenStreetMap is at the verge of being the dataset that powers the world, quite literally. What's between where we are today and making OpenStreetMap the source for global geographic data, is that OpenStreetMap simply can't be used in many applications where it would be the ideal solution. These lost opportunities matter because they are what keeps OpenStreetMap from having the impact it should have. As Serge Wroclawski succinctly argued in his essay on why the world needs OpenStreetMap, OpenStreetMap's purpose is to democratize who decides what's on the map:
Every time I tell someone about OpenStreetMap, they inevitably ask "Why not use Google Maps?" From a practical standpoint, it's a reasonable question, but ultimately this is not just a matter of practicality, but of what kind of society we want to live in.
OpenStreetMap simply won't matter if it doesn't power the applications that millions of individuals use to search, navigate and contextualize each day. The more OpenStreetMap is used, the more impactful each of our work is, and the more incentives we create to join the movement. We should not be afraid of that.
For your reading pleasure: Here's are the entire 4,000 words of a license we should be throwing out: ODbL 1.0. I will be speaking about this topic at the State of the Map US conference in Washington DC. Join the conversation here or on Twitter.
Share what you've been working on, or present your vision for OpenStreetMap at this year's State of the Map US in Washington DC April 12 - 13.
You have until February 2nd (this Sunday) to submit your session.
You'll find the submission form here: http://stateofthemap.us/
Looking forward to hearing from you.
Presentations at State of the Map US 2013. Photo: Justin Miller.
This is an update on the ongoing import of New York City buildings and addresses. For background read up on New York City and OpenStreetMap cooperating through Open Data
We have taken the time to take a close review of existing uploads. Here are some issues we've found that are worth highlighting as we restart the import.
- Make sure every upload to OpenStreetMap completely validates and all critical warnings are resolved before you update.
- Critical warnings are at least any warnings or errors that stem from
- Buildings overlapping with buildings
- Buildings overlapping with other features they cannot overlap with such as roads
- Resolve not only buildings duplicate with existing buildings but also addresses duplicate with existing addresses
- Merge point of interest information from existing nodes to new buildings when they clearly building-level such as schools, fire houses, super markets, etc.
To get started head over to the tasking manager carefully (re) read instructions and grab a task.
Make your life easier and get these JOSM styles for buildings and addresses by emacsen. They'll allow you to see issues with the data better. Learn how to install them in JOSM docs.
If you have any questions, fire away here on the comment thread.
On the imports list I recently raised the question on whether to tag addresses on buildings ways or not. Specifically, if there is only one address for a given building polygon, should the address tags sit on the building's ways or should the address tags sit on a separate node within the building? Obviously, if there is more than one address per building, there is no other way but mapping them as nodes separate from the building way.
Eric Fischer just ran an analysis to figure out what is actually the current convention in OpenStreetMap. Here's the short answer: addresses are tagged on building ways where possible. By a wide margin.
Read on for the numbers.
Address tagged on building ways (left) is the more common approach in OpenStreetMap versus address tagged on a separate node (right).
The rough numbers break down like this:
- 10 million buildings carry addresses on the way.
- 3 million buildings contain one or more address nodes.
- 4 million address nodes sit within a building.
So the maximum theoretical number of buildings with a single address node is 3 million minus 1. Contrast this with 10 million buildings with the address information on the way. This still assumes one crazy building containing one million address nodes and it does not discount redundantly tagged addresses in the case of POI nodes that duplicate the address of the building they sit in.
Here are the full numbers (OSM planet, September 25 2013):
Buildings 91,917,857 Buildings with address on way 9,386,811 Buildings that contain one or more address nodes 2,960,363 Address nodes within a building 3,858,096 Address nodes that are not on or within a building 10,135,036 Addresses on a node of the building way 673,975
(An address node is defined here as a node that contains an
addr:housenumber tag and is not part of a building way.)
Good or bad?
For now, I'll personally stick to this convention as it's established. For the same reason I also want to stick to it for the ongoing New York City building and address import.
In principle though, I question tagging addresses on building polygons. It's a special case with no benefits while separate address nodes would work in both, the case where there are multiple addresses per building polygon and where there is only one address per building polygon.
Last Saturday we officially kicked off the NYC building and address import with a community session hosted by OSM-NYC and Public Labs at the Pfizer building in Brooklyn. The goal was to get the local NYC OSM community involved in this large data undertaking and at the same time harden our import process.
Over 20 people attended, and we knocked out 158 of the over 5000+ sub-tasks total. Both turn out and tasks accomplished were great and exceeded what I expected for a casual Saturday afternoon event.
Working through this import we're learning very interesting lessons:
- OSM data structure is significantly different from traditional GIS, nailing down conceptual differences when translating to OSM takes time.
- Importing is a high inertia problem, partly due to sheer volume but also due to the lack of a solid tool chain like safe roll back tools or established conversion tools.
- Expect interesting quality issues in your source data. NYC data for instance has inconsistent address formatting in the source.
- Doing a fully automated import is non-trivial. For example, in NYC, buildings often intersect with misaligned TIGER roads. That's one big reason this import is not fully automated.
- Once all data is uploaded, we'll need a QA check on inconsistent data to catch any errors introduced by humans during the upload.
- This all feels a little like heart surgery.
Here are a couple of pictures and screenshots from the Saturday event. If you'd like to get involved drop me a line. Again, the import is on hold until a couple of issues are sorted out, but you're welcome to join.
Because I love it.
Open data is changing the world. OpenStreetMap, as a true open data project of the commons, is proving its viability with continuously growing contributor numbers and expanding adoption. We're well under way to replace what has been historically the realm of governments and proprietary-data companies with amazing open data that is not better because it's cheaper, but actually provides fresher and more detailed data because it's open and community driven. It's exciting to be a part of this in my job as data lead at MapBox, as an individual contributor and as a board member of the OpenStreetMap US chapter.
This last year on the board of OpenStreetMap US has been amazing. Working together with Jim, John, Randy and Martijn on the board and with the great support of community members like Kathleen Danielson, Bonnie Bogle and Ian Dees, we've brought the organization to a new level. We're honing in on our goal to not only promote OpenStreetMap in the United States, but to make it bigger, stronger and more diverse. Here are some of the things we've accomplished:
- Ran the biggest OpenStreetMap conference to date, bringing together almost 400 members of the US and international community in San Francisco.
- Ran quarterly editathons promoting OpenStreetMap on a local level. Twelve cities are participating in the October editathon, and more than 1,000 edits came out of the last one.
- Relaunched OpenStreetmap US to do a better job introducing OpenStreetMap to the world.
- Connected with government agencies to promote the use of OpenStreetMap and explore areas of cooperation.
For the next year I want to stay 100% focused on continuing to grow OpenStreetMap in the US and beyond. I am convinced we can do this only by uniting the many voices of our project and by being as open and inviting as possible to newcomers. This is why it will be so important to nail our conference again. The importance of State of the Map US for the growth of OpenStreetMap cannot be overstated. It is the main tool we have to convene our community, pull in new individuals and institutions and discuss the future of the project. OpenStreetMap brings together interests from very different backgrounds: it's being used and improved by individual mappers, businesses, governments and nonprofits. Individuals work with it as data consumers, data producers, software developers, designers and researchers. This diversity is exactly our strength and is exactly what we need to continue to grow OpenStreetMap.
I would love your vote for pushing on this vision on the board of OpenStreetMap US.
OpenStreetMap US elections are running from October 5th to October 12th, to vote, you simply need to be a member of OpenStreetMap US. Signing up only takes a minute, find out all details about the election over on the OpenStreetMap.us blog.
OpenStreetMap colored by contributor id
There has been lots of talk about groups on OpenStreetMap.org lately. In early 2013 Mikel called for better social tools, including groups on OpenStreetMap, and lately more often groups have been mentioned as a replacement for our ailing mailing lists. Saman had a version of groups in his blue sky mockups for OpenStreetMap.org. Tom's posted a sketch for groups as pull request.
I'd like to add a dose of skepticism in this discussion: I don't think we should implement groups on OpenStreetMap.org right now, there are better alternatives to get started with if our goal is to make OpenStreetMap more social and let mappers connect better.
- Most conversations ideally don't require groups.
- It's hard to do social software right, groups in particular.
- Social media platforms are distributing.
(1) Most conversations ideally don't require groups
When you stop to think about it, groups are a crutch. They require you to set up a space with a topic and name (even if it's just a couple of clicks), then people need to find it, subscribe to it and sustain interest in the group. If the group doesn't go well, it bleeds members and lives on as a distracting zombie. Ideally, you'd be able to have conversaions ad-hoc around a certain topic or locality. That's one reason why you don't find groups at all or in a dominant role on some of the most successful social networks today.
(2) It's hard to do social software right, groups in particular
What was the last forum or groups software you used that didn't suck? Right. It's hard to do groups right on an interaction design level. I personally haven't seen general group discussion software ever done right, but what I do know is this: whatever we embark on means significant investment - or falling short on expectations. The risk to wind up with another level of noise in our already brittle social space is real.
(3) Social media platforms are distributing
Today OpenStreetMap enthusiasts gather in spaces on mailing lists, Meetup.com, Twitter, Facebook, forums, and Google Groups. Whatever we build competes in this space. Right now, we shouldn't attempt to build the better replacement for all of this, but think of OpenStreetMap.org as a compatible layer, allowing mappers to bring OpenStreetMap into their respective social online environments with ease.
Instead of introducing groups as a large new feature on OpenStreetMap.org I suggest we fix current social functionality on OpenStreetMap.org. This would vastly improve how mappers connect on a local and global level and would allow us to take an iterative approach, giving us real returns at each step, building on firm, well known ground. Here's a first back log:
- Great opt-out email notifications for edits, diary posts, comments of who you're following and posts you've commented on.
- Make it much easier to see who's mapping in an area
- Introduce public wall-style messaging, allowing conversations in the open.
- Ideally shut down private messaging to avoid abuse (which is happening according to administrators).
- This is small: Rename 'friend' to 'follow' - because that's what it is, no one confirms a friend request on OpenStreetMap.
- Kill the home location feature including the map on the profile
- Replace the useless friend listing and 'in your area' listing on your profile with a list of latest edits by who you're following
- Encourage users to link to local groups from their profile (Facebook links, meetup.com links, mailing lists links, wiki links, etc.)
- Possibly: vote up (down?) comments on diary.
Each of the above steps is small compared to implementing groups - still, each one will require dedicated work. Together they are designed to move us forward in a solid fashion from where we are right now. And note: some of the features like notifications could come in handy if we still wanted to introduce groups at some later point.
So what about our mailing lists?
Done right, the above improvements will already take important weight off of our mailing lists, we should iterate from there. With improved notifications and commenting on diaries we'll have much better spaces for meaningful discussions. I assume that much of our outcome oriented work will continue to move to GitHub. It's also going to be interesting to watch how Map Club will move in this realm. In addition, I suggest evaluating Discourse.org, a promising new discussion forum by Jeff Atwood the maker of Stack exchange.
What do you think?
Disclaimer: I am offering this simply as food of thought for those who're interested in pushing on social features right now. From a MapBox team perspective, we're not queueing up any immediate work on the social features mentioned here.
Finding out fast who's modified the map is hugely valuable to review changes in areas you care about, to connect to new mappers or to just show how fresh the map is.
Unfortunately, the part of OpenStreetMap.org that's supposed to provide this functionality - the history tab - is functionally broken. I'd like to suggest here a straightforward way to fix it, punting on some of the hard engineering problems that fast browsing of historic changes bring with them.
To recap if you're new to the issue, here's why the history tab doesn't work today: virtually anywhere in the world you'd like to see the latest changes of a particular area on OpenStreetMap, what you'll actually get is large-scale changes whose bounding box happens to intersect with with the area on the map you're viewing while not actually impacting any data in the area you're viewing.
The underlying engineering problem is Hard: changes to OpenStreetMap are organized in changesets, each one of which can contain up to 50,000 edits and whose modifications can be geographically huge. Querying all changesets that actually modified data in an arbitrary bounding box of the world and displaying them in reverse chronological order is computationally expensive while at the same time it should happen in milliseconds to satisfy a web request and allow for fast browsing.
Fixing the history tab
At the Chicago hack weekend Tom and I created a prototype that completely punts on the expensive problem of fast browsing for the entire changeset history. The approach we've taken is essentially to present you with a map and a list of the latest changes on visible elements first, then only reveal the history of an element when you click on it.
Conceptually, this is very straightforward and supported by existing APIs, computationally this is dirt cheap. This is not actually a novel approach in OpenStreetMap (most editors do something like that) but it is viable as an alternative to today's history tab.
The prototype doesn't do any data processing itself and is actually just a simple HTTP and JS application hosted on Github pages. It uses the OpenStreetMap API's map call. The latter means it is querying OpenStreetMap in an unefficient way, but this is a comparatively simple problem to fix. It could just as well query a very simple tiled data source.
Check out the result for yourself. I think it is already a very useful browser for exploring changes on OpenStreetMap. With few iterations this could be much faster and a viable fix to OpenStreetMap's history tab.
- Prototype: http://osmlab.github.io/latest-changes
- Codebase: https://github.com/osmlab/latest-changes
Related conversation on [dev] list can be found here: http://lists.openstreetmap.org/pipermail/dev/2013-May/026891.html
We just relaunched LearnOSM - the step by step resource for learning OpenStreetMap. LearnOSM was launched in 2011 by the Humanitarian OpenStreetMap team for the workshops they are giving world wide. It has grown into a resource used by OpenStreetMap newcomers and trainers well beyond its roots in humanitarian aid and disaster risk management.
Read up on the new LearnOSM and learn how to contribute to translations and other improvements by Jue Yang, the designer and developer behind the new face of LearnOSM:
Andy Allan recently ported the OpenStreetMap standard style from pure Mapnik XML to CartoCSS and TileMill. This is exciting as it's a huge step towards making contributing to the style more accessible. The port is nearly perfect and kinks are being worked out right now. I took a minute to spot check and pull together a couple of screenshots showing just how close this awesome piece of work is. I hope to see this go up soon on OpenStreetMap.org. Andy's port wouldn't change anything about how tiles are being rendered on OpenStreetMap, all that changes is how the style for those styles would be created: In the future, we'd use TileMill and generate the Mapnik XML from user friendly Carto CSS.
Here is San Francisco in the new OSM-carto port, just looks like the existing map:
For spot checking I use the comparison app that Ian and Tom cobbled together using OpenStreetMap US servers and bl.ocks.org. It shows the existing OSM Standard style to the left (I'll just call this style 'OSM' for the remaining post) and the new port to Carto CSS on the right (I'll call it 'OSM-carto' for the remaining post). Note: right now it's slow / down as performance problems are being figured out.
What follows here is a quick log from my review, others have been busy spot checking, too, head over to the issue queue to find out more.
School labels are different
School labels aren't bold in the new OSM-carto style. This has been fixed in the meantime.
Complex junctions seem to be rendering great with roads correctly rendered on top of each other and labels placed well.
OSM-carto has halo on secondary
OSM has no halo on secondary, while OSM-carto does have one in orange (see Castro street label) #25.
OSM-carto has no halo on tertiary
To the contrary, OSM has a white halo on tertiary highways where OSM-carto has none #24.
Label placement seems to be slightly different
This might be due to slightly stale data and generally doesn't seem to make a difference in terms of cartographic quality. In this difference rendition between the OSM and the OSM-carto style you can see how all labels seem to be offset by a certain factor and some street labels are placed at different positions along the way.
Low zoom levels
Low zoom levels look almost perfect at quick inspection. I do not know where differences in labels (see Sapporo for instance) come from.
Mid zoom levels are almost 100% the same
Differences in landcover order
There are some known differences in the order of land cover. This is being worked out right now in #15. Here is an example where the difference in landcover order surfaces in the visual result.
Stoked about test driving the great work of Tom, Richard, John, Saman and others on the new iD editor. Here's a quick screenshot. You can get started yourself with iD fast, just clone the repo, point your browser to the index.html file in it and get started editing. Note that if you'd like to upload your changes to the test server, you'll need an account on it. Report any bugs you're finding on the issue queue.
We had good turnout for the OpenStreetMap and MapBox workshops in preparation for Desarrollando America Latina.
The videos of the webcasts are up now, all in Spanish:
We're on the bus now heading back from New York City to Washington DC after an eventful week (read about the $575k Knight awarded Development Seed for OpenStreetMap work on the MapBox blog).
Here are some pictures of the OpenStreetMap intro workshop Ian and I held yesterday at foursquare's headquarter's. Thanks to the NYC OSM community leads Liz, Serge and Eric for the initiative and promotion.
Thanks to David Blackman of foursquare for kindly offering foursquare's great digs in Manhattan for this workshop. Join the OSM NYC Meetup Group if you'd like to stay in the loop on all things mapping NYC.
A great crowd of about 30 people turned out for the workshop.
Kicking off with examples of the many ways of how OpenStreetMap data can be used.
We quickly went hands-on and started editing and improving data on OpenStreetMap using JOSM.
Ian showing how to build quick OpenStreetMap based maps using Migurski's shapefile extracts and TileMill
Liz Barry explains how DIY aerial imagery can be used in OpenStreetMap
And yeah, the view from foursquare's office is killer
We'll arrive on Wednesday afternoon and head out on Monday afternoon. Hotel TBD. On Saturday, I'll give a talk about how we're improving OpenStreetMap with Foursquare.
I'm looking forward to meeting / seeing many great people. Drop me a line if you'd like to get in touch ahead of time.