Facebook is releasing a third update to Daylight, our complete, downloadable preview of OpenStreetMap data.
📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.3.osm.pbf (57GB).
Earlier this year, Facebook released Daylight as part of our work with OpenStreetMap. We use maps to let our users find friends, businesses, groups and much more. OpenStreetMap (OSM), the open source wiki map, has a substantial global footprint of map data built and maintained by a dedicated community of global mappers making OSM a natural choice for Facebook.
Every day, our mapping teams work to confirm OSM’s millions of community contributions for consistency and quality by excluding intentional and unintentional edits that are incompatible with our use cases. Daylight is the result of that process.
With this third release, we’ve continued our collaboration with the Microsoft Building Footprints project on an extension to Daylight. Users of Daylight can continue to apply missing building footprints in the United States, Canada, Uganda, and Tanzania to render more complete-looking maps:
Larry Freil at Microsoft has posted a detailed exploration of this data, including suggestions for how to apply buildings data to Daylight for rendering OpenStreetMap with Mapnik. Microsoft provides the building footprint data under the Open Data Commons Open Database License (ODbL).
Throughout the remainder of this year, we will continue to publish releases of Daylight and conflated building footprints files from Microsoft approximately each month. Additionally, we will release details of the OSM tags we consider for inclusion in Daylight after converting them to a format suitable for use with existing community data tools.
If you have any questions about this data distribution, we have created a #daylightdistro_feedback Slack channel in OSM US. Members of the team will be there periodically to answer questions. You can also email the team at firstname.lastname@example.org.
Learn more about the technology behind our process from our engineering team:
Download Daylight Map Distribution files in OSM-compatible formats:
If you’ve benefited from OpenStreetMap data in your work, consider joining the OSM Foundation as a voting member to help steer the future of this global open data project! Time is running out to join 90+ days before the expected OSMF December Annual General Meeting as an eligible voting member.
Comment from mmd on 12 September 2020 at 09:27
Positive ids in ms-ml-buildings-v0.3-positive.osc.bz2 appear to be way too large and would cause a segmentation fault or an assert error depending on which osm2pgsql version you’re using (https://github.com/openstreetmap/osm2pgsql/issues/965). Renumbering via osmium before running osm2pgsql is definitely required here.
Also, don’t use the corresponding osc file with negative ids. osm2pgsql will no longer support negative ids in the future and your workflow will simply break (https://github.com/openstreetmap/osm2pgsql/issues/1097).
Maybe you could take that to your Slack feedback channel as well for those folks not following your blog post.
Comment from migurski on 12 September 2020 at 15:50
Thanks mmd! This explains an issue that Ian was reporting. Seems like negative IDs are too small and big IDs are too big. We’ll dig in a little, perhaps maintainers of some of these tools would be receptive to uint64’s.
Comment from mmd on 12 September 2020 at 16:14
The positive numbers would still fit perfectly well into uint64, so that wouldn’t be much of an issue here. osm2pgsql uses an internal node cache that derives some block numbers from object ids. If those ids happen to be too large as in this case, block numbers exceed internal limits and eventually the whole processing aborts. Renumbering makes sure there’s no conflicting ids while at the same time keeping object ids in a reasonable value range.