OpenStreetMap

Just Released: Third Update to Daylight Map Distribution

Posted by migurski on 4 September 2020 in English (English).

Facebook is releasing a third update to Daylight, our complete, downloadable preview of OpenStreetMap data.

📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.3.osm.pbf (57GB).

Earlier this year, Facebook released Daylight as part of our work with OpenStreetMap. We use maps to let our users find friends, businesses, groups and much more. OpenStreetMap (OSM), the open source wiki map, has a substantial global footprint of map data built and maintained by a dedicated community of global mappers making OSM a natural choice for Facebook.

Every day, our mapping teams work to confirm OSM’s millions of community contributions for consistency and quality by excluding intentional and unintentional edits that are incompatible with our use cases. Daylight is the result of that process.

Daylight Map Distribution

With this third release, we’ve continued our collaboration with the Microsoft Building Footprints project on an extension to Daylight. Users of Daylight can continue to apply missing building footprints in the United States, Canada, Uganda, and Tanzania to render more complete-looking maps:

Visual comparison between Daylight v0.3 and Daylight with Buildings

Larry Freil at Microsoft has posted a detailed exploration of this data, including suggestions for how to apply buildings data to Daylight for rendering OpenStreetMap with Mapnik. Microsoft provides the building footprint data under the Open Data Commons Open Database License (ODbL).

Throughout the remainder of this year, we will continue to publish releases of Daylight and conflated building footprints files from Microsoft approximately each month. Additionally, we will release details of the OSM tags we consider for inclusion in Daylight after converting them to a format suitable for use with existing community data tools.

  • Daylight planet files are composed of 100% OSM data, released under the terms of the Open Database License
  • We’ve checked public OSM changes contained in the distribution, and allowed only those which have been evaluated through validation algorithms and manual processes against commonly identified use-cases to provide the best mapping experience to our display maps
  • We publish the data in PBF and OSC formats, common file formats universally supported by OSM tools and already used for planet-scale file distribution

How To Reach The Team

If you have any questions about this data distribution, we have created a #daylightdistro_feedback Slack channel in OSM US. Members of the team will be there periodically to answer questions. You can also email the team at osm@fb.com.

Learn more about the technology behind our process from our engineering team:

Download Daylight Map Distribution files in OSM-compatible formats:

If you’ve benefited from OpenStreetMap data in your work, consider joining the OSM Foundation as a voting member to help steer the future of this global open data project! Time is running out to join 90+ days before the expected OSMF December Annual General Meeting as an eligible voting member.

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Comment from mmd on 12 September 2020 at 09:27

Positive ids in ms-ml-buildings-v0.3-positive.osc.bz2 appear to be way too large and would cause a segmentation fault or an assert error depending on which osm2pgsql version you’re using (https://github.com/openstreetmap/osm2pgsql/issues/965). Renumbering via osmium before running osm2pgsql is definitely required here.

Also, don’t use the corresponding osc file with negative ids. osm2pgsql will no longer support negative ids in the future and your workflow will simply break (https://github.com/openstreetmap/osm2pgsql/issues/1097).

Maybe you could take that to your Slack feedback channel as well for those folks not following your blog post.

Comment from migurski on 12 September 2020 at 15:50

Thanks mmd! This explains an issue that Ian was reporting. Seems like negative IDs are too small and big IDs are too big. We’ll dig in a little, perhaps maintainers of some of these tools would be receptive to uint64’s.

Comment from mmd on 12 September 2020 at 16:14

The positive numbers would still fit perfectly well into uint64, so that wouldn’t be much of an issue here. osm2pgsql uses an internal node cache that derives some block numbers from object ids. If those ids happen to be too large as in this case, block numbers exceed internal limits and eventually the whole processing aborts. Renumbering makes sure there’s no conflicting ids while at the same time keeping object ids in a reasonable value range.


Login to leave a comment