pnorman's Diary

OpenStreetMap Cartographic: A client-side rendered OpenStreetMap Carto

Posted by pnorman on 25 May 2020 in English.

Cross-posted from my blog

I’ve been working on a new project, OpenStreetMap Cartographic. This is a client-side rendering based on OpenStreetMap Carto. This is an ambitious project, as OpenStreetMap Carto is an extremely complex style which shows a large number of features. The technical choices I’m making are designed so the style is capable of handling the load of osm.org with minutely updates.

I’ve put up a world-wide demo at https://pnorman.dev.openstreetmap.org/cartographic/mapbox-gl.html, using data from 2020-03-16, and you can view the code at https://github.com/pnorman/openstreetmap-cartographic.

Preview image

Incomplete parts

Only zoom 0 to 8 has been implemented so far. I started at zoom 0 and am working my way down.

Admin boundaries are not implemented. OpenStreetMap Carto uses Mapnik-specific tricks to deduplicate the rendering of these. I know how I can do this, but it requires the changes I intend to make with the flex backend.

Landuse, vegetation, and other natural features are not rendered until zoom 7. This is the scale of OpenStreetMap Carto zoom 8, and these features first appear at zoom 5. There are numerous problems with unprocessed OpenStreetMap data at these scales. OpenStreetMap Carto gets a result that looks acceptable but is poor at conveying information by tweaking Mapnik image rasterizing options. I’m looking for better options here involving preprocessed data, but haven’t found any.

I’m still investigating how to best distribute sprites.

Technology

The technology choices are designed to be suitable for a replacement for tile.osm.org. This means minutely updates, high traffic, high reliability, and multiple servers. Tilekiln, the vector tile generator, supports all of these. It’s designed to better share the rendering results among multiple servers, a significant flaw with renderd + mod_tile and the standard filesystem storage. It uses PostGIS’ ST_AsMVT, which is very fast with PostGIS 3.0. On my home system generates z0-z8 in under 40 minutes.

Often forgotten is the development requirements. The style needs to support multiple developers working on similar areas, git merge conflicts while maintaining an easy development workflow. I’m still figuring this out. Mapbox GL styles are written in JSON and most of the tools overwrite any formatting. This means there’s no way to add comments to lines of codes. Comments are a requirement for a style like this, so I’m investigating minimal pre-processing options. The downside to this will make it harder to use with existing GUI editors like Fresco or Maputnik.

Cartography

The goal of this project isn’t to do big cartography changes yet, but client-side rendering opens up new tools. The biggest immediate change is zoom is continuous, no longer an integer or fixed value. This means parameters like sizes can smoothly change as you zoom in and out, specified by their start and end size instead of having to specify each zoom.

Want to help?

Have a look at https://github.com/pnorman/openstreetmap-cartographic and have a go at setting it up and generating your own map. If you have issues, open an issue or pull request. Or, because OpenStreetMap Cartographic uses Tilekiln have a look at its issue list.

Discussion

Comment from Sanderd17 on 25 May 2020 at 17:55

Regarding generating the comments in JSON, why not code it in JS and let it output JSON. Any JSON is a valid JS object (hence the name), so you can just copy-paste parts of what you have. And in a main JS script, you can join them up into a single object (with comments in the code about what a certain part does), and print that object.

The added advantage is that you can have variables and simple calculations.

If you include it in a Mapbox GL JS website, you don’t even have to print out the JSON to a separate file, but you can pass the JS object directly to their map constructor. So during development, it’s just as well refreshing a page.

You can put your style in a file like this: export default const style = { // Your current style with added comments, can be loaded directly into mapbox too }

And to generate the actual JSON using a simple script like this: ``` import style from ‘./style.js’ import fs from ‘fs’

fs.writeFileSync(‘style.json’, JSON.stringify(style, null, ‘\t’)) ```

Comment from Sanderd17 on 25 May 2020 at 17:56

Ugh, that code got unformatted as shit. But hopefully you can figure it out.

Comment from bryceco on 30 May 2020 at 06:43

If I understand correctly this is strictly a client-side endeavor, but in order to contribute it requires setting up a server with fairly hefty storage and performance, and also expertise with Tilekiln. Is there any way to make it easier to contribute to just the client side, allowing contributors to focus on the meat of the project?