Tiles, FCGI ...and pies

Posted by Harry Wood on 2 February 2010 in English (English)

A last Wednesday's London pub meet-up we had PIES! The Daniel Gooch in Bayswater seems to be run by a Scottish chap with access to little Scottish pies. We brought along our own Scottish chap who verified the scottishness of the pies. Rather good, and a bargain at £4.50. Other good things about the pub included cashback at the till, cheap soft drinks, and an interesting ceiling. The pub might have lost marks for having a big loud football match on, but we managed to hide down a funny little hole away from the noise. Here's photos of us in our funny little hole:

The Daniel Gooch 2 IThe Daniel Gooch 3 The Daniel Gooch 4 The Daniel Gooch 1 flickr

Lots of interesting conversation:

We talked about the new FCGI C++ implementation of the OpenStreetMap "map" call (code) which had been deployed earlier that day. This boosts the speed of data downloads for the OpenStreetMap editors, although the speed-up is mainly noticeable when fetching very large amounts of map data. This might just encourage more people to abuse the feature (It's intended for editing software, but gets used for other things where planet extract downloads would be more appropriate) so we joked about adding a wait in there :-)

That lead into discussion about acceptable use of the tile server. Currently the serving of OpenStreetMap maps is very centralised, with massive bandwidth being used to blast out tile images from one server. The end-users are often viewing the map on the homepage, but often not. There's now a massive range of websites using embedded maps, and mobile apps displaying maps direct from our tile server. As this bandwidth usage grows, it could become an issue. Andy was making the point that the foundation should be thinking about offloading the strain of tile serving. We would do this by tightening up the Tile Usage Policy. We could disallow any non-project related site to display tiles from the main server. This is a pretty upsetting scenario. In the short-term cutting off access to thousands of sites and apps using OpenStreetMap would cause massive disruption confusion and widespread severe annoyance, after that it would also mean less people would turn to using OpenStreetMap because we would no longer be doing the google thing of providing ultra-easy embeddable maps. But the point is that people would still take planet downloads and run tile servers elsewhere, in fact this would immediately become much more prevalent. In this way the project would continue to serve maps to the world, and would organically become rather pleasantly de-centralised leaving the resources at the project core to be focussed on the important bit: the read/write wiki-style map data API. It's a little bit alarming to think about these scenarios and consequences, but it's a sort of hypothetical "what if" discussion at the moment. Tile serving bandwidth is not infinite though.

We talked about holding another hack weekend. Between us we could probably find a London venue, although Matt has been talking to Frederick about doing one in Germany too. Need to remember to talk about it this again at the next meet-up, and actually pick a date perhaps.

We talked about completion, and in particular the state of the U.K. It was decided that "we should just finish it". So there you have it folks! You heard it here first. We're just going to finish the U.K. now. ...actually no. I think you've probably heard that before somewhere :-) I'm not sure I understood the point of this conversation. There was a serious plan to do with B roads and minor roads, but I don't remember. Anyway the conversation was surely a symptom of mapping withdrawal. An early itch starting to appear as we ponder the summer ahead. (Update: Andy blogged Finishing the UK road network)

Excellent evening. I think The Daniel Gooch is one we should go back to. The next pub meet-up is already scheduled and looking like it's going to be a big one. Wednesday February 10 - Ye Olde Cheshire Cheese on Fleet Street. Come along! (Although we chat about these in-depth topics, we're always hoping to get more newbies joining us)

Location: Peel Precinct, London Borough of Brent, London, Greater London, England, NW6 5BP, United Kingdom

Comment from Ollie on 2 February 2010 at 10:16

Looking forward to it. (The pub & the summer!)

Comment from Tomash Pilshchik on 5 February 2010 at 15:34

What you say about the tile server is interesting. I agree that the work of rendering tiles should be offloaded. However, this must be done with care since the tiles are the chief advertisement for the project.

I don't think the Tile Usage Policy should be tightened up until their is a viable alternative. If there were a viable alternative, there would probably be no need to tighten it up since those who deploy maps would probably prefer to be able to adjust the style sheets.

Currently, far too much hardware and effort is required to set up a tile server capable of serving the entire planet to even a small number of users. The entire planet file must be downloaded. A few days are required to create a giant SQL database. Then the data gets out of date and one must do it again. Sure, there is a way to do incremental updates, but last time I looked the documentation described several different methods (some of which seemed to be obsolete) and did not seem to describe the procedure in detail. (I would love to hear that I am wrong about this.)

Of course, even with incremental updates, hundreds of tile servers pulling down gigabytes of data which they will never render does not sound like an efficient system. Wouldn't it be better to set up some public tile data servers like that used for Tiles@home? Then only the tile data servers would need to store the whole planet.

Comment from Harry Wood on 5 February 2010 at 16:01

Yes. At the moment there's only one renderer which makes makes maps which look nice, and Mapnik has a reputation as a mysterious C++ black box with hideous stylesheet language built on a massive "enterprise" grade database. Developers are making progress at improving the ease of installation though (just recently actually) You can actually run it as a quite a lightweight thing rendering a localised tileset reading direct from osm.xml and using split out css style or spreadsheet formatted stylesheets. As much as anything else we need some good friendly tutorials on doing these things.

Other technology stacks are fun of course. I can't really take tiles@home seriously, but I do agree that other rendering systems written in other languages / using other data access methods could at least foster a bit of competition. So far nobody's really given Mapnik any good competition though. Maybe Kosmos comes close to Mapnik's antialiased non-text-overlapping goodness. Nobody pays much attention to poor old Kosmos because it's M$.NET

Mapnik is built on top of PostGIS, which is a "proper" spatially indexed database. The tile data servers idea is kind of a half-assed naive attempt at doing something similar. ...but as I say, different technology stacks are fun. I did play with OJW's tile data server and phprender (before it broke) I haven't looked at "Roma" much but I imagine if somebody bridged the gap from roma to Mapnik, that could be an interesting combo.

Login to leave a comment