Say hello to the giant Multipolygons

Posted by imagico on 18 December 2017 in English (English)

With release 4.6 OSM-Carto now much more strongly than before encourages you to map waterbodies and water covered areas of rivers (riverbank polygons) with multipolygons as large as possible. The established and documented practice of dividing riverbank polygons into small, easy-to-handle areas, maybe even exclusively with closed ways instead of more complex multipolygons as it is documented on the wiki, has now been declared undesirable by OSM-Carto.

someone stole parts of the Lena...

Some might remember the multipolygon fixing efforts from earlier this year, the numbers are raising again and will be on the same level as before the fixing effort in 1-2 years. It is also well known that large multipolygons break more often and more likely stay broken than smaller ones. Yet incentivising merging of small polygons into larger ones as done by OSM-Carto has no influence on that of course, because ... oh look, over there, an ape with three heads...

Is this still a map in sense of an attempt to visualize the actual geography?

And it is not that this problem is unexpected or no one has pointed it out before.

I could go on contemplating about the underlying causes of this kind of problem but i have already done so in the past.

Hint for the wannabe map painters among you: If you have just painstakingly mapped thousands of small lakes in some area and loathe they are invisible on the map at all but the highest zoom levels just merge them into a giant multipolygon as well. Multipolygons with thousands of outer rings - no problem. And if the combined area is large enough you can make them show up this way - arbitrary thresholds nonwithstanding.

The work of the mappers: The work of the mappers

What OSM-Carto shows of this: What OSM-Carto shows of this

Is this still a map in sense of an attempt to visualize the actual geography?

Comment from Alan Trick on 18 December 2017 at 19:00

Honestly I think falls into the same sort of category as tagging for the renderer. It's a little different becaues large polys are actually a PITA to edit sometimes, so maybe it's more accurate to call it "tagging for the editor" sometimes.

At any rate, whether a map wants to show small features at low zoom levels is entirely a question of map design. I don't see why all those lakes should necessarily be mapped into a large multipolygon. With rivers, it makes sense to have one large multipolygo for the whole thing. But for a collection of small lakes, I don't get it.

Comment from imagico on 18 December 2017 at 19:19

Mapping for the renderer means mapping against established conventions to achieve a certain rendering result. Splitting riverbank polygons into small parts has always been documented practice and it universally used by mappers as you can now see in the standard style.

You probably did not notice my mapping tip to create a MP for thousands of lakes together was ironic. It is meant to illustrate that even if you are fine with the standard map style pushing mappers to a certain style of mapping against established practice and common sense this would still lead to completely ridiculous incentives in this case.

Comment from Alan Trick on 18 December 2017 at 20:11

Ultimately, spliting up a large body of water into smaller ploys is a violation of "One feature, one OSM element". Historically, this principal has often be waved because the editors just were really quite bad at dealling with large ploys and multipolgons (hence "tagging for the editor"). However, the situation has improved a little bit, and as it improves, I expect the principal will be more closely adheared to.

Comment from nebulon42 on 18 December 2017 at 20:48

Imagico I value your cartographic knowledge and I know that you already have presented your arguments before so I understand your frustration. I was not involved in that release and - so I think - neither were you. But since you are a maintainer of openstreetmap-carto: Do you think this blog post is appropriate or maybe you should have brought this up on the issue tracker or in other form? I'm inclined to answer no on the first part of the question.

Comment from imagico on 18 December 2017 at 21:39

@Alan - I think that is a misunderstanding of the "One feature, one OSM element" rule in two separate ways.

First the riverbank polygons do not represent the river, they represent the water covered area of the river. This area can be split in mapping just like landuse mapping can be split. The river itself is represented by the linear ways with waterway=river and potentially a waterway relation collecting those ways.

Second i think "One feature, one OSM element" does not mean you need to merge all features that are part of a larger scale concepts into one OSM element (like all natural=wood areas that are part of the Amazon rain forest into one giant multipolygon). In case of a river mappers standing in Hamburg are mapping the Elbe locally as a way waterway=river, name=Elbe - like here:

Mappers in Lovosice will map their river locally with waterway=river, name=Labe:

This is not in violation of "One feature, one OSM element" because to the local mappers this is not the same feature obviously. The larger scale concept of a river as a global feature is represented as a waterway relation in addition but that is a concept separate from the river as a local feature.

All of this has nothing to do with the subject at hand: The problematic mapping incentives created by using way_area filtering on riverbank polygons in the standard style. Even if you want to interpret "One feature, one OSM element" differently and create thousands of new giant MPs extending in some cases across whole continents to represent the water covered areas of rivers as global concepts the right way to do that is to discuss this with the mapping community and achieve consensus on that matter, not to push this from a prominent map style for convenience of the map designers.

@nebulon42 - i understand your point and normally if it had just been a loss of quality in the map this is about i would not have said anything - hoping that even if those making the changes won't listen to me they will over time recognize the problems and either fix them or ask for and accept help to do better. But this is going to have a massive effect on mapping pushing mappers strongly in a certain direction so i think it is important to bring this up. And as you noticed i discussed this matter on many occasions in the map styling context so arguing the same point yet again in style development after my views on the matter have already been ignored there many times did not seem a very productive course of action.

I have considered resigning my position as maintainer but as long as OSM-Carto is prominently featured on the OSMF infrastructure i don't think this is the responsible thing to do. So i will continue providing critical feedback and provide alternative styling ideas to whoever will listen. I am grateful if there is a possibility to do that in style development but if not i will also do so in other places like here.

Comment from BladeTC on 19 December 2017 at 21:58

This will be good to increase the quality of the data available and encourage good mapping practices. But the downside is that will be a lot of things to fix that only experienced mappers and JOSM users can do.

"One feature, one OSM element": easy to find, hard to fix. A large file can be very difficult to manage, and need some knowledge about multipolygons. It's a good move to encourage mappers to improve their technical knowledge of OSM, but at the same time we need to avoid making things difficult to new ones, because can be left even more things to fix.

My conclusion, it is a step ahead, we have a lot of work to do, and everyone will benefit.

Att, BladeTC

Comment from kocio on 23 December 2017 at 09:33

I'm an OSM Carto co-developer, who was one of a few people active with this change. Unfortunately this particular problem has not been reported when we were making decisions and testing it, as nebulon42 noticed (who is also co-developer). I learned about it from the fresh weeklyOSM 387.

I think the main problem to think about is proportions and think about the complex problems, not just one particular issue related to it.

Until recently we had no big lakes visible on low zoom levels and (if I remember) that encouraged to tag them as coastline, both of which was much worse in my opinion. My investigation showed that it's possible to fix these issues by filtering out sub-pixel water areas. It was purely technical obstacle, and solving it allowed us to show all the small, but over-pixel areas on low zoom. So when we just stopped rendering "water haze" of sub-pixel size and we were able to show big waters, we end up also with "water haze" of small, but over-pixel water areas. That's why we tried to limit also this haze - but this time it was not a technical problem, it was purely cartographic decision, which partially reverts the change with small water areas.

Now - it's about fixing low levels mainly. The new filter removes smaller waters on z0-z7 only and is regressive (it's getting weaker when zooming in). On z8+ (which is still big) we just got rid of sub-pixel areas, not small, but over-pixel. We were testing it to not loose to much and we have found that sub-pixel haze was not that big.

I hope that background makes the whole problem a bit more clear. For me personally limiting over-pixel haze on low zoom levels makes map a bit more readable - we don't have to show all the possible details everywhere, "actual geography" is what we have on aerial pictures. Of course somebody might want to overcome the filter, but we can see if it will really happen or not.

Comment from staytuned on 24 December 2017 at 07:46

@kocio: If you're just after filtering lakes, then why do you try to handle all water areas equally? It's perfectly possible to handle riverbanks (that are more often linearily shaped than not) in a different way than lakes.

@imagico: Supporting your comment above, add that the water surface of rivers is most often not level / homogenous, as is for lakes. E.g. weirs, dams, mouths and alike separate surfaces of a watercourse naturally. If such elements are present, multipolygon riverbank split points need not be distributed arbitrarily.

Comment from staytuned on 24 December 2017 at 08:14

@kocio: Actually, to achieve the same effect on riverbanks, it's better to filter on the river (mean?) width than on the surface area size of individual segments. If a river gains width, you will still loose parts below the threshold (upstream parts only, that may be sub-pixel sized).

This would be tolerant to the actual mapping practice used and will not cut out segments to display in a seemingly random way.

Obtaining width of a linearily shaped area may not be easy though, but should not be harder than calculating the area size. As a fall- back obtaining width by looking at the tags of the center line may be feasible.

Comment from kocio on 24 December 2017 at 09:21

I think it should be possible to treat riverbanks different than lakes. However this entry warns about filtering any kind of water on low zoom levels (see the last example with lakes).

And what is important to notice in the first place is that any act of limiting encourages cheating. If we allow rivers to be unfiltered, there is a danger that somebody will tag the lakes like riverbanks for example. So while being scared about big multipolygons is perfectly valid point of view, it does not say why the filtering was introduced and doesn't compare the problems and the gains of some possible solutions. Writing diary entry might be as subjective and narrow as it gets (it's personal after all), but when making decisions you should be aware of more than one problem and in the end accept that no solution is perfect.

So if you have some propositions what could be fixed, please report them on osm-carto issue tracker, so we could discuss them and choose the most appropriate solution. Diary comments is not the best place to improve osm-carto rendering.

Comment from imagico on 24 December 2017 at 15:35

I just wanted to point out that the description of the historic development in OSM-Carto by kocio is not quite right. The problem with low zoom waterbody rendering has been discussed and fully analyzed many years ago - i linked to some parts of the discussion in the blog entry above - others are referenced from there. I also discussed the topic a bit more than year ago here from a somewhat different perspective (the rendering side) which was also referenced from the style development here.

The 'solution' now introduced is an approach that has been well known in rendering of OSM based maps and used in many maps, in particular ones without a mapper feedback mandate/function and commercial maps where cost efficiency is the primary goal for many years with similar low quality results. This approach has been rejected several times in the past in OSM-Carto (see in particular here and here).

I agree that the OSM-Carto issue tracker would be a better place for discussing the problem but a productive discussion would require all participants being fully aware of the previous state of discussion and a common understanding of the cartographic goals. At the moment i can see neither of these here. The problem is that while it is easy to recognize the issues with the current approach (as i pointed out in this diary) the cartographic and technical context is complicated and it is tempting to try ignoring parts of this context in the attempt to facilitate understanding. Ideas like "just stopped rendering water haze of sub-pixel size" are an example of such over-simplification. If you want to make it really simple it is best to accept that way_area filtering for filled polygon rendering just does not serve any positive goal, it is purely a method of reducing rendering costs at the expense of quality. This obviously is an over-simplification too but not a harmful one like the other way round.

Comment from kocio on 24 December 2017 at 18:20

We might fix the problem with riverbanks not filtering them by size, but rendering them from some zoom level probably:

You might discuss it there or open new issue.

If you want to make it really simple it is best to accept that way_area filtering for filled polygon rendering just does not serve any positive goal, it is purely a method of reducing rendering costs at the expense of quality.

I can not agree with that whole sentence, even if it is true in some details. In my opinion it definitely serves a positive goal: reducing the high frequency noise. While sub-pixel filtering was just a purely technical decision (my intention was only reducing rendering costs, exactly as you say, visual changes at high frequency was not planned), current (v4.6.0) filtering is made exactly for this reason (hiding unwanted details) - it has nothing to do with reducing rendering costs. Of course you might argue (a) if this really needed and (b) if this is the best method to achieve this goal, and I guess b) is probably what you mean - but that's not what you just wrote.

For me personally these are 2 things that should not be mixed: 1. Cutting sub-pixel haze is acceptable, small trade off for showing big lakes. Not showing them on low zoom is unacceptable for me and many other people - it was a huge, common cartographic problem reported many times. 2. Current (much bigger) filtering is not necessary for me, it's just one possible way to skin the cat, but there may be some better ones.

I will oppose any proposition which breaks rendering big lakes on low zoom as nasty regression, but anything else is welcome and I can discuss any proposed solution - especially pull requests.

Login to leave a comment