OpenStreetMap

How to improve OSM: kill the bureaucracy

Posted by Tom Morris on 1 December 2014 in English (English)

I've been mapping for a while now. I have some issues with the way OSM currently works.

  1. The current tagging/import approval process is silly. To get a new tag approved formally, you are supposed to post about it on tagging-l. Then you are supposed to write a wiki page, then have comments on the wiki page, then have an RfC on the wiki talk page, then announce the RfC on the mailing list. If someone says "yeah, I'm not wild about this idea", that's pretty much a blocker for it happening at all. The alternative to this process is to simply start putting tags on things when you edit them and create an informal convention. Because the former process is so ludicrously painful and slow, it is much easier to tell people who you meet who say "I'd really like to start putting X on OSM" to just go ahead and do it quietly without telling anyone, and then it probably won't be a problem.

  2. The wiki doesn't match reality. The RfC process is so ludicrously over-complicated partly because nobody is sure whether the wiki exists to document existing behaviour or to specify what ought to be done. That is, nobody is sure if the wiki is descriptive or normative. It has thus ended up being both and neither.

  3. There doesn't seem to be a process for ever tidying the wiki, and no feeling that we should ever bring discussions to an actual consensus. Over on Wikipedia, for all its faults, admins do at some point close discussions and say "here's the rough consensus: X". On OSM, the discussions just rumble on forever.

  4. The culmination of this slow and silly process for approving proposed tags or imports is that now when friends come to me and say "I've got this really awesome idea for using OSM and we could contribute data back", I now say "yeah, good luck with that: the community process is so slow and bureaucratic, it often isn't worth bothering with". That's toxic. It's absolutely ghastly that smart, informed programmer types come to me and say "we've potentially got some amazing data we could contribute to OSM" and my answer is "that sounds awesome but the process is so broken that you shouldn't bother doing it officially, just do it quietly and unofficially".

There's some simple things we could do to fix this.

  1. Close most mailing lists. Mailing lists suck. Over at microformats.org we eliminated all our mailing lists and just use a wiki instead. See wiki is better than email.

  2. Get rid of most of the RfC process. It doesn't work.

  3. Switch to a default of "let's allow things to happen, then fix them when they break" rather than the current "let's have long, pointless mailing list discussions and then say no".

  4. Develop a process that is based around evidence collection. When I proposed the dress code feature, I provided comprehensive examples of different types of dress code that people were actually publishing on the web. The point is that when deciding on a new tag or feature proposal, we should be guided by the evidence of what people are already trying to express in other settings. The alternative process to this is called "making shit up out of whole cloth". It has a proud and noble tradition in standards-making. Alas, that tradition is also accompanied by a history of complete failure.

If we want enthusiastic contributors to work on making OSM fill all sorts of interesting and unpredictable new use cases, we should welcome those people in rather than keep around processes that are slow, bureaucratic and alienating.

Comment from jgpacker on 1 December 2014 at 12:53

The import approval process indeed can be demanding, but that's because it's so easy to make a bad import and so hard to fix it. It is far from being silly, and in my experience almost all imports that skip the approval process are bad imports (mainly because they tend to be made by an inexperienced user).

The wiki doesn't match reality.

Can you give some examples ? I'm not saying it does match, but we can always fix the problems.

There doesn't seem to be a process for ever tidying the wiki, [..]

I can say there are always a number of users regularly improving the wiki. A wiki is never complete, and is always in need of more helping hands (especially for non-english languages).

It can be common in the wiki to describe both the current state of tagging and the community's consensus. Nothing wrong with that.

Hide this comment

Comment from Tom Morris on 1 December 2014 at 13:00

The imports that "skip the approval process" are the people who go "urgh, I can't be fucked, I'm going to go watch TV".

The wiki not matching reality: well, the page for Key:wikidata doesn't tell you where the mailing list discussions are happening. It's not joined up. Moving all the discussion on to the wiki would mean that people could see what's going on.

As for lack of process for cleaning the wiki? There doesn't seem to be any kind of deletion process or rules for saying "this stuff is old, we can get rid of it now". I may have overlooked them.

Hide this comment

Comment from Vincent de Phily on 1 December 2014 at 13:04

Contrary to imports, there's no such thing as a tagging approval process. There are conventions to help reaching consensus, which is quite different. OSM has always been pretty much the "let's allow things to happen, then fix them when they break" phylosophy.

The "descriptive vs normative" can be a bit hard to navigate, but really the wiki should do both, and it usually does. Many pages describe both "what's in the db" and "what is the prefered method".

The wiki certainly does need TLC. The "process" to tidy the wiki is to just do it. That doesn't necessarily mean conclude the discussion, but the pros and cons of each view can be more clearly documented.

The level of bureaucraty is subjective. You're turning people away for reasons I wouldn't have batted an eylid about.

OSM has loads of communication channels. Many people actually prefer mailing lists, but there's IRC, a forum, blogs, social media, private messages and changeset comments... You name it. Besisdes, you want to fix wiki-related problems by adding another wiki ??

Hide this comment

Comment from Tom Morris on 1 December 2014 at 13:07

Contrary to imports, there's no such thing as a tagging approval process.

Good. So I'll just crack on and start adding tags to objects, document it on the wiki and unsubscribe from the waste of life that is Tagging-l. That's a win for today.

Besisdes, you want to fix wiki-related problems by adding another wiki ??

Nope. One should be enough. :)

Hide this comment

Comment from SK53 on 1 December 2014 at 13:23

@Vincent de Philly: you have to explain to an inexperienced wiki/OSM user why their new descriptive tag pages gets added to "Tags lacking a proposal" category. There is an approval process & to many in the outside world it looks 'official' (see the many articles about sexism in OSM based on one of the Childcare proposal pages, to see this). Whereas you and I know this is something favoured by an active minority, this is not always apparent to non-contributors, or contributors (as Tom's comments show).

@Tom Morris: Bravo, well said.

I'm not sure that getting rid of the mailing lists will help: take a look at some github pages for CartoCSS, and you'll see that OSM is adept at creating never-ending 'angels on pinheads' discussions in any communication form known to man.

Changing how tags are documented, and eliminating the voting on proposals, on the wiki requires some form of executive decision. I would strongly support moving to purely descriptive pages for key: and tag: spaces. Normative descriptions can be placed on Country and other relevant pages. To be effective this would require quite rigorous policing of the descriptive pages, which I suspect would fail because the most active maintainers of wiki pages also tend to be those most attached to the "Proposal/Voting" mechanism.

Hide this comment

Comment from Sanderd17 on 1 December 2014 at 14:44

The tagging mailing list is indeed a pain because most of all, we can't change the tagging when we want: If a tag is in use, no matter how bad it is, it's the "used" variant, will be documented so on the wiki, and will keep to be used. The tools (like mapnik) will also only accept the tags that appear often in the DB, and thus ignore the new ones.

So actually, proposing a tag (and letting everyone use it), requires patching all tools that use the data, and lobbying to get the patch in.

The tagging mailing list should only aim to give guidance about using tags. Point out what should be done better (to get some consistency), or why a feature shouldn't be mapped (we should only map stable, geographic data).

Anyway, for imports, or mass edits, it's different. Those need to be discussed before they're executed, because a bad import can ruin the hard work of many individual editors. And even a good import can be counter-productive when there's no community behind it to maintain the data (and only lack of data creates a solid community).

Hide this comment

Comment from RobJN on 1 December 2014 at 18:18

Close most mailing lists. Mailing lists suck.

The longer I think about this, the better the idea becomes. We have a large contributor base but only a few people get involved in any communication. Although we should recognise that many mappers are happy to just edit and not talk, we should be able to grow our talking community a bit more. Right now it's confusing and we have entrenched members with loud voices. Comms needs a rethink.

Taking tagging as an example: we shouldn't have a formal process, but we do need some place where people can say "hey, I want to tag this, but I don't have knowledge from around the world - what do you think about the tag v=k?". A "Tags lacking a proposal" category is a bad thing.

Anyone want to work together in developing a communications strategy? If we work it under the CWG we could ask for funding if some development is needed. First step would be to look at what the ideal situation would be.

Rob

Hide this comment

Comment from RobJN on 1 December 2014 at 18:34

Wiki discussion pages can look good and be easier to use: https://www.mediawiki.org/wiki/Extension_talk:VisualEditor

I'm thinking aloud here, but what if we had a communication portal on the main OSM website that included:

  1. Topic (title)
  2. Primary theme (e.g. Skybox for Good)
  3. Tags (e.g. Legal)
  4. Associated wiki page (optional)

The tags allow me to follow a discussion about the primary theme as it expands (I missed a Skybox for Good email the other day as I was watching "talk" and it got posted to "legal" mailing list).

The optional link to associated wiki page could either simply insert a link to the discussion on the wiki discussion page (enhancing future discover-ability) or put the entire discussion on the wiki (but using the front end developed for OSM.org)

Hide this comment

Comment from chillly on 1 December 2014 at 18:35

The tagging mailing list is wonderful. It keeps most of the pointless, circular discussions about tags that no one cares about off the other, more useful lists.

Hide this comment

Comment from joost schouppe on 1 December 2014 at 21:13

@Rob, would very much like to help out. I've got a thinking session on my to do list about dealing with and preventing (new) contributor mess-ups, which sounds like a similar communications problem

Hide this comment

Comment from AndiG88 on 1 December 2014 at 21:34

The tagging mailing list is wonderful. It keeps most of the pointless, circular discussions about tags that no one cares about off the other, more useful lists.

Which nobody else uses either. At least here in Germany it's 99% Forum.

Hide this comment

Comment from Pieren on 2 December 2014 at 11:03

I went through the arguments of this wiki-better-than-email but none of them is valid in our context. Our tagging mailing list is not providing "material of any substance". It's just discussions around what will go into the wiki page exactly like we have in the wiki "Talk" pages. The "signal to noise", "search" and "PD" are not relevant as well (use google tip "site:" if you like). Some people don't like mailing list because they simply don't have a correct email client supporting mailing lists.
They are counter arguments about the wiki. The main being that many people simply never used a wiki until they wanted to discuss about a tagging proposal. Don't forget that our audience is not limited to nerds.
I agree on one point : the process to adopt a tag can be heavy and slow. But you can bypass this process and simply use your owns. By doing this, you also take the risk that your tag is undocumented or badly documented and then unproperly used or interpreted by others. You also have to understand that tags are not just a wiki and a k/v in a database. When you create a tag, you also want that this tag is used by others and not only for rendering. Once a tag is adopted by data consumers, it is very hard to modify them (although not impossible, I did this for the tag emergency=aed in the past.

Hide this comment

Comment from Tom Morris on 2 December 2014 at 11:52

If the tagging mailing list is "not providing "material of any substance"", why keep it? ;-)

Hide this comment

Comment from Pieren on 2 December 2014 at 12:40

As I said, it is to discuss about a proposal, not to store the proposal itself. It is not more and not less of what the wiki "Talk" page is doing. Excepted that you don't have the learn the wiki syntax.

Hide this comment

Comment from RobJN on 2 December 2014 at 17:52

You can get over having to learn the wiki syntax by installing the VisualEditor extension. The advantage of using the wiki talk pages is that it's easy to find at a later date. The disadvantage is that your comment may go unnoticed for ages. Hence some sort of platform that links to the wiki but works similar to a mail list/forum could be good.

Hide this comment

Comment from Omnific on 2 December 2014 at 20:19

Regarding the import issue: I completely agree, something needs to be done. The proposal requirement for a large scale import is great (avoiding legal issues, etc), but the biggest issue is that other contributors, especially in the US where imports are desperately needed, are incredibly slow to respond. The mailing lists are beyond useless for the general OSM population and cater to the old, established mappers, and are quite off-putting to new contributors (it was for me).

Hide this comment

Comment from Richard on 3 December 2014 at 21:23

Please no! Imports really aren't desperately needed in the US. What's desperately needed in the US is for more people to clean up the disaster area that was the first import of all, TIGER, rather than piling more and more unmaintained, out-dated data on top.

Hide this comment

Leave a comment

Parsed with Markdown

  • Headings

    # Heading
    ## Subheading

  • Unordered list

    * First item
    * Second item

  • Ordered list

    1. First item
    2. Second item

  • Link

    [Text](URL)
  • Image

    ![Alt text](URL)

Login to leave a comment