OpenStreetMap

HiddenStreetMap

Posted by alexkemp on 9 June 2019 in English (English)

How Open is OpenStreetMap?

StreetMap yes, but I’m not too sure about the Open bit.

Consider the following:

  1. These Diary Pages are Closed Tight Shut
     
    The HSM admin unilaterally closed all Diary pages to the outside world on 18 May. Ostensibly that was a short-term measure to stop a spam-storm. We will know more if access for the SEs ever get restored to those pages.
     
    The admin action means that only those that already know that there are Diary Pages can visit them. In addition, it means that there is zero method of finding any specific page on this site, as there are zero internal search facilities. That is a dramatic problem, since a great many people have taken hours & hours to give large amounts of help for mappers via these pages, all of which is now invisible.

  2. The Map Pages are Closed Tight Shut
     
    There are millions & millions of HSM nodes, ways & relations within the Map accessed from osm.org, and not a single one is listed in any Search Engine. That is also a deliberate measure by the HSM admin although, unlike the Diary pages, it has been a feature for a great many years. It means that not a single one of the millions of Points of Interest (PoI) that have been entered by all of HSM’s mappers will ever be found by any general user. That raises a very important point:– is that what you want, and does it have to be that way? And if by any chance your response is “well, nothing I say will make any difference” you now know why OSM has become HSM.

  3. All Admin action appears to be Unilateral - Zero Consultation
     
    Apart from the website issues page, which only a few geeks will even know exists, there appears to be zero consultation between those that direct action on the site & the HSM community.
     
    No-one can know everything, and an open-source movement is supposed to provide a method for an additive process to be in place. Is that happening with HSM?

  4. The HSM Admin are removing items that they do not like without consultation
     
    I’ve had code snippets in some recent pages which the admin have removed. They have never said anything to me, nor asked before removing them. That is disgusting, and the opposite of open. Welcome to the gulag.

Comment from Warin61 on 9 June 2019 at 23:40

OSM data is searchable, just not using the common web search engines

https://www.openstreetmap.org/search?query=way Tower of London

OSM is not everything for all, but free data for map makers.

Comment from Glassman on 10 June 2019 at 01:47

All Admin action appears to be Unilateral - Zero Consultation

If these were an iD developer the “community” would be all over the fact. What makes the difference?

Comment from TomH on 10 June 2019 at 07:18

Wasn’t it you that suggested blocking indexing of the diary pages as a temporary measure? Certainly it’s something we can revisit now.

As to why we don’t allow the geodata to be indexed it’s because of the load hundreds of searching engines crawling billions of objects puts on our servers.

Comment from alexkemp on 10 June 2019 at 09:16

Wasn’t it you that suggested blocking indexing of the diary pages as a temporary measure?

No TomH, that was you. My suggestion was to place all new users into moderation (be hidden & unable to post) until they had made some edits to the map. That was a non-destructive measure that would instantly stop spam. I think that the Diary pages are useful, you see.

the load hundreds of searching engines crawling billions of objects puts on our servers

I’m a former 15-year professional webmaster that paid his own hosting costs and, therefore, I’m well aware of that fact. At this moment HSM has 90 servers that are handling upto 250 connections per second. SFS has one server that is handling ~177 API queries/second + additional forum visits with ease. HSM is also missing proper Content Negotiation on Diary pages, which increases load and bandwidth considerably. If those pages are any indication, then the whole site load could be reduced without sacrificing access.

You see, I am not only capable of independent thought, but also indulge in that activity. I am inviting others to try a little as well.

Is that what everybody wants, and does it have to be that way?

For me, the answer is “No” and “No”. If everybody else decides differently, then I think that we should change the name from “OpenStreetMap”. to “HiddenStreetMap”.

Comment from alexkemp on 10 June 2019 at 09:29

OSM data is searchable, just not using the common web search engines

Yeah, I got that, Warin61. The point is that they can NOT be searched from outside of HSM, only from inside. That fact means that they are hidden for the greatest part of the population.

I like being part of an open-source movement, and do not actually want to be part of some Hidden Society.

Comment from ndrw6 on 10 June 2019 at 12:40

I don’t mind having diary pages delisted from search engines if that helps reduce spam. It doesn’t make the project any less open and it matters that the community communication tools are kept functional (this includes them not being overwhelmed by spam).

We will probably need a full text search feature, though.

Comment from alexkemp on 10 June 2019 at 13:24

Hi ndrw6

I don’t mind having diary pages delisted from search engines if that helps reduce spam.

You write as if the two are necessarily connected. If that were true then the Internet world would be entirely composed of gulags that cannot connect. Now I accept that the totalitarian remnants of this world (China, Russia, North Korea, etc.) are like that, but that is not how the Internet was designed, nor how the rest of the world carries on, nor what is necessary to prevent spam, nor what I want for this world’s future.

It doesn’t make the project any less open

If you truly believe that then you are in a state of denial. Seriously, if you think that the best way of fixing problems is by killing the patient then you had better drink your final Kool-Aid.

it matters that the community communication tools are kept functional (this includes them not being overwhelmed by spam)

Yup, completely agree with that one, ndrw6.

I’ve been fighting spam for 20+ years and know for a certainty that HSM has been deploying the wrong methods. In fact, it has been deploying zero methods as best as I can tell. Any organisation that says “Let’s shut it all down as to stop the spam” has lost their tiny minds and actually hates what they are doing. No mother would ask a doctor to kill her child as to fix a disfiguring disease unless she hated her child, or had gone insane, or both.

Comment from ndrw6 on 10 June 2019 at 23:11

Just stating my preference: if for any reasons spam can’t be contained and getting rid of diaries was on the table, I’d be OK with making them accessible to logged in users only. It wouldn’t make them any less open but we would have lost their promotional aspect. I consider diaries a useful community building and communication tool. Definitely better than following hundreds of Facebook or Twitter accounts.

Comment from alexkemp on 11 June 2019 at 15:11

@ndrw6:

I’d be OK with making (diaries) accessible to logged in users only.

Why on earth would you want that?

Going back 20 years most forums allowed anonymous posts. Then spam began to bloom. In the current era not a single internet forum allows anonymous posts. However, almost every forum allows anonymous viewing & searching for most of their site, which is completely different.

Modern forum admin have had to learn how to deal with spam, or go out of business. What has become obvious is that none of the HSM admin have any experience of dealing with spam. Now, ignorance is not a fault, but dumb-arse reactions to spam definitely are.

The HSM equivalent to anonymous posts is allowing non-editors to post. For some reason, some influential members have historically taken against enforcing map-edits on posters. That has led directly to our current spam problems. It is now time to decide - do you want an OpenStreetMap or a HiddenStreetMap? And remember, most of HSM is already hidden from common view. Do you want that to continue?

To try to be clear, these are the proposals:

  1. In general, allow anonymous GET visits
  2. Restrict POST privilege to users that have made xx map edits

There would in addition be other suggestions related to SE access & bots generally + technical aspects on HTTP/1.0 Content Negotiation, which is missing, but I want to consult on that first before I put any suggestions forward.

Login to leave a comment