OpenStreetMap

Andy Allan has commented on the following diary entries

Post When Comment
The moderation queue. The first 1000 issues 9 days ago

Thanks mavl for posting these statistics! It's great to see that the system is being well used - it certainly took a lot of development work, by a lot of different people, in order to get it fully working and deployed.

Rob - there's already spam detection and filtering in the website, but there's always room for improvement. Perhaps these reports and issues can be used to help improve the filters? If anyone is interested in doing this, then they could contact the DWG, who currently handle most of the moderation tasks.

When OpenStreetMap met Mapbox-GL : 🍚IDLY-GL 6 months ago

@Andy Allan: as you're mentioning this point: the pull request to remove those remaining slow parts in the map call is already out there, waiting to be reviewed, merged and deployed: https://github.com/zerebubuth/openstreetmap-cgimap/pull/142

That PR is unrelated to what I've described above. Sure, it speeds the map call up, but it doesn't make it any more cacheable that now.

When OpenStreetMap met Mapbox-GL : 🍚IDLY-GL 6 months ago

@Komяpa Of course it would be great if we could support unlimited requests on the /map call. But since that's not the case today then I think it's nicer to give an early warning rather than a potential bigger problem later on!

As I'm sure you're aware, but for the benefit of others who might be interested, the /map API call is hard to scale for two reasons. Firstly, it works like a WMS service with arbitrary extents, rather than like tiles with fixed extents. Secondly, it's crucial for all editing software that it provides read-after-write consistency, so that when a changeset is saved, the next /map request is guaranteed to contain that fresh data. So both these reasons make it hard to cache responses and without any caching it's hard to scale.

There have been proposals to fix both of these over the last few years. This would be by making map calls tiled, and by providing a "not-before-changeset" parameter or similar for the consistency issue. Then we could cache responses without breaking the editing workflow, and support more use of the /map call. But like many things, we need more people coding, and more community support for those who are already coding. Suggesting that we should deliberately violate the policy, and "break OSM" so that "someone pays attention" would not be the best approach! :-)

... and so it begins 6 months ago

Good luck with your project! We also provide development versions of the main OpenStreetMap website, so if you'd like to add loads of notes without cluttering up the real OSM database, then please use one of them. For example, the "master" option is the same code as on openstreetmap.org but with a test database.

When OpenStreetMap met Mapbox-GL : 🍚IDLY-GL 6 months ago

Nice project! However, it violates the API Usage Policy. The API is only provided for editing OpenStreetMap data. If you want to display OSM data then you need to get that data via planet.openstreetmap.org or a service that is built on top of the planet feeds.

Not Yours, OpenStreetMap 6 months ago

The second part of my response is focussed on two projects mentioned here that I'm closely involved in. Perhaps unlike other core contributors, almost all of the work I do is not as a means to the end, but instead as a means to get more people involved, to lower barriers to entry, and to speed up future development.

The first is the cartography.

Then it was converted to CartoCSS, made prettier, and contributors started flocking in.

I think I did the right thing here, do you agree? That the conversion was the correct thing to do? I deliberately added 6 more maintainers and gave them all the same power and authority as I had. But you go on to say that it's become bleak, that nobody knows what to do, and in your eyes the project is a failure. So I would like to hear from anyone what I did wrong, and what I should have done differently? Should I have stayed as the only maintainer? Or something different? Clearly some mistakes must have been made, and I'd like to hear about them.

The second is the website:

Two guardians do not let through any unconventional change: it’s like amidst a crumbling world we must hold on to what we already have. They don’t see that the power of their grip is what crumbles their world.

Well, I'm one of those unnamed "guardians" I guess. But I've never considered what I do there as clinging to power, or preventing changes, or anything of the sort - but obviously you think differently. I'd like to hear more about your point of view. I've worked hard to refactor the code for the last two years, I've put in hundreds of hours on getting the moderation branch ready, and I've reviewed and dealt with lots of issues and pull requests, many of which have been outstanding for years. All of this is focussed on getting more people involved, not fewer.

I haven't made many big user-facing changes since that's not what I think is the most pressing problem that I can solve. Like with the stylesheets - I didn't redesign the map from scratch, I took the moribund project, made some technical changes to make it easier for others to get involved, and opened it up to the community. I'm trying to do the same thing for the website, taking the existing project, making technical changes to make it easier for others to get involved. I hope someday soon to add another half dozen maintainers to that project, and move on to the next most pressing project in OSM.

That's my point of view. But it's clear that you disagree fundamentally with two of the largest projects that I'm involved in - there's no harm in that, everyone has their own views. I'd like to know more details on what you believe I'm doing wrong, and what I should be doing instead, to reinvigorate OpenStreetMap, since what I've been doing for the last few years apparently isn't working.

Not Yours, OpenStreetMap 6 months ago

I'm going to divide my response into two parts, in separate comments.

First, my response to the general theme. I agree it often feels like OSM is missing a certain spark that existed years ago - a willingness to bump API versions, or reconfigure everything, or rewrite code or prose from scratch - but that's not my point. My point is to ask the rhetorical question - what could be done to snuff out these sparks? What could be done to make sure nothing ever changes again? What could be done to drive out the passion, the creativity, the resourcefulness and the enthusiasm from everyone involved?

Posts like yours, that's what. Posts like Serge's. Posts that do nothing other than berate everyone for being shit, that provide no solutions, and that just bring everyone down.

This isn't a 1500 word essay that encourages contributors. This isn't a 1500 word essay that makes anyone think "you know what, I'm going to help with this". This isn't a 1500 word essay that highlights something good that has been done and encourages more good things to happen.

If there's any grand problem in OpenStreetMap, it's that the loudest talkers are the ones who are bemoaning the lack of work that everyone else is doing. It's a great shame. It's so discouraging.

But it feeds into the wider point. The sort of people who put up with this environment are rare, so we have few core contributors. Those who are here are generally thick-skinned, battle-hardened, stubborn - or all of the above. So it should come as no surprise that change is infrequent, risks are not taken, the status quo is maintained. The irony is that the very act of complaining about the status quo drives more people away, and reinforces the status quo further.

So if you want to see change, don't moan about the lack of it. That approach has been done repeatedly, and we can all see the damage that it does.

Focus instead on what you like. Celebrate the successes. Cheer on the progress, however little you might find. You have a rare gift for writing and communicating ideas in OpenStreetMap, so use it to encourage the bright future that we all want to see.

Motorway Junction Node Placement 10 months ago

I'd much prefer option 2, since it more accurately represents what exists on the ground. Options 1 and 3 start introducing angles and corners ("Warning! Sharp bend ahead!") that don't exist.

Option 2 is the best approximation of the route that a fully informed driver will take, namely a straight line from the point of lane departure to the exit. Imagine instead that drivers follow option 3 - 45 degree turns just before the gore? I don't think that would be right.

Metro Mapping Proposal (and what's wrong with proposals) 12 months ago

not after stalled pull requests to the OSM website

I know this is off-topic, but I'm trying to work through all the pull requests for the OpenStreetMap website code, and deal with them one way or another - and hopefully deal with them all more quickly in the future. Any help is welcome. Last week I tackled three of the oldest, but there's still some that are 5 years old!

diary spam about 1 year ago

This particular thing actually was a previous GSoC task, but one that hasn't yet been polished off ready to be merged. But I'm happy to slowly work away at reviewing and improving it, albeit a long, long time after the GSoC student and mentor involved have moved on to other things.

GSoC isn't the best solution when the problem is a lack of regular contributors. We can easily end up with a big feature and nobody to merge it or maintain it. We need some people who are willing to help (and that might involve learning to code, many developers like me are self-taught) with all the routine tasks, big and small, that would improve OSM for everyone.

If EWG want to help encourage more contributors then I'm all for that. Making improvements - however small - to the website has a huge impact on OSM since it's used by literally every single contributor. So I hope that there's someone who is reading this diary who will realise that they can play their part too. That's why I mentioned it!

diary spam about 1 year ago

Yes, there's work in progress to allow users to flag diary entries (and other things, like changeset comments) for moderators and admins to look at.

I'll just say here that there's very few people actually writing code to improve the website - surprisingly few, given the thousands of people who use the website and the API each day, and the hundreds of developers in the OSM ecosystem. So if anyone can help us out, help is definitely appreciated!

Adding vector tiles to the components diagram over 1 year ago

Just a minor point, but the OpenCycleMap and Transport layers both have the same rendering stack, so I'm not sure why there's a separate "Transport Renderer" on there. Both layers are an osm2pgsql -> PostGIS -> vector tiles -> raster tiles type of stack.

3 years of welcome messages, more than 3400 of them almost 2 years ago

Vincent de Phily - you would need to choose them randomly for it to be effective. If your choice of send/not-send is based on a particular factor (e.g. location) then it skews the results of the experiment.

3 years of welcome messages, more than 3400 of them almost 2 years ago

You could try an A/B experiment for a while - take the next 1000 users, send welcome messages to 50% of them, and then see if there's any difference between the two groups of users.

100 € for multiple social accounts in OSM about 2 years ago

I think you meant pull request to openstreetmap-website :-)

Heavy Usage on OSM Sites about 2 years ago

Hi Alex - the question of tileserver policy, and the outage on Friday, are unrelated. Friday's outage was related to the addition of more disks to our "services" machine Ironbelly, and while the RAID array was rebuilding it caused the machine to become overloaded. Other machines depend on ironbelly, including the servers that power the website and the editing API. We slowed the RAID rebuild, and stopped some non-essential services (including chef runs, log analysis etc), to get the website back online as quickly as was possible.

The network traffic you see on the 2nd November to ramoth is unrelated to the above. In this case, we were rebuilding indexes on the primary database server katla, to compact them which saves space and makes things work faster. The large amount of network traffic is the main server (kalta) distributing the rebuilt indexes out to the slave database servers, including ramoth.

These are all isolated from the tileservers, which have their own servers and databases, and weren't affected by either the ironbelly issues or the coredb index rebuilding. The policy discussion on them is a long-term thing, and the amount of traffic to the tileservers doesn't have any impact on the traffic to the website and/or the API.

I hope this provides some more information to you!

100€ for a subscription to diary comments about 2 years ago

Good idea - let's see if the grant works! I've been working recently to make the code (and the tests) around diary entries easier to work with. It's not as simple yet as it could be, but I'd be happy to review pull requests. Hopefully in future these sort of features will be even easier to develop.

Note that it's not OWG as a group, but the maintainer for the openstreetmap-website codebase, that decides whether pull requests are merged.

Ramblings about State of the Map about 2 years ago

what do you say to famous people like Andy Allen or Frederik Ramm ?

Anything you like! We're all nice people and happy to chat. Next time you see me, please come and say hello. :-)

Responding to suspicious changes over 2 years ago

Of the 250 changesets where you didn't get a reply, how many of the users involved have made further edits? It might be reasonable to expect a response from mappers who are continuing to be involved in the project, but you might be waiting for mappers who never contribute again.

The story of the oldest node in OSM. over 3 years ago

Remember that there were no changesets back in 2005, so it wasn't really the "fourth changeset ever" so much as the 4th artificially-created changeset at the point we created them (in 2009).

Also, most of the low-numbered nodes have been used and re-used a number of times, usually because of badly-executed bulk uploads using them instead of using new node ids. Then someone has to move everything back and/or delete the bulk upload.