OpenStreetMap

Mapillary Have a Special Hell Reserved Just For Me

Posted by alexkemp on 25 November 2016 in English. Last updated on 30 November 2016.

Mapillary is a Swedish organisation that, like Marvin the Paranoid Android in Hitchhiker’s Guide to the Galaxy, has a Data Centre for storing photographs as big as the planet. When you Register with them you can store GPS-registered photos on their site (really useful when surveying for later mapping).

My profile on Mapillary shows that I’ve uploaded 3,500 photos and have travelled 179.6 km whilst doing that. It also shows that I’ve uploaded the last 81 photos 6 times (making 486 total uploads in that sequence).

I’m currently mapping in the north of Nottingham in a district called ‘Gedling’ (south of Arnold Lane and north of Westdale Lane). 81 is a very typical number for me to shoot in a morning or afternoon whilst mapping. I used to use the Mapillary app in JOSM to upload, but tend to upload directly from a browser these days (the JOSM app requires a confirmation within a browser, so I cut out the middleman).

The sequence went very normally with those 81 photos, except that the Mapillary browser did not confirm the uploads within my profile. At first, I also got zero reply from Mapillary support. I kept trying to upload…

I eventually got an email from Katrin at Mapillary support, and she copied the email to Peter. According to the email that Peter sent this morning, the issue was because the “harvester for manually uploaded images has not been running” (he restarted it, so all 6 identical sets of images were harvested at once). Problems with a Harvester seem the correct kind of issue for this time of year.

Update:

I sent an email to Peter saying “So, no-one could manually upload photos? And I’m the only one that manually uploads photos?? Good lord.” Fortunately, he seems to have a sense of humour. He replied that:

  • no web-uploaded images have been processed in the last 2 days
  • that affected ~200 people
  • it involved ~500k images
  • mobile apps use another method, so uploads did not actually stop
    (they halved, hence no-one at Mapillary noticed)

Joke:

A Medical Doctor gets called out to a Sawmill. He has to push through a large crowd of sawmill workers all busily gawping at some scene occurring in the middle. As he gets closer the noise of the mechanical circular saw gets louder & louder…

At the centre is a man with the stump of his left arm wrapped in a bloody cloth. He is stood near the circular saw, whilst being comforted by some colleagues. His hand lies amidst the sawdust at the base of the saw.

The doctor: “Good lord, man, what happened to you?”

The man (a little unsteadily): “Well, doctor, I was working at the saw, and I went like this…” (gestures) “…and GOOD GOD! THERE GOES THE OTHER ONE!”

Coda: 100 million Mapillary Photos Uploaded

An email from Mapillary boasts that they now have 100 million photos on Mapillary. The link says that it has taken 2½ years & 2.1 million km, and was 10m photos last year. My calculation based on the numbers given earlier suggest that it is currently 500k uploads/day (from both mobile apps & web-uploads) = 182.5m/year.

This all suggests a large speed-up in the last year.

Location: Gedling, Carlton, Gedling, Nottinghamshire, England, NG4 4BH, United Kingdom

Discussion

Comment from escada on 25 November 2016 at 14:25

It was not only their harvester that stopped working, one of there web server had to be restarted as well earlier this week.

Comment from alexkemp on 25 November 2016 at 16:11

Interesting comment, @escada. With ~400 people uploading 1,000,000 photos every 2 days (manual + mobile) (73k people => 183m photos yearly), then god knows how many folks viewing those photos, the possible miracle is that the servers manage to stay afloat. I spent 15+ years managing a single webserver and recall continual firefighting as I battled attempted scrapes (max 403 / second from Technicolor) & spam. Yet that was with a quiet machine. Goodness knows what Mapillary or OSM must be like.

Comment from escada on 25 November 2016 at 16:40

I had informed them via twitter, here’s their reply: https://twitter.com/peterneubauer/status/801408335299969024

I’m also part of a dev team that builds, deploys and maintains a webapp for a rather small audience. But yes, it is sometimes a problem to keep things running. Even with all kind of monitoring tools, it is still a challenge. So I’m certainly not complaining.

Log in to leave a comment