OpenStreetMap
10 years of OSM data history
Posted by tyr_asd on 6 October 2017 in English. Last updated on 9 October 2017.Tomorrow is the 10 year anniversary of OSM’s API version 0.5. This is the version of the OSM-API that first exposed (among other things) the version number on all OSM objects, making it possible to access the full history of every object modification from this point onward.
This means that very soon, the full history planet file published on planet.osm.org will contain more than 10 years of editing history which can be investigated, evaluated and analyzed (using tools like the OSM history database oshdb that’s currently under development at HeiGIT on the University of Heidelberg, which I presented earlier this year at the State of the Map).
Of course, OpenStreetMap as a project exists for a bit longer than that (about 13 years now) and there was already quite some data mapped before the OSM API 0.5 was introduced 10 years ago.
Here’s an interesting side note: There already existed a history call in OSM’s API 0.4 (and apparently even in 0.3, see the comments below), but unfortunately this historic data apparently hasn’t been preserved in the newer versions of the OSM API, meaning that it is also not available in the (relatively) easy to use full-history planet dumps which are available nowadays. As far as I can tell, this “prehistoric” OSM data has basically been lost (though some of it might be reconstructible by analyzing the list of very old planet dumps on planet.osm.org). Does someone of you perhaps know what exactly happened to that data back then?
(Cake made by Liz LeBreuilly, picture uploaded by Blackadder, CC-BY-SA-2.0.)
Discussion
Comment from Richard on 7 October 2017 at 10:59
I don’t think that’s quite accurate.
OSM has always been versioned, from the very first pre-Rails site onwards. (All of this was managed in a file called dao.rb, I think, if you can find it in trac.) However, when segments were abolished in the move from 0.4 to 0.5, the history was not migrated into the revised database schema.
I believe someone (Firefishy?) has a database dump from the time of the changeover, so it would theoretically be possible to reconstruct the pre-0.5 history.
Comment from tyr_asd on 9 October 2017 at 09:06
Thanks for the clarification and background information, Richard! (But what exactly do you think is not quite accurate in my text?)
Comment from Richard on 9 October 2017 at 10:38
Just the bit about introducing the version number on objects. The version number was already present in the db - see https://github.com/openstreetmap/openstreetmap-website/blob/06f3473d1c64d2e0a1ecc6f1a9c4679a52bc4761/db/create_database.sql - it just (IIRC) wasn’t visible through the API. So “exposed” would be better than “introduced”. Only a minor gripe - I’m just slightly aware it’s good to get these things on the record before we all forget them. :)
Comment from tyr_asd on 9 October 2017 at 15:14
Right. I’ve changed the wording now. :)
Btw: I managed to find the pre-rails ruby code of the website/api you pointed at earlier: https://trac.openstreetmap.org/browser/subversion/sites/www.openstreetmap.org/ruby/api/osm/dao.rb?rev=2808#L1062 which seems to correspond to OSM’s API version 0.3 (while the rails-port started at version 0.4 afaics).
Comment from Richard on 9 October 2017 at 15:41
Brilliant. Haven’t seen that code in years!