OpenStreetMap

Experimental Overpass attic support in Achavi (augmented change viewer)

Posted by ikonor on 17 July 2014 in English (English)

achavi screenshot

As you might know, Overpass API has a new "attic data" feature (see Overpass API v0.7.50 almost done and SotM EU workshop). It allows to query for augmented changes by date range and bounding box. I'm very excited about this, because it now really makes sense for visualizing updates in an area, while iterating over minutely augmented diffs was a bit of an overkill. So I couldn't resist and made a hack to utilize this in Achavi:

http://nrenner.github.io/achavi/

Zoom in to your area of interest and press load to see changes within the last 24 hours. For continuous monitoring bookmark the Permalink and simply press load each time, the last load date will be preset as start date.

The old version will probably be replaced once the attic database is rebuilt. Code is in the attic branch (only for the brave).

Many thanks to Roland for implementing the attic feature!

Changeset

There also is an experimental feature for visualizing changesets:

  • drag & drop a changeset link onto the map
    (e.g. URL from changeset page or changeset link from WhoDidIt feed)
  • or use "changeset" URL parameter (optional "&relations=true"),
    e.g. .../achavi/?changeset=24168261

Load URL or file

Augmented diff files - i.e. the result of a custom attic adiff query - can also be loaded:

  • drag & drop local file onto the map
  • or pass the URL to a file as "url" parameter,
    e.g. /achavi/?url=http://.../charging_stations.xml
    This example is an uploaded result of the following query selecting updated charging stations for the recent Wochenaufgabe (en: Project of the week): http://overpass-api.de/api/interpreter?data=[adiff:"2014-07-01T00:00:00Z","2014-07-14T24:00:00Z"];((area["name:de"="Deutschland"];area["name:de"="Schweiz"];area["name:de"="Österreich"];);node["amenity"="charging_station"](area);>;);out meta geom;

Note that the url feature is somewhat limited due to cross-origin security constraints.

Limitations

  • attic data goes back until September 2012 (license change)
  • inconsistencies in old attic data (before June), there will be a database rebuild to fix
  • smaller bounding boxes (>= z15) take a couple of seconds, while larger can take one or two minutes or may time out, esp. with relations
  • only basic support for Relations in Achavi, thus disabled by default, check 'relations' to include
  • Achavi as a whole is still alpha
  • live update feature is removed for now, probably makes sense to separate. Has anybody been using this?
  • ...

Future

Unsure. Right now, there is other stuff I want to get on with. And before adding any more new features, some major refactoring should be done and probably switching to Leaflet.

Of course there is a lot to improve and there are many ideas, e.g.

  • having the textual list of changes of Roland's demo combined with the drill-down summary of Frederik's "Cool Name Goes Here" (WIP), both interacting with map visualization.
  • another idea was to somehow integrate the change visualization into Overpass Turbo to filter changes with your own attic queries, perhaps along with some custom MapCSS styling.
  • highlight only changed segments, esp. for long ways (landuses) and relations
  • ...

Comment from yalla on 17 July 2014 at 17:13

Wow, excellent!

Hide this comment

Comment from Zverik on 17 July 2014 at 19:24

So basically, if you add RSS, achavi would completely replace now abandoned new OWL?

Hide this comment

Comment from ikonor on 17 July 2014 at 21:11

RSS would need to be provided by a separate service, as achavi is a pure client-side JS project relying on the Overpass API. And Overpass API doesn't have changeset meta data yet (I'm using the OSM API for the single changeset feature), so changeset comments would be a missing piece for RSS and in comparison to the OWL history list.

Also, I wouldn't say "completely replace" as OWL was meant to be integrated into openstreetmap.org which achavi clearly is not. But I guess it can serve the same purpose, just with a different approach.

Hide this comment

Comment from d1g on 18 July 2014 at 11:09

Cool but I think it is missing feature "changed geometry" istead of "full old" + "full new geometry". But maybe I miss something.

How many changesets it can handle? In %? Have you tried to feed it with random changesets from full planet.osm?

I tried to feed it changeset with 2500 ways and 20K points. But it didn't even try to load CPU or memory... Is there upper limit how big changeset should be? My PC is okay (latest Chrome, 8GB ram, 4core cpu).

Hide this comment

Comment from ikonor on 18 July 2014 at 13:19

Yes, "changed geometry" is one of the main missing features, that's what I meant with "highlight only changed segments" in the Future section.

I have no idea how many changesets it can handle, but OpenLayers 2.x is not good at handling many features, that's one reason why there is the OL3 rewrite. The idea is to switch to Leaflet, but not highest priority right now. Reading from planet.osm is not really supported, Overpass augmented diff is the only format that will be supported.

I have set a loading limit of 50M characters because larger XML files were crashing the Browser, there will be a console error when reached.

Hide this comment

Comment from Diomas on 18 July 2014 at 13:39

Best diff viewer, I've seen!

I wonder, if it could handle elements that were deleted in some changeset that consists of only deletions? The problem is such changesets doesn't leave any marks on the bbox osm data: no object -- no changeset references.

My understanding is client-side history viewer just can't do this without storing planet diffs somewhere. Am I wrong?

Hide this comment

Comment from ikonor on 18 July 2014 at 15:00

Yes, it can handle and visualize deletes and delete-only changesets (example).

The problem with regular OSM data is that deleted elements are not included (only in full history planet) and regular diffs (OsmChange format) have delete information but without geometry (node lat/lon or way nodes are missing).

For the server part, achavi is relying on the Overpass API - where all the hard work is done - and it's Augmented_Diffs (now using the adiff query) that contain the old version with geometry for modified and deleted objects.

The previous Overpass version created augmented diff files by gathering the state of all changed objects before applying the minutely diffs. The new Overpass attic feature keeps a complete version history since the license change and creates augmented diffs on-the-fly.

Hide this comment

Comment from Hawkeye on 18 July 2014 at 23:20

Great. thank you. Bookmarked!

Hide this comment

Comment from poppei82 on 19 July 2014 at 09:31

GREAT!!!!!! Ist it possible to see HOW geometries changed?

Really, really useful tool! Should be integrated in OSM main page or JOSM to support mapping!

Hide this comment

Comment from ikonor on 19 July 2014 at 10:04

You can see how geometries changed by comparing old (dark, brownish red) and new (green) geometries (see legend). Highlighting only the changed segments is not implemented yet. Turning the "changes" layer off and on might help. For ways, added, deleted or moved nodes are shown. You might need to zoom in as nodes and old geometries are not shown on lower zoom levels for perfomance reasons.

Hide this comment

Comment from poppei82 on 19 July 2014 at 15:27

Thanks for explanation! I like this tool!! Thanks a lot!

Hide this comment

Comment from Stalfur on 19 July 2014 at 18:55

Looks and works brilliant!

Hide this comment

Comment from nidi0 on 23 July 2014 at 21:01

Excellent tool - really useful and even nice looking !!

Hide this comment

Leave a comment

Parsed with Markdown

  • Headings

    # Heading
    ## Subheading

  • Unordered list

    * First item
    * Second item

  • Ordered list

    1. First item
    2. Second item

  • Link

    [Text](URL)
  • Image

    ![Alt text](URL)

Login to leave a comment