Now that Vespucci 12 is already nearing release (if you are helping with the translations, please have a look at them asap, as they are the major hold up), it’s time to think of what is coming next.

Vespucci has supported unconnected mapping since day one (nearly a decade ago) and that has improved over the years with addition of support for reading and saving files in normal OSM and JOSM xml format (the JOSM format stores information on local changes).

In version 10.1, early this year, we added support for MBTiles files for local on device background imagery sources, supporting building an imagery source while you are online on your desktop and avoiding having to download to Vespucci imagery cache manually which tended to be rather tiresome.

But despite all of this, the main problem remained that you need to keep OSM data files on device reasonably small because the contents would be read in to memory and while such areas can be quite large on a modern phone, things tends do to get slow. And as we all know when you make a selection in advance, Murphy comes in to play and you are surely are going to miss exactly the area that you suddenly stumbled in to.

To get around all of that I’ve been investigating a compact, indexed in one way of the other, on device format for raw OSM data for quite a while, and now have a solution that works even better than what I originally envisioned.

The format is simply a MBTiles format sqlite database containing tiled OSM data in PBF format, nothing special. Some aspects of the generated files are not even particularly refined, for example there is currently only one tile size supported which make it a bit inefficient for empty areas. However it already works well as is, so no need to rush to improve that.

Having all of Switzerland (not exactly an empty country in OSM) will use 433MB, which compares quite well to the original PBF file it was generated from (295MB). Some of the overhead is due to having to duplicate ways crossing tile boundaries and including relations in every tile that has a member element and tiles with little content not compressing well, but overall the size increase is not big enough to cause concern.

To update the tiles it is currently necessary to re-tile the data (after either downloading a new extract or keeping the extract up to date from the diffs). This is not a big deal, generating the tiles for Switzerland from scratch only takes a couple of minutes, but naturally the holy grail is to update in situ on device, doing away with the need for a desktop completely. There are some hurdles that need to be taken before we are there, but it seems to be completely possible. Even if that turns out to be not practical there are some potentially interesting intermediate steps that I’m exploring, for example updating the tiles with the local edits (after they have been uploaded and received definitive IDs).

Some of this functionality is likely to be available in 12.1 which I expect to be available in Q1 2019.

The fist beta/test version of Vespucci 12.0 is now available from the play store (together with the monthly update for the current release, 11.2.3).

Simple action mode

The current way (pre version 12 that is) of adding new objects with a long press goes back to the work done by Jan Shejbal during GSOC 2012 (a couple of years before I had anything to do with the project). This has the advantage that it doesn’t need an extra menu, you can inspect where you are placing the object before actually creating it, and in general that it is fast. The downside is that it can be a bit fiddly at times and users need to be told about it in advance and as we all know RTFM is not very popular.

Simple action mode replaces the long click action on the screen with a menu driven way of creating new objects. Long clicks are disabled as long as the mode is active.

Tapping the large green floating button will show a menu. After you’ve selected one of the items, you will be asked to tap the screen at the location where you want to create the object, pan and zoom continues to work if you need to adjust the map view.

The mode can be toggled on and off via an item in the main menu. See a comparision of the two modes for more information.,

Support for MapRoulette

MapRoulette tasks are now supported in the task overlay. While mostly MapRoulette tends to be revolve around aspects of mapping that can be done remotely, in line with the support for OSM Notes, Osmose bugs and custom tasks, it does seem to advantageous to be able to view and resolve such issues when you are actually on the ground at the location.

To be able to change a tasks status you will need to set you API key. If you have not set it, you will be asked on first upload, preferably you should set the key before making any changes via the “Tools” menu. This is a first cur at support, and may evolve over time.

Notes are movable

Newly created OSM Notes that haven’t been uploaded yet can be moved by selecting the Note and then dragging. Behaviour for downloaded Notes remains unchanged.

Most recently used support for OSM keys, values and roles

This adds support for storing, retrieving and prioritized selection display of keys, values and roles that have been recently used. When possible the mru values are associated with a preset.

Persistent storage is provided by an XML format file (“mrutags.xml”) in the publicly accessible “Vespucci” directory on device. The contents of the file can be inspected, and for example used to indicate what is missing from the presets.

In-app function to provide developer feedback

You now can, by using the “Provide feedback” function in the main menu, directly open a new issue on our github issue tracker with the most important information about your device and app version automatically included. If you have a github account you can simply login with that, if you don’t, you can submit an issue pseudo-anonymously with your OSM display name included. If you are using the later and haven’t yet authorized Vespucci with the OpenStreetMap API, you will be asked to do so.

The anonymous issue creation is only available in the official builds or if you are building the app yourself if you have configured a github personal access key during the build process.

Miscellaneous

• more layer configuration, specifically for the tasks and grid layers is now accessible from the layers dialog.
• using google login during the authentication process should work again.
• alert behaviour for notification groups is now configurable.
• improved UI and error behaviour when merging objects.
• preset items are now alphabetically sorted by default in the displays, this can be changed in the preset itself.
• Nominatim searches can now be limited to a bounding box.
• in multi-select mode we now show a total element count in the header of the tag editor.
• the build process now uses proguard which results in a roughly 20% smaller APK file.

And last, but not least …

I added https://twitter.com/sp8962/status/1070308345725239297

I just received a request to add a preset for amenity=language_school to my fork of the JOSM presets (that you should be using instead of the original :-)) Nothing to said against that, it is a reasonably popular feature with nearly a 1’000 uses.

Naturally a preset that is essentially just a stub isn’t really helpful, so as always I checked what additional properties should be added, and tagging the taught languages is an obvious attribute that is interesting.

Much to my dismay it seems as if in early 2016 it was infected with the “move values to key” plague. Instead of having two keys, lets say

language

and

language:main

containing a list of the languages, we now have more than two hundred potential keys for an in principle, simple attribute, making both data consumption, editing and creating a preset more than just difficult (as the keys can have at least three values plus unset, they can’t even be modelled with a checkbox in a preset).

At the time somebody further thought it was a good idea to retag existing language tags to the broken schema:

The beta should be available on googles playstore and from out github repo in a couple of days. No fancy pictures of new features as there are none.

End of support of Android 2.3 and 3.x for builds distributed on google play store

As announced previously this version no longer supports Android versions prior to 4.0 on googles playstore due to restrictions that google is now imposing. It is however possible to either build your own APK with support for earlier Android versions and likely we will be distributing such a version via F-Droid and our github repository.

Do be able to do this we now have two build flavors:

• current that is built against a current Android support library and supports Android 4.0 (14) and higher.
• legacy that is built against the last support library that supported 2.3, but will naturally run on any later Android version too-

Right now the current build mainly profits from bug fixes to the support libraries, for example the issue with sub-menus on tablets is resolved in the version we build the current flavor against.

This is not a commitment to supporting 2.3 indefinitely, but as we suspect that google will be increasing the minimum Android that practically can be supported in the future we expect that the legacy build will continue as a vehicle to support older versions.

Changes for Android 2.3

On devices that only supply 32MB or less of heap to apps, memory usage of Vespucci had gotten to the point at which the app could potentially completely run out of available memory. To work around this a bit we’ve stopped loading the simplified country boundaries which saves roughly 4MB, this implies that any country specific features will not work. Further we recommend to, instead of the standard bundled preset, to load the untranslated one (see the on device Preset help page) which will save roughly another 3MB (mainly because we don’t build a translated search index then).

Miscellaneous

• Internationalization support for OSM Notes and other tasks has been improved.
• Note and bug status is displayed in the disambiguation menu.

We released Vespucci version 11.0 at the beginning of July, 2 months later, the first beta of 11.1 is now available on googles play store and from our github repository..

Before going in to some details on 11.1, a quick word on the plans for the rest of the year. I hope that we can limit the testing period of 11.1 to the month of September and release early October.

My current plan is that 11.2 will have no new features and will solely be a vehicle to deal with googles forced update policy (see https://github.com/MarcusWolschon/osmeditor4android/issues/773 too). This also implies that there will be no update/bug fix releases of 11.1 after November 1st, so get on with testing :-) and that the google play version of the app is very unlikely to support pre-Android 4.0 devices any more.

Version 12, if all goes well including some interesting UI changes, will become available early 2019 in time for Vespuccis 10th anniversary. Back to 11.1.

Improved preset handling

The internal representation of presets has been overhauled allowing us to implement some new features without adding to already very byzantine code. The major changes for users are:

• Tri-state checkboxes: this is similar to how JOSM handle checkboxes in the preset display. Preset check fields that can have an explicit “no” or “off” value, start off in an indeterminate state and then can be toggled to on and off. A long press will return the checkbox to the indeterminate state. Previously such check fields were represented by a “radio button” like UI. Check fields that do not set an off value continue to work as is.
• Support for checkgroups. Checkgroups are groups of checkboxes, these can have a label and are handled very similar to multi-select fields, in most cases you will not even see a difference. The change allows a much more compact layout for presets that have many checkboxes, for example the payment methods preset. And caters for a general trend to tagging in such a style.

The default preset has been reworked to work with the new features. The effect of these changes is in general a more compact representation of objects with a large number of check fields, the most notorious example the payment methods preset that currently has 34 check fields. In 11.0 that looked like this:

In 11.1 it looks like:

with individual check fields selected in a separate dialog:

Readers that saw my, slightly gone wrong, talk on presets at SotM 2018 in Milan, will realize that this change plays a bit in to the observation that there is a clear trend to move tag values in to the tag key space. While I disagree with this, in the end we still need to provide mappers tools to deal with it.

Taginfo search initiated manually

Version 11.0 introduced “auto-presets”. These continue to be available but the online query needs to be explicitly started by tapping a button in the preset search result form. We’ve found that the performance of the feature varies so much across mobile networks that doing the query automatically was too confusing and annoying.

The corresponding preference to enable/disable the feature has been removed. There are some further improvements to this functionality that didn’t quite make it in to the first beta, but which will be included in the release.

Improved tag copy & paste

Tag copying and pasting has been redone. New functionality includes:

• pasting tags to selected elements in the map view (keyboard short cut <Ctrl> T)
• copying of GeoJSON Feature properties to the tag clipboard

The later functionality allows you to copy property data from a GeoJSON layer to OSM object, naturally this shouldn’t be misused).

Miscellaneous

• Keyboard shortcuts in the main screen now work again after they stopped working a while back.
• Samsung DeX support, Vespucci now reacts to changes in display density and dock insertion/removal.
• Support for free form windows has been improved.
• Lots of minor improvements.

Work has proceeded nicely on the next major release of Vespucci and while there are still some rough edges and not quite everything has been added that I intended to, it definitely makes sense to test this version. THe beta APK is available from googles play store and our github repository.

Layer control

The most obvious visual difference in version 11 is the layer control on the main map view.

Currently it is not possible to change the ordering or add more than one layer of a specific type, supported functionality:

• Hide/Show button turns drawing of the layer off/on. Hiding a layer doesn’t free any resources associated with the layer.
• Zoom to extent. Zooms and pans the screen so that the whole extent of the layer is displayed, if the extent cannot be determined this will zoom out to the full web-mercator coverage. Note: on the data layer this may not be particularly useful if you have downloaded areas that are far apart.
• Menu button.
  • Tile based layers:
    • Select imagery. Same contents as on the preference screen, if multiple layers have been used, a most-recently-used list will be displayed above this menu entry, allowing quick layer switching. Selecting the “None” entry from the list will disable the layer, and requires re-enabling it via the “+” button on the layer dialog.
    • Flush tile cache. Moved here from main menu.
    • Background properties. Set contrast of layer, moved here from main menu.
  • GeoJSON layer.
    • Change style. Show the layer styling dialog.
    • Discard. Delete the layer including any saved state.
  • GPX layer. The GPX layer is currently mainly controlled via the entries in the GPS menu.
    • Change style. Show the layer styling dialog.
  • Photo, Grid and Task layers.
    • Disable. Turn this layer off, needs to be re-enbled via preferences. For the tasks and photo layers this will free resources if the app is exited and re-started.
• ”+” button:
  • Load GeoJSON layer. Loads a GeoJSON layer from a file, any existing one will be replaced.
  • Map background. Shown if the background layer has been disabled, allows the same selection as from the preferences screen.
  • Map overlay. Shown if the overlay layer has been disabled, allows the same selection as from the preferences screen.

Support for GeoJSON layers

A single (currently) GeoJSON based layer can be displayed on the map, individual elements are selectable. On loading the GeoJSON data from a file colour, stroke width and a tag for displaying a label can be selected.

A typical use case would be to verify third party data in GeoJSON format on the ground and similar mapping activities.

Re-arranged Preferences

The two preferences screens have been re-arranged with just the most important settings on the first screen and the “Advanced preference” screen split up in to multiple sub-screens.

The Authors and Licence screen has been move to the main menu, just as the Debug screen has.

Undo/redo of individual checkpoints

Previously selecting an undo or redo checkpoint from the undo menu (long press on the undo icon), always undid/redid all checkpoints back to that point in time. In Vespucci 11 a dialog is shown that allows you to select the previous behaviour or to undo/redo just the single selected checkpoint.

While selecting just one checkpoint will result in consistent data, it may have unintended consequences as all elements changed in the checkpoint will be reinstated to the stored state when the checkpoint was created, except if elements are referenced, for example way nodes or relation members, that have been deleted in later edits. References to deleted objects will not be re-created, including relation memberships in later deleted relations.

Example: as the first operation you split a way, as the second step you delete one of the two halves of the way. Undoing the first operation will only add way nodes back that were not removed in the second step.

Simple tag changes in a checkpoint can in general be undone without negative consequences.

Review changes before upload

Entries on the list of the created, changed and deleted elements can now be selected and the actual changes inspected. Elements that have failed the validator tests will have a red icon displayed.

Auto-Presets from taginfo

The results in the preset search are now complemented by results from querying Jochen Tops taginfo service, currently this requires network connectivity to work and will only return results for tags that have at least one wiki page. In general so generated presets should not be taken as gospel and the tagging will typically need some manual work.

Selected search results from taginfo are added to a special preset file that is stored on device in the public Vespucci directory, this can be used as a starting point for creating a proper preset for the object in question.

This feature can be turned off in the Advanced preferences “Data and Editing settings”.

Miscellaneous

• Support for icons in preset combos and multi-selects.
• The info display for selected elements will now show the original state of the element side by side with the current state if the element has been modified.

The full change log is available here

Known problems

Upgrading from previous versions

• For known problems with this build please see our issue tracker

First a note, it seems as if I never blogged about the 10.1 update, information on those changes can be found in the release notes on the Vespucci website.

The beta release of 10.2 that is now available in the beta channel on the google play store, or from the releases on github does not change an awful lot that is end user visible outside of a new upload UI, however there are two core changes that I want to touch on quickly.

Support for “network” location providers

Historically Vespucci has only supported using the on-device GPS location provider, or nothing at all. That meant that you were unable to get a rough location approximation on devices that didn’t have onboard GPS, or that had GPS disabled for example to reduce power requirements. The main reason for this is that on the one hand we wanted to avoid location information potentially tainted by your devices Android provider and avoid our users position being tracked by them.

We now support using so-called “network” location providers, that is location sources that derive your position from the mobile network, WLAN and other signals your device is receiving. If you’ve enabled such providers on your phone, more on that later, Vespucci will use all available providers for centering the map display on your position and for auto-downloads, tracks will still exclusively be generated from GPS data.

The change in opinion is mainly due to less and less people caring about such matters and at least google tracking in any case (see for example https://www.theverge.com/2017/11/21/16684818/google-location-tracking-cell-tower-data-android-os-firebase-privacy), further allowing such providers enables better indoor positioning which is a clear advantage.

If Vespucci detects that network positions can at least potentially be used, it will display this icon instead of the classic GPS icon on the screen and will alert you to which provider it is currently using via toasts (the short on-screen messages).

Modern devices running a google variant of Android have three location mode setting (besides turning location services completely off):

• Device only - use only the on device GPS location information, does not require sharing your location data with google
• Battery saving - doesn’t use GPS, instead uses mobile network, WLAN and other signals to determine your location, requires sharing of your location data with google
• High accuracy - uses GPS and other signals to determine your location, requires sharing of your location data with google, this is typically only more “accurate” than Device only if receiving GPS signals is seriously impaired

Vespucci does not use the Google play servers “fused” location service and remains usable independent of if you are running it in a Google sanctioned environment or not.

Better https support

Given the push for more and more services on the Internet to be accessible only via encrypted transport (https) Android apps are faced with two challenges:

• the standard Java API for accessing http services does not support protocol level redirects, that is http to https or the other way around this is not difficult to work around but would still require codes changes at every impacted place in the code
• more and more sites are turning off TLS 1.0 support for security reasons, unluckily TLS 1.1 and 1.2 are only supported from Android 4.1 on and are only enabled by default since 4.4, again addressing this requires touching all the same code as above

In the end I decided to address these issues by migrating all the networking code to OkHttp that we’ve already been using for some things, for example for map tile retrieval since 10.1. As OkHttp exposes a different programming model and it didn’t require massive changes, it wasn’t a drop in replacement and we appreciate all feedback on the changes as some aspects of the networking code are difficult to test automatically.

Two notes:

• yes this means that users with devices running Android 4.0 and older are not able to access any services that have turned off TLS 1.0, for example you will not be able to update the imagery configuration on the fly from github.
• modern Android versions actually use OkHttp under the hood wrapped in code that emulates the standard Java API, so we are not doing anything particularly exotic.

I’ve updated, as every quarter, the statistics on our wiki.

One thing we can see on

is what looks like growth flattening out.

Splitting out the new contributors from maps.me and other sources gives us the following graph:

Which seems to show maps.me as a source of new editors declining (and other sources continuing to grow).

If we look at just the maps.me part, it becomes very clear:

The trend is not really a surprise, maps.me had a large installed base when it first announced the editing feature and that was bound to lead to a burst of people trying it out in the beginning. Even though the rate of new contributors has dropped it is still at a very respectable ~20% of the total.

The overall graph without maps.me looks like:

which continues to show a reassuring upward trend.

PS: yes we passed through 1’000’000 actual contributors to our map data on December the 14th, but explaining what that means and why there are numerous different metrics that can be considered that all differ a bit is a complicated thing.

Just in time for “X*mas we’ve started rolling out Vespucci 10 “X”-

I discussed the new C-Mode and configurable validator previously in this diary entry, but there are quite a few other changes:

Version number change

Even though Vespucci has been around for over 8 years, we used version numbers below 1 in a rather nerdy understatement fashion. I’ve been guilty of slightly weird numbering before .

The problem with this is that these days nobody understands if you are not at at least at version 3 after a couple of months and people may actually think something is wrong, so we’ve decided to do a Mozilla and jump to version 10.

While semantic versioning doesn’t really make a lot of sense for applications that don’t expose an API, we will be sticking with a major . minor . patch system for the internal numbering for now.

Support for synonyms in the preset search

The preset search will now use the same list of synonyms that the iD editor does additionally to the internal preset search index. The new functionality is independent of preset translations and uses the same fuzzy matching as the index search. Additional synonyms should be added on transifex to the iD translations.

One of the reasons I based this on existing technology, even though the current system has its warts (we should probably simply gather synonyms completely separately from our translation platform) , is that OSM development already suffers enough from rampant NIH and there is no real added value in asking translators to add synonyms to two different systems.

Support for custom tasks

You can now load (and save) tasks in a simplified Osmose JSON format. The format is not particularly forgiving and must follow the following example:

{
	"description": [
		"lat",
		"lon",
		"error_id",
		"elems",
		"subtitle",
		"title",
		"level",
		"update"
	],
	"errors": [
		[
			"47.3050383",
			"8.3702817",
			"11187837418",
			"way396965872",
			"This is a silly error of type 1",
			"Silly Errors",
			"2",
			"2017-03-26 20:30:16+02:00"
		],
		[
			"47.2930434",
			"8.3615897",
			"11187837446",
			"way397334779",
			"This is a silly error of type 2",
			"Silly Errors",
			"2",
			"2017-03-26 20:30:16+02:00"
		]
	]
}

The value for “error_id” should be unique in the file. Note the custom tasks can’t be uploaded, however you can save the open tasks to a file for later reuse.

Miscellaneous

• The icon for splitting a selected way is now a pair of scissors.
• Improved label rendering.
• If a PresetFilter is filtering just on one preset item, apply that automatically when creating new elements.
• Support i18n attribute in presets to indicate that a text field can have i18n suffixes
• Improved tile download behavior when zooming or switching layers.
• Support for object_keys and value_type attributes in presets (current value_type opening_hours, conditional, and website are supported). See the list of JOSM preset extensions for more information.
• Support for using patterns for way rendering, added styling for natural=cliff, natural=coastline and man_made=embankment.
• Improved conditional restriction editor with opening_hours editor support.

With the last release 0.9.9 running nice and stable and the monthly updates to it contain mostly updated presets and imagery configurations, it is time to give you a preview of what I’ve been working on for 0.9.10.

Vespucci has long highlighted objects that had fixme tags and streets that are missing name tags, the code for this goes back to 2010, long before that was considered award worthy.

Over the years we have added support for warnings from Osmose and OSM Notes, making it easy to spot issues that might need work. In 0.9.9 I added preliminary support for highlighting objects that haven’t been checked for a longer time.

But it is undoubtedly true that with OSM data becoming denser and denser and in many areas near complete it is non-trivial to find objects that need your attention..

Enter C-Mode

The solution to this has two components:

A mode that only shows elements that have warnings.

And re-factored validation code that adds user configurable tests for missing tags and makes the resurvey warning time fully configurable (additional tests are easy to add in the new code and more will be added before release).

The missing tag check works together with the preset system, to generate a warning an object needs to be both missing the tag, and the matching preset needs to contain it too.

This means that if, for example, you want to highlight missing “name” tags you don’t need to specify a long list of which objects really need that tag, it is enough that the preset suggests that a name tag should be added to the object.

This does have a slight downside in that the quality of the presets is key to this working well, and one of the reasons that the presets have received a lot of attention over the last couple of months (and that extends to JOSM too, see for example https://josm.openstreetmap.de/ticket/15143 ).

But the key point is that you can configure which keys you consider important and which not.

An early beta build of 0.9.10 is available https://drive.google.com/drive/folders/0B9pKLmh8s1h8bFI5bGd4VnhYWkk?usp=sharing a version signed with our release key will likely be available from google play store in one to two weeks.

[Update 2017-11-22]

Users that are receiving beta releases from the google play store should be getting the first beta release now. Release notes can be found on device and here.

A couple of weeks back I regenerated the data in the “Name Suggestion Index” from a current planet dump, adding a largish number of new entries. The index is used by iD and Vespucci to generate canonical spellings for well-known brands and to apply the correct presets at the same time.

Naturally the raw list contains a lot of nonsense, and that’s why there are two ways to reduce noise to an acceptable level: one, a list of (mis)spellings that are mapped to a canonical value https://github.com/osmlab/name-suggestion-index/blob/master/canonical.json and, two, a filter https://github.com/osmlab/name-suggestion-index/blob/master/filter.json that removes names that we don’t want, for example Bank for banks.

Previously you could drop names only globally. Now you can drop them specifically for a type of object. For example, in older versions anything with the name “Casino” was dropped. Now only casinos with the name “Casino” are. (That was the reason why, in earlier versions, the suggestions didn’t work for the French supermarket chain of that name.)

The index is not perfect, mainly because it is not country-specific (and creating such and index would be, IMHO, too much work). But, it works quite well, even given its limitations.

Now, why am I writing this: The update added a lot of names in non-Latin scripts and other new entries that need to be checked for whether or not they are actually useful. Considering that iD is used by the majority of new mappers, improving the index has a direct effect on the quality of their contributions.

You may already have it on your device and this is old news, but yesterday we pushed the release version of Vespucci 0.9.9 to the google and amazon app stores.

The release notes list most of the changes including preliminary documentation on the opening hours editor.

I would specifically like to thank Mateusz Konieczny and Holger Jeromin for beta testing (Mateusz particularly for finding a last minute issue).

0.9.9 took a long time to get to this state, mainly due to a lot of under the hood work which should make upcoming releases much easier. I’ll be trying to step up the release frequency again and expect 0.9.10 including C-mode to be available by the end of the next quarter.

out there is trying to run Vespucci 0.7.0 which dates back to April 2011 on a Xiaomi Redmi Note 4 with Android 6.0.

While the app will run, 0.7.0 was released a while before 64bits for OSM element ids became necessary (early 2013), and trying to parse current data will lead to a crash.

With other words: please upgrade to a current version!

Thank you

Vor gut einem halben Jahr habe ich einen Blogpost zur Totalrevision des VGWR verfasst, nun ist es so weit: Anfangs Juli wird die revidierte Fassung in Kraft gesetzt. Da das BFS jetzt auch den Text dazu publiziert hat (revidiertes VGWR als PDF), kann man jetzt schon einige Aussagen dazu machen:

• generell scheint sich die Verordnung in wesentlichen Punkten verbessert zu haben,
• leider scheinen die Rohdaten aus dem GWR der Zugangsklasse A immer noch nur via Einzelabfragen verfügbar zu sein und nicht als Datensatz, aber immerhin scheint es weiter keine Nutzungseinschränkungen zu geben,
• zwar bleibt der neue Auftrag an die SwissTopo Geocoding-Dienste mit den Adressdaten zu betreiben bestehen, aber ein Downloaddienst der Daten soll angeboten werden.

Offene Fragen:

• die Verordnung sieht eine Frist mehreren Jahren vor in dem der Adressdatensatz aufgebaut werden soll. Das ist doch etwas gar lange für einen Datensatz der im Prinzip schon besteht. Wenn es auch verständlich ist, dass die Validierung etwas dauert, ist das für uns kein wirklicher Zusatznutzen. Wir, und wohl auch viele andere potentielle Nutzer der Daten, könnten auch mit einem entsprechenden Extrakt aus den GWR Daten leben (den das BFS ja auch schon bisher angeboten hat). Ich sehe keinen Grund wieso der nicht schon im Juli verfügbar sein könnte, und natürlich wäre das wirklich im Sinne von Open Data, anstatt nur einen veredelten Datensatz anzubieten.
• wie immer werden leider die Details der konkreten Nutzungsbedingungen offengelassen, d.h. es ist immer noch völlig offen ob die Daten dann tatsächlich zu offenen Bedingungen erhältlich sein werden und ob wir sie dann auch in irgendeiner Form in OSM nutzen können.

PS: ab 1.1.2018 werden die Adressdaten des Kanton Zürichs zu sehr offenen Bedingungen verfügbar sein, so oder so wird also der Bestand an wirklich offenen Adressdaten für die Schweiz in absehbarer Zeit stark zunehmen.

Now despite the difficulties of garnering attention when your app doesn’t break things, Vespucci development has carried on and I’m pleased to announce that the first “official” beta release of 0.9.9 is available from googles play store and from our [github repository] (https://github.com/MarcusWolschon/osmeditor4android/releases).

While there are quite a few user visible changes, most of the work since the release of 0.9.8 has been under the hood, improving the maintainability, testing, documentation and code quality throughout the app, making it much easier to contribute to its development.

It should however be noted that Vespucci has a relatively large code base and contributing to the code (but not to the documentation, presets and other aspects of the app) remains fairly complex. See vespucci on OpenHub for more on that.

The release notes give an overview of what has changed, here just some highlights:

• an Indoor mode see wherecamp blog post
• tag and preset based filters

• way improvement handle use now follows the same pattern as iD
• javascript based scripting

And where is the elusive openinghours editor? I’ve [resumed work on it] (https://github.com/simonpoole/OpeningHoursFragment) and there is a reasonable chance that it will be included in the 0.9.9 release even though there is still a fair bit of work to do.

In November 2015 I produced a blog post on the relative sizes of our national communities and their development over time. When I updated the contributor stats on our wiki a couple of days back and looking at the massive impact the influx of first time users using maps.me have on the contributor numbers (not necessarily on anything else as I have pointed out in earlier posts), I thought it would be interesting to update the numbers and have a look at what has changed over slightly more than a year.

The most notable changes in the top 20 ranking by absolute community size are Russia overtaking France, Ukraine moving up 6 places and the Philippines moving in to the top 20 at the expense of the Czech Republic.

It is reasonable to assume that these changes are mainly due to the differences in the popularity of maps.me, No surprise and expected that it is most popular in Russia and the Ukraine, not so clear why it is so popular in the Philippines:

Numerous small changes in the contributors per capita ranking, but I suspect the fact to take away from this is that even in countries that already had the best results we are still improving the penetration levels.

In the first blog I singled out Germany and the United States as the two largest communities with very different growth patterns. Particularly in Germany there has been some concern as both the number of active contributors over short periods and the absolute number of edits have been decreasing slightly over the last two years. However new contributor influx does not show the same trend and we might put the “negative” trend simply down to the major urban areas of Germany being extremely well mapped.

As in 2015 the trend in the United States is very different, with a very large growth peak in December 2016. 3500 mappers in a month is roughly what would be required to grow at the same rate relative to population as Germany, so this should be a great achievement.

Unluckily it is not.

Closer inspection shows that a large number (in the 1000s) of the accounts created in December and in the preceding months are fake accounts created by one or more SEO companies. While it is rather unclear if the data added itself is legit, the accounts themselves are not.

Back to numbers with a more positive context, the continental distribution of mappers continues to even out, while Europe continues to dominate, both Asia and South America have increased their relative share.

The raw data for the stats is available as LibreOffice Calc files. Notes on the methodology can be found in the original blog post.

Three years back I started sending welcome messages to new mappers on behalf of SOSM, using a simple tool chain based on the RSS feed of new mappers from Pascal Neis.

The message is not customised as from the beginning it was seen as a replacement for the welcome message that had gone away with the web site redesign. The text (in 4 languages, no Romansh version though) is designed to welcome the new contributor and give pointers to the resources available nationally.

Has it worked? Difficult to say, as the whole point was a replacement of something that existed before, we didn’t expect drastic change and we didn’t get it.

And, no, I’m not stopping :-)

Two weeks back I was invited to Wherecamp Berlin, on the one hand to participate in the opening panel and on the other to have a short talk on the state of indoor mapping, more about that later.

Naturally the foreseeable impact of self-driving vehicles and machine learning in general was the, obvious, main topic of the opening panel. But outside of “things are going to change” I’m not sure that there was any other conclusion from a couple of oldish men without crystal balls.

From an OpenStreetMap point of view it was encouraging to see so many OSM based projects and operations presenting at the event mingling with essentially all the big names in geo-business.

Stefan from GraphHopper presenting at Wherecamp Berlin.

From a bit of a nerdy view, the first day presentations from Qualcomm, Broadcom and Google on what we will likely be getting in the way of GNSS support in upcoming mobile phones was quite interesting.

At the conference the first phone (the Aquaris X5 Plus) officially supporting the Galileo system was presented together with hackathon arranged around it. I didn’t get to actually testing one, so no further comment on if it actually worked and if there was any noticeable improvement. This seemed to be part of a marketing campaign that will lead up to Galileo being declared operational this December if all goes well. This probably doesn’t really mean much from a practical point of view as the number of working and commissioned satellites will still be rather low, but it probably does indicate that Galileo is slowly becoming real.

Qualcomm however did announce in their talk that all phones based on a current Snapdragon processor should support Galileo, naturally likely needing a corresponding firmware upgrade first. Slightly humorous was numerous of the presenters referring to the Samsung Galaxy S7 as an example of such a phone, even though in Europe the S7 uses a Samsung Exynos processor (which however, even though not documented, I suspect will support Galileo too).

Perhaps the most interesting talk in the series was from Broadcom on their project to develop a dual frequency consumer grade GPS chip. Dual frequency GPS chips have up to now been the domain of survey equipment manufacturers and the military. Naturally it is questionable if this technology will actually make its way in to real consumer devices, but if it does, we will get significant better GPS data without resorting to using DGPS with external reference stations.

This brings us to the last of the three GPS related topics: as you may have heard google will be supporting receiving “raw GPS data” from your mobile phones chip set (if it supports in the 1st place) from Android Nougat on. This implies that you could use your phone as a “rover” in a typical RTK setup or simply post process GPS data for better results. But this really all only makes sense if you have access to reference data and if google is going to provide something in the way of that is currently completely unclear.

Indoor

Besides inviting me to the opening panel, the organizers had twisted my arm a bit to participate in the afternoon session on indoor mapping in OSM. Background: slightly over two years back I was involved in the effort to devise a reasonably consistent and simple tagging scheme (see: https://forum.openstreetmap.org/viewtopic.php?id=25961 ) to replace the previous IndoorOSM tagging that was a bit popular at the time, but had a couple of technical problems and was in general rather complicated. The result of that work is SIT, but when we were working on the scheme we had one big problem: we really didn’t have any working code and only a small number of examples to work with and so when we finished the first version of the specification in 2014 it was not really clear if everything we had designed would work.

Floor plan elements in OSM, IndoorOSM blue vs. SIT red.

Fast forward to 2016, those of you that participated in SOTM 2016 may have seen by the talks by Cynthia Gutton. Antoine Riche and Adrien Pavie on the SNCF mapping 388 french train stations. The tools (among others iD-indoor and openlevelup) developed in parallel and for this project, brought indoor mapping large steps towards being, even if not completely mainstream, something that can be reasonably done in OSM. Antoine and Adrien repeated their SOTM talk in Berlen, and additionally we saw presentations from DB (Deutsche Bahn), Mentz, and OpenStationMap on their mapping of train stations in Germany and elsewhere.

One point that I stressed in my talk at the end of the session, is that while from a number of aspects indoor mapping is a good fit for OSM, for example because we can seamlessly integrate pedestrian and accessibility routing outdoors and indoors, if comes at the cost of extreme amounts of details that make editing essentially impossible without dedicated editor support. Adrien has already proved that this can be done in a reasonable fashion with iD-indoor, not to be outdone I threw together a proof-of-concept for Vespucci for the conference (in the mean time this has reached a quite stable status).

Berlin main station in Vespucci indoor mode.

In other News

One of the more intriguing presentations on Sunday was from the company TerraLoupe “Aerial large scale landmark detection via deep learning”. While who is actually the contractor remains in the dark, the interesting aspect was using machine learning to identify objects (for example emergency phones), text on street signs etc. along German motorways by using high resolution oblique aerial imagery.

Also of note was the presentation by Holger Dietrich on a version of the Wheelmap app with support for google Tango devices (currently the only such non-developer device is the Lenovo Phab 2 Pro), which allows surveying 3D objects, in the case for Wheelmap for example the width of an entrance and step heights.

Slides from the talks are available from the conference webssite.

Einige Leser haben vielleicht einen leicht genervten Tweet von mir gesehen als ich diese Woche entdeckt habe, dass die Vernehmlassung über die Revision des VGWR an uns vorbeigegangen ist, ohne dass wir die Möglichkeit hatten unsere Meinung dazu anzubringen.

Um was geht es?

Das Gebäude- und Wohnungsregister ist ein schweizweiter Datensatz der sämtliche bewohnte Gebäude und Wohnungen in der Schweiz umfasst. Neben Merkmalen die für OpenStreetMap nicht interessant sind, enthalten die Daten auch einen vollständigen Adressdatensatz und zum Beispiel auch die Anzahl Stockwerke für jedes Gebäude. Die GWR Daten sind zwar nach der jetzigen Rechtslage nicht direkt in OSM nutzbar aber wir haben immerhin Zugriff auf einer daraus erzeugten Strassennamensliste den wir zur Qualitätskontrolle nutzen.

Aus den GWR-Daten generierte Adressen werden von SwissTopo auch auf map.admin.ch angeboten, auf welcher Rechtsgrundlage die Publikation beruht ist allerdings völlig unklar. Dies tangiert OSM insofern, dass in der Schweiz man keinen urheberrechtlichen Schutz so publizierter Daten erwarten kann, und der Nutzer somit auf magische Art und Weise wissen müsste, dass das aktuelle VGWR eine abschliessende Aufzählung der erlaubten Nutzungen enthält und diese Daten in keiner Art und Weise für OSM genützt werden dürfen. Eine ähnliche Problematik stellt sich auch mit öffentlich zugänglichen Datensätze dessen Nutzung im Geoinformationsgesetz geregelt wird, aber immerhin muss man da nicht ganz so obskure Kenntnisse der Rechtslage haben um sich richtig zu verhalten.

Was soll sich nun ändern?

Der definitiver Text des revidierten VGWR, der Anfangs 2017 in Kraft treten soll, ist nicht verfügbar, deshalb bezieht sich diese Analyse auf den für die Vernehmlassung verteilten Entwurf. Dieser regelt viel administratives neu, zum Teil aus föderalistischer Sicht durchaus nicht unproblematisch, und erweitert das GWR um die nicht bewohnten Gebäude. Relevant für OSM sind vor allem aber die folgenden Punkte:

• Die eigentlichen GWR Daten bleiben weiterhin nicht nutzbar, da ein vollständiges Verbot der kommerziellen Nutzung besteht.
• Im Rahmen einer gleichzeitigen Revision der Verordnung über geographische Namen (GeoNV) wird die SwissTopo ein amtliches Adress- und Strassendatensatz aus den GWR Daten erstellen. Dieser Datensatz wird die Zugangsberechtigungsklasse A (öffentlicher Zugang) erhalten.
• Die SwissTopo wird verpflichtet mit den aus dem GWR abgeleiteten Adressdaten ein Geokodierungs-Dienst zu betreiben

Gibt es demnächst einen schweizweiten Adressdatensatz den wir nutzen können?

Man muss natürlich wissen, dass Verordnungen vor allem von den betroffenen Ämtern geschrieben werden und mit Sicherheit SwissTopo genau diese Bestimmungen im Text haben wollte. Problematisch ist das zwar die Berechtigungsklasse A für die Adressdaten festgeschrieben wird, dies aber nur eine notwendige Voraussetzung für die Veröffentlichung als offene Daten ist und nicht eine solche vorschreibt. Da von den aktuell 63 von SwissTopo auf opendata.swiss angebotenen Datensätze keine einzige zu offenen Bedingungen angeboten wird, kann davon ausgegangen werden, dass dies auch bei den Adressdaten nicht anders gehandhabt wird.

Der weitere ordnungspolitische Fehltritt findet sich darin, dass SwissTopo “verpflichtet” wird Geokodierungs-Dienste anzubieten. Wären die Adressdaten tatsächlich offen und frei verfügbar, könnte jeder, mit geringem Aufwand, selbst einen solchen Dienst gratis oder gegen Entgelt anbieten. Der Verdacht liegt nahe, dass die wirkliche Absicht ist das Dienstleistungsangebot der SwissTopo und ihren Marktanteil auch weiterhin zu sichern und keinen Markt für solche Dienstleistungen auf Basis der „offiziellen“ Daten entstehen zu lassen.

Was bedeutet dies für OpenStreetMap?

Aus OSM Sicht wird es vielleicht möglich sein die Adressdaten zur Qualitätssicherung zu verwenden, wie wir es bereits mit einigen kantonalen Datensätzen mit restriktiven Lizenzbedingungen machen. Wir können auch hoffen, dass die verbleibenden Kantone die solche Daten selbst sammeln, diese uns dann zu offenen Bedingungen abgeben da in der neuen Konstellation eine, auch nur potentielle, kommerzielle Verwertung völlig aussichtslos sein wird.

Da wir mit den kantonal Berner Adressdaten zusammen wohl bald 2/3 der Schweizerischen Adressen zur Verfügung haben und unsere Abdeckung laufend besser wird, sind wir auf gutem Weg den wirklich offenen Adressdatensatz für die ganze Schweiz anzubieten.

As at the begin of every new quarter since a couple of years, I’ve updated the Contributor Statistics on our wiki.

Naturally the most noticeable thing is the large jump in new users and active users starting in April 2016, which is naturally mainly due to new incoming edits from maps.me users. This is, on first principles, naturally not a bad thing, and annoyances due to bad edits aside, exposing more people to OSM is good.

What I have however noted in discussion is that the impact of the growth of the raw number of contributors is massively overestimated. For example the number of edits (not changesets) per month is essentially unchanged:

Which is not a surprise given that a maximum 0.18% of the edits per month in 2016 to date have been made with maps.me even though at least 1/4 of the active users used the app in the relevant periods. A preliminary comparison with editors that started off with iD show that over the same period the total number of edits per contributor using maps.me is smaller too.

There may be some longer term gains that we can’t see yet, at least anecdotally I’ve seen one maps.me user do an initial edit and then switch to iD, but the numbers indicate that right now that is the absolute exception.

[Update] Small clarification: the 0.18% number includes all maps.me edits not just new contributors.