OpenStreetMap

SimonPoole's diary

Recent diary entries

Vespucci 14.1 and 15.0 outlook

Posted by SimonPoole on 30 April 2020 in English (English)

Seems as if I never wrote a diary entry on 14.1 that has been available for a couple of months now. The release notes can be found as always on the Vespucci documentation site.

Obviously version 15 is quite far along at this point and major parts of the work have already been completed. In particular the migration to the AndroidX libraries and re-targeting for Android API 29 have been finished and, as a consequence, support for Android versions before 4.0 has now been completely dropped.

I’m not planning on adding major new functionality in this release, after major internal changes a shake-down period tends to be required due to device specific issues that are only exposed when the app gets actual use. Changing too much just compounds that. There are however a number of small changes (for example indicating the number of pending changes for upload on the main menu bar) and performance improvements, that will make the user experience more pleasant.

Work on the release can be followed in the Vespucci github repository.

Vespucci 14 preview

Posted by SimonPoole on 7 September 2019 in English (English)

While keeping a quarterly minor or major release schedule is a bit of an uphill battle, I believe the work on the next major version of Vespucci has progressed far enough that the first beta will be made available later this month (snapshot development builds are available now).

The fall releases always tend to suffer from porting to the next mandatory Android API/SDK version and this one is not different. I assume that this typically consumes a man-month of development work, this time around it doesn’t seem to be quite so bad however there are still a few issues I haven’t resolved that need to be fixed before release.

The bad news is that this will be the last release that will support an Android 2.3-3.0 compatible build. The reason is that to be able to continue to follow googles minimum target API/SDK rules we will have to switch to a new packaging of the Android libraries that provide backward support for older versions of Android (this implies that the estimated one man month for following google changes for next year is very optimistic). As all the package names have changed it is simply not feasible to use an old support library as we have done up to now for the legacy build with minimal addition version specific code. Undoubtedly it is a side effect of the packaging change that google is quite happy with.

My intent is to revive the legacy build for 4.0-4.3 devices when google drops support for them, which is likely not far away.

Changes in download menu

We’ve now removed the “Download at other location” as the functionality can just as well be obtained by using the find function and then downloading. The Download current view function now merges data, to replace the data in memory you will need to selec Clear and download current view.

These changes makes the behaviour more similar to JOSM and reduces some unnecessary clutter.

Automatically apply preset when property editor is invoked

This is a significant behaviour change relative to earlier versions were you had to apply the preset manually. When the property editor is invoked the best preset will be applied (without optional tags), with other words fields will be shown with empty values for such tags. Buttons to apply the preset without optional tags have been added too.

Automatically applying the preset has some dangers as, depending on the preset, existing values could be overwritten, if that happens a warning will be shown.

The feature can be turned off in the Advanced preferences.

Pan and zoom auto downloading

This release adds support for downloading data and tasks in a more conventional way by retrieving the data covered by the current screen. To keep the amount of data in memory in check a manual facility to prune the data in storage to the screen contents (plus a tolerance) is available via the layer dialog entries for the data and task layers. Additionally the data layer will try to “auto-prune” the stored data once it has reached a configurable number of Nodes in memory Advanced preferences.

The auto-download speed limit preferences will be observed both for data and tasks.

IMPORTANT: this facility should only be used with caution as it has the potential to severely tax both the OpenStreetMap APIs and your device. The former can be reduced by using an offline data source.

Internal photo viewer

The internal viewer is used be default in place of trying to start an external on device gallery or viewer app, it allows deleting and “sharing” of photos directly from Vespucci. When started via a click on a photo icon it will load references to all photos currently in view.

In the current version the photo is shown in a dialog, however we intend to integrate it in a split window view in the future.

The feature can be turned off in the Advanced preferences.

THe preset search will now search while you are typing and display the results directly in the preset display. As we integrated the name suggestion index in to the same search in 13.1, the separately available name search dialog in non-simple mode has been removed.

On-the-fly custom presets

A custom preset can be built from existing tags in the “Details” view in the property editor. Select the tags you want to include, then select “Create custom preset” from the menu and then enter a name when you are prompted. The new preset can then be found in the “Auto-preset” group.

The function tries to do the “right thing” by not including the values for tags that have “name” semantics, and setting the current value for combo and multi-select fields as the default value.

“Tip” display function

A function has been added to display tips on certain functions. Every tip will be displayed only once, optional ones can be suppressed by setting an option in the display itself, non-optional ones will always be displayed once.

Support for multi-valued but editable tags

Some tags, for example phone numbers, lane values, allow multiple values that can’t be or are difficult to edit with the previously available forms. 14.0 adds a view for phone numbers and for editable multi-select fields that will display multiple vertically arranged text fields that can be individually edited, and in the case of multi-select fields will have individual drop downs.

Automatic phone number formatting

This is an experimental feature.

When entering a new phone number the number is automatically formated correctly for the country the object is in. Further the same formating is applied when invoking the property editor, a warning will be displayed if this actually happens.

Unjoin way

Previously unjoining a way required selecting a node at where you wanted to unjoin ways and resulted in all ways except one receiving new nodes. This makes sense for example when you are trying to rewire a junction, however not so much if you want to “unglue” landuse joined to a road and similar constructs. While this wasn’t a classical use case for Vespucci, but as the app gets used more and more for nearly everything, we’ve now added support for more classical unjoin options.

There are two option when you have selected a way that has shared nodes with other ways:

  • Unjoin this will simply replace all nodes in the way with copies of the original nodes.
  • Unjoin dissimilar this will replace nodes as above, but additionally replace nodes in similar ways if necessary to maintain connections.

An example of the later function: assume you have landuse glued to a way from both sides and additionally other roads connecting to to it. If you use the basic Unjoin option the way will be disconnected form the landuse but also from all the roads connecting to it, if you use Unjoin dissimilar the connections to the roads will be maintained.

Miscellaneous

  • Selection of OAM imagery has been moved to the layer dialog and imagery configuration removed the preferences with the exception of adding custom imagery.
  • Allow Vespucci to consume intents to show photos from other apps. Note: due to the ongoing changes in the way Android controls access to files this may or not may work reasonably.
  • If you have left a changeset open after an upload, you can close it before the next upload with an option in the upload dialog.
  • Undo and redo checkpoint lists have been moved to separate tabs in the display.
  • Go back to last edit function added, this will pan and zoom to the bounding box of the top most undo checkpoint.
  • “Squaring” / “Straightening” threshold is now adjustable in the Advanced preferences. The process itself is now done asynchronously and doesn’t lock up the device anymore.
  • The preferences can now be changed from within the property editor.
  • Multi-select now support copying and cutting multiple objects to the clipboard, and pasting of multiple objects.
  • Merging a node in to another node or way will now show a context menu if there are multiple candidate target elements and offer the option of merging with all candidates.
  • Lots of stability improvements.

Vespucci 13.1 Beta Highlights

Posted by SimonPoole on 13 June 2019 in English (English)

This is a minor release mainly containing UI improvements. You can install the beta from the google play store (after enabling the “beta” releases setting), or obtain it from the Vespucci repository.

Updated opening hours editor

The opening hours editor (a separate project found here) has received a number of updates, including

  • tapping the time bars will start a large time picker, this was previously only available via the menu.
  • ability to load and save templates to on device files.
  • support for region and object type specific templates.

Use of SAF file picker for Android KitKat and later

Previously, to give a consistent look and feel over as many different Android version as possible, we were using a third party file picker/selector for those operations that require a file to be selected for input or output. Unluckily in recent Android versions removable “SD card” storage has become unavailable to the third party library which is rather annoying as this is the location where you typically want to save larger files.

To work around this we are now using the system SAF (Storage Access Framework) file picker for Android KitKat (19) and later, in older Android version the third party file picker should work fine. Using the SAF file picker does have the additional advantage that cloud-based storage location are usable for operations that don’t need direct file access, so you can save and load files from your favourite cloud storage provider if you want to.

Download offline data directly with automatic configuration

To make using offline OSM data in MSF format easier I’ve added a small UI that will display the available files from mapsplit.poole.ch, download them with the Android download manager and configure a new API entry if necessary.

Yes, I know we should be using lavish amounts of corporate newspeak to generate “excitement” among the more gullible here, but frankly I can’t be bothered. We’ve mentioned the work on complete offline support more than enough as is.

Support for multi-line comments and source in the upload dialog, and multi-line text fields in the form tag editing UI

Comment and source field are now multi-line making adding longer texts far easier. Further multi-line input fields are supported in the form based tag editor, depending on the “length” attribute in the presets.

Improvements JS support

The layout of the JS console has been improved and scripts from preset fields are now evaluated after all fields have been set, allowing for order independent consistent behaviour. Further the key - preset mapping is now visible to scripts.

GPX and GeoJson files can be loaded via Intent

Clicking a GPS or GeoJson/Json file in a file manager should now show Vespucci as an app that can read the files.

Go to coordinates function and OLC support

The “GPS” menu now features an item “Go to coordinates” that will display an input field that will accept coordinates in a wide range of formats and Open Location Codes. Further, the element information dialog will display the OLC equivalent of the coordinates for Nodes.

We do make an attempt to resolve shortened OLC codes, but this requires network connectivity and is not guaranteed to give the same results as a google based implementation or even remain stable over time.

Improvements JOSM style remote control

We now support imagery configuration and draft comment and source tags, see Controlling Vespucci from other apps.

Miscellaneous

  • Data style configuration provided that doesn’t render multi-polygons, this may help performance on slow devices.
  • Lots of stability improvements.
  • Support reading Overpass API generated non-standard OSM XML data from files.

MapSplit format OSM data files

Posted by SimonPoole on 6 May 2019 in English (English)

If you remember in Going completely offline with Vespucci last December I gave some insight in to the improved offline support in Vespucci 13.0. Now that version 13 is available, I need to loose a couple of words on how to generate the files.

tiles

The original mapsplit application was created back in 2011 by well known OSMer Peter Barth, at the time it was used to prepare input for OSM2World. The slight rework that I started late last year adds support for producing output in MBTile format (instead of copying a couple of 100’000 or so files to your mobile :-)), higher and variable maximum zoom level for the tiles and an optimisation pass.

When generated from OSM source data that has referentially complete ways, the individual tiles are referentially complete too. That means a tile will contain all nodes referenced by ways included in the tile, and all relations that have members in the tile (but not all members of those relations). If input data contains metadata fields (that is OSM id, version and date) the resulting files can be used by OSM editors just as data from the API. Naturally the data will be at least a bit stale and that needs to be taken in to account when using it in an editor.

While you can simply download the current release and generate files yourself, to make life easier for user that just want to quickly try things out, I’m generating some files for some regions on a daily basis on https://mapsplit.poole.ch/. Within reason I can add more if there is interest.

To configure MapSplit files as a data source in Vespucci, see the 13.0 release notes.

Still in the pipeline is support for updating files (best would be on device, but that will need some work).

Why we should care about googles scoped storage on Android

Posted by SimonPoole on 2 May 2019 in English (English)

Some of you may have heard of the upset around the introduction of “scoped storage” in the upcoming Android Q (10) release, see for example androidcentral

So why should you care about this change, after all it promises to enhance users privacy, and it isn’t exactly new that the goog doesn’t really care about burning developers time?*

Well there is a special angle to this new feature for “geospatial” apps, it essentially inhibits direct file access and assumes content can be accessed as a stream of bytes. That is all fine and dandy if you are writing something that just access pictures, videos and other media files, but it breaks down as soon as we are using larger structured files.

Classic example a MBTiles format file containing images or vector tiles or anything else, or a geopackage file and so on. Now there are workarounds of sorts, but they essentially all require copying around potentially multiple GBs of data, potentially duplicating it on device and are not really practical or user friendly.

A further twist is that a user will typically want to persist larger downloads over app de- and reinstalls, data in app private directories doesn’t survive this. For example vespucci creates a separate directory exactly to circumvent this issue.

See https://commonsware.com/blog/2019/03/26/death-external-storage-can-haz-file.html for some more information and https://issuetracker.google.com/issues/128591846#comment50 is a comment from a well known Mapbox developer (in case you are wondering why the later doesn’t turn up when googling, yes, the goog doesn’t actually index its own issue tracker).

Now, after protests, google has essentially pushed back enforcement of this by a year to Android R, but that is just kicking the can down the road a bit.

*In the case of paid devs, well they are paid, the others are simply expected to spend a couple of $1000 per year equivalent amount of time to accommodate whatever google has decided to break this time around,

Happy 10th anniversary Vespucci!

Posted by SimonPoole on 5 March 2019 in English (English)

Yes, 10 years ago was the first documented public commit to Vespucci

This seems to have marked public availability including the wiki a couple of days later.

10 years is methusalem age for Android apps given that the first commercial Android device became available at the end of 2008, Naturally both functionality and code base has developed a lot since the beginnings.

In any case thank you to all the developers that have contributed, lots of them long before I became involved and special thanks to Marcus for his contributions and for continuing to provide the access to the google play store.

Version 13

Back to current times, version 13 is nearly ready for a beta release that I expect in a couple of days.

Some of the highlights:

Support for multi-polygon, improved area and turn restriction rendering and improved styling configuration

Vespucci will now render multipolygons, the standard style for this and some common area types is to have a thick border on the inside of the polygons similar to other editors. In principle it is possible to use a fill style, however particularly large multipolygons will typically have some non-downloaded members which will lead to rendering artifacts.

Turn-restrictions will be indicated with an icon near the via node or way.

The style configuration has been substantially revamped to allow cascading styles (and by that requiring less typing) and matching on arbitrary tags or tag combinations, further some previously hard wired constants are now exposed for configuration, for example the maximum zoom level at which icons are displayed.

The style format currently remains undocumented outside of the source code and the configuration files themselves

Support for applying osmChange format files

osmChange format files can now be applied to loaded data. This makes the functionality to save such files much more useful, for example potential problems can be inspected and fixed in the saved files and then re-applied without having to use JOSM.

Support for MapSplit tiled OSM data

MapSplit sources contain original OSM data in PBF format tiles. These files can be configured as read-only data sources and be used for editing on the go when there is no expectation of network coverage or there is only slow Internet connectivity available. While it was previously possible to simply load larger areas directly in to memory, this has a larger impact on performance that is avoided by being able to load data on demand.

See this blog post for more information.

Support for reading PBF format OSM data files

Vespucci can now read OSM data from files in PBF format, the most popular binary format for OSM data.

Updated contributor statistics for 2018

Posted by SimonPoole on 1 January 2019 in English (English)

I’ve just updated the contributor statistics for the whole year 2018.

As I pointed out here 2017 update blog post some the graphs look a bit odd due to the one-time effect of an initial large number maps.me users contributing as the editing feature was introduced.

This effect continued to decrease during 2018, but for completeness sake here are the updated graphs from the blog post for 2017 that continue to show continuing growth even without maps.me.

I suspect that the slight decrease in unique active contributors 2018 is due to the decrease in new mappers from maps.me too, in any case the number of mappers that edited at least in one previous year continues to grow, and as that might be the most important metric overall the numbers continue to look positive.

Going completely offline with Vespucci

Posted by SimonPoole on 20 December 2018 in English (English)

Now that Vespucci 12 is already nearing release (if you are helping with the translations, please have a look at them asap, as they are the major hold up), it’s time to think of what is coming next.

Vespucci has supported unconnected mapping since day one (nearly a decade ago) and that has improved over the years with addition of support for reading and saving files in normal OSM and JOSM xml format (the JOSM format stores information on local changes).

In version 10.1, early this year, we added support for MBTiles files for local on device background imagery sources, supporting building an imagery source while you are online on your desktop and avoiding having to download to Vespucci imagery cache manually which tended to be rather tiresome.

But despite all of this, the main problem remained that you need to keep OSM data files on device reasonably small because the contents would be read in to memory and while such areas can be quite large on a modern phone, things tends do to get slow. And as we all know when you make a selection in advance, Murphy comes in to play and you are surely are going to miss exactly the area that you suddenly stumbled in to.

To get around all of that I’ve been investigating a compact, indexed in one way of the other, on device format for raw OSM data for quite a while, and now have a solution that works even better than what I originally envisioned.

The format is simply a MBTiles format sqlite database containing tiled OSM data in PBF format, nothing special. Some aspects of the generated files are not even particularly refined, for example there is currently only one tile size supported which make it a bit inefficient for empty areas. However it already works well as is, so no need to rush to improve that.

Having all of Switzerland (not exactly an empty country in OSM) will use 433MB, which compares quite well to the original PBF file it was generated from (295MB). Some of the overhead is due to having to duplicate ways crossing tile boundaries and including relations in every tile that has a member element and tiles with little content not compressing well, but overall the size increase is not big enough to cause concern.

To update the tiles it is currently necessary to re-tile the data (after either downloading a new extract or keeping the extract up to date from the diffs). This is not a big deal, generating the tiles for Switzerland from scratch only takes a couple of minutes, but naturally the holy grail is to update in situ on device, doing away with the need for a desktop completely. There are some hurdles that need to be taken before we are there, but it seems to be completely possible. Even if that turns out to be not practical there are some potentially interesting intermediate steps that I’m exploring, for example updating the tiles with the local edits (after they have been uploaded and received definitive IDs).

Some of this functionality is likely to be available in 12.1 which I expect to be available in Q1 2019.

Vespucci 12.0 BETA Highlights

Posted by SimonPoole on 9 December 2018 in English (English)

The fist beta/test version of Vespucci 12.0 is now available from the play store (together with the monthly update for the current release, 11.2.3).

Simple action mode

The current way (pre version 12 that is) of adding new objects with a long press goes back to the work done by Jan Shejbal during GSOC 2012 (a couple of years before I had anything to do with the project). This has the advantage that it doesn’t need an extra menu, you can inspect where you are placing the object before actually creating it, and in general that it is fast. The downside is that it can be a bit fiddly at times and users need to be told about it in advance and as we all know RTFM is not very popular.

Simple action mode replaces the long click action on the screen with a menu driven way of creating new objects. Long clicks are disabled as long as the mode is active.

Tapping the large green floating button will show a menu. After you’ve selected one of the items, you will be asked to tap the screen at the location where you want to create the object, pan and zoom continues to work if you need to adjust the map view.

The mode can be toggled on and off via an item in the main menu. See a comparision of the two modes for more information.,

Support for MapRoulette

MapRoulette tasks are now supported in the task overlay. While mostly MapRoulette tends to be revolve around aspects of mapping that can be done remotely, in line with the support for OSM Notes, Osmose bugs and custom tasks, it does seem to advantageous to be able to view and resolve such issues when you are actually on the ground at the location.

To be able to change a tasks status you will need to set you API key. If you have not set it, you will be asked on first upload, preferably you should set the key before making any changes via the “Tools” menu. This is a first cur at support, and may evolve over time.

Notes are movable

Newly created OSM Notes that haven’t been uploaded yet can be moved by selecting the Note and then dragging. Behaviour for downloaded Notes remains unchanged.

Most recently used support for OSM keys, values and roles

This adds support for storing, retrieving and prioritized selection display of keys, values and roles that have been recently used. When possible the mru values are associated with a preset.

Persistent storage is provided by an XML format file (“mrutags.xml”) in the publicly accessible “Vespucci” directory on device. The contents of the file can be inspected, and for example used to indicate what is missing from the presets.

In-app function to provide developer feedback

You now can, by using the “Provide feedback” function in the main menu, directly open a new issue on our github issue tracker with the most important information about your device and app version automatically included. If you have a github account you can simply login with that, if you don’t, you can submit an issue pseudo-anonymously with your OSM display name included. If you are using the later and haven’t yet authorized Vespucci with the OpenStreetMap API, you will be asked to do so.

The anonymous issue creation is only available in the official builds or if you are building the app yourself if you have configured a github personal access key during the build process.

Miscellaneous

  • more layer configuration, specifically for the tasks and grid layers is now accessible from the layers dialog.
  • using google login during the authentication process should work again.
  • alert behaviour for notification groups is now configurable.
  • improved UI and error behaviour when merging objects.
  • preset items are now alphabetically sorted by default in the displays, this can be changed in the preset itself.
  • Nominatim searches can now be limited to a bounding box.
  • in multi-select mode we now show a total element count in the header of the tag editor.
  • the build process now uses proguard which results in a roughly 20% smaller APK file.

And last, but not least …

I added https://twitter.com/sp8962/status/1070308345725239297

More moving values in to keys madness

Posted by SimonPoole on 4 November 2018 in English (English)

I just received a request to add a preset for amenity=language_school to my fork of the JOSM presets (that you should be using instead of the original :-)) Nothing to said against that, it is a reasonably popular feature with nearly a 1’000 uses.

Naturally a preset that is essentially just a stub isn’t really helpful, so as always I checked what additional properties should be added, and tagging the taught languages is an obvious attribute that is interesting.

Much to my dismay it seems as if in early 2016 it was infected with the “move values to key” plague. Instead of having two keys, lets say

language

and

language:main

containing a list of the languages, we now have more than two hundred potential keys for an in principle, simple attribute, making both data consumption, editing and creating a preset more than just difficult (as the keys can have at least three values plus unset, they can’t even be modelled with a checkbox in a preset).

At the time somebody further thought it was a good idea to retag existing language tags to the broken schema:

Vespucci 11.2 BETA Highlights

Posted by SimonPoole on 30 October 2018 in English (English)

The beta should be available on googles playstore and from out github repo in a couple of days. No fancy pictures of new features as there are none.

End of support of Android 2.3 and 3.x for builds distributed on google play store

As announced previously this version no longer supports Android versions prior to 4.0 on googles playstore due to restrictions that google is now imposing. It is however possible to either build your own APK with support for earlier Android versions and likely we will be distributing such a version via F-Droid and our github repository.

Do be able to do this we now have two build flavors:

  • current that is built against a current Android support library and supports Android 4.0 (14) and higher.
  • legacy that is built against the last support library that supported 2.3, but will naturally run on any later Android version too-

Right now the current build mainly profits from bug fixes to the support libraries, for example the issue with sub-menus on tablets is resolved in the version we build the current flavor against.

This is not a commitment to supporting 2.3 indefinitely, but as we suspect that google will be increasing the minimum Android that practically can be supported in the future we expect that the legacy build will continue as a vehicle to support older versions.

Changes for Android 2.3

On devices that only supply 32MB or less of heap to apps, memory usage of Vespucci had gotten to the point at which the app could potentially completely run out of available memory. To work around this a bit we’ve stopped loading the simplified country boundaries which saves roughly 4MB, this implies that any country specific features will not work. Further we recommend to, instead of the standard bundled preset, to load the untranslated one (see the on device Preset help page) which will save roughly another 3MB (mainly because we don’t build a translated search index then).

Miscellaneous

  • Internationalization support for OSM Notes and other tasks has been improved.
  • Note and bug status is displayed in the disambiguation menu.

Vespucci 11.1 BETA

Posted by SimonPoole on 30 August 2018 in English (English)

We released Vespucci version 11.0 at the beginning of July, 2 months later, the first beta of 11.1 is now available on googles play store and from our github repository..

Before going in to some details on 11.1, a quick word on the plans for the rest of the year. I hope that we can limit the testing period of 11.1 to the month of September and release early October.

My current plan is that 11.2 will have no new features and will solely be a vehicle to deal with googles forced update policy (see https://github.com/MarcusWolschon/osmeditor4android/issues/773 too). This also implies that there will be no update/bug fix releases of 11.1 after November 1st, so get on with testing :-) and that the google play version of the app is very unlikely to support pre-Android 4.0 devices any more.

Version 12, if all goes well including some interesting UI changes, will become available early 2019 in time for Vespuccis 10th anniversary. Back to 11.1.

Improved preset handling

The internal representation of presets has been overhauled allowing us to implement some new features without adding to already very byzantine code. The major changes for users are:

  • Tri-state checkboxes: this is similar to how JOSM handle checkboxes in the preset display. Preset check fields that can have an explicit “no” or “off” value, start off in an indeterminate state and then can be toggled to on and off. A long press will return the checkbox to the indeterminate state. Previously such check fields were represented by a “radio button” like UI. Check fields that do not set an off value continue to work as is.
  • Support for checkgroups. Checkgroups are groups of checkboxes, these can have a label and are handled very similar to multi-select fields, in most cases you will not even see a difference. The change allows a much more compact layout for presets that have many checkboxes, for example the payment methods preset. And caters for a general trend to tagging in such a style.

The default preset has been reworked to work with the new features. The effect of these changes is in general a more compact representation of objects with a large number of check fields, the most notorious example the payment methods preset that currently has 34 check fields. In 11.0 that looked like this:

In 11.1 it looks like:

with individual check fields selected in a separate dialog:

Readers that saw my, slightly gone wrong, talk on presets at SotM 2018 in Milan, will realize that this change plays a bit in to the observation that there is a clear trend to move tag values in to the tag key space. While I disagree with this, in the end we still need to provide mappers tools to deal with it.

Taginfo search initiated manually

Version 11.0 introduced “auto-presets”. These continue to be available but the online query needs to be explicitly started by tapping a button in the preset search result form. We’ve found that the performance of the feature varies so much across mobile networks that doing the query automatically was too confusing and annoying.

The corresponding preference to enable/disable the feature has been removed. There are some further improvements to this functionality that didn’t quite make it in to the first beta, but which will be included in the release.

Improved tag copy & paste

Tag copying and pasting has been redone. New functionality includes:

  • pasting tags to selected elements in the map view (keyboard short cut <Ctrl> T)
  • copying of GeoJSON Feature properties to the tag clipboard

The later functionality allows you to copy property data from a GeoJSON layer to OSM object, naturally this shouldn’t be misused).

Miscellaneous

  • Keyboard shortcuts in the main screen now work again after they stopped working a while back.
  • Samsung DeX support, Vespucci now reacts to changes in display density and dock insertion/removal.
  • Support for free form windows has been improved.
  • Lots of minor improvements.

Vespucci 11.0 BETA

Posted by SimonPoole on 14 May 2018 in English (English)

Work has proceeded nicely on the next major release of Vespucci and while there are still some rough edges and not quite everything has been added that I intended to, it definitely makes sense to test this version. THe beta APK is available from googles play store and our github repository.

Layer control

The most obvious visual difference in version 11 is the layer control on the main map view.

Currently it is not possible to change the ordering or add more than one layer of a specific type, supported functionality:

  • Hide/Show button turns drawing of the layer off/on. Hiding a layer doesn’t free any resources associated with the layer.
  • Zoom to extent. Zooms and pans the screen so that the whole extent of the layer is displayed, if the extent cannot be determined this will zoom out to the full web-mercator coverage. Note: on the data layer this may not be particularly useful if you have downloaded areas that are far apart.
  • Menu button.
    • Tile based layers:
      • Select imagery. Same contents as on the preference screen, if multiple layers have been used, a most-recently-used list will be displayed above this menu entry, allowing quick layer switching. Selecting the “None” entry from the list will disable the layer, and requires re-enabling it via the “+” button on the layer dialog.
      • Flush tile cache. Moved here from main menu.
      • Background properties. Set contrast of layer, moved here from main menu.
    • GeoJSON layer.
      • Change style. Show the layer styling dialog.
      • Discard. Delete the layer including any saved state.
    • GPX layer. The GPX layer is currently mainly controlled via the entries in the GPS menu.
      • Change style. Show the layer styling dialog.
    • Photo, Grid and Task layers.
      • Disable. Turn this layer off, needs to be re-enbled via preferences. For the tasks and photo layers this will free resources if the app is exited and re-started.
  • ”+” button:
    • Load GeoJSON layer. Loads a GeoJSON layer from a file, any existing one will be replaced.
    • Map background. Shown if the background layer has been disabled, allows the same selection as from the preferences screen.
    • Map overlay. Shown if the overlay layer has been disabled, allows the same selection as from the preferences screen.

Support for GeoJSON layers

A single (currently) GeoJSON based layer can be displayed on the map, individual elements are selectable. On loading the GeoJSON data from a file colour, stroke width and a tag for displaying a label can be selected.

A typical use case would be to verify third party data in GeoJSON format on the ground and similar mapping activities.

Re-arranged Preferences

The two preferences screens have been re-arranged with just the most important settings on the first screen and the “Advanced preference” screen split up in to multiple sub-screens.

The Authors and Licence screen has been move to the main menu, just as the Debug screen has.

Undo/redo of individual checkpoints

Previously selecting an undo or redo checkpoint from the undo menu (long press on the undo icon), always undid/redid all checkpoints back to that point in time. In Vespucci 11 a dialog is shown that allows you to select the previous behaviour or to undo/redo just the single selected checkpoint.

While selecting just one checkpoint will result in consistent data, it may have unintended consequences as all elements changed in the checkpoint will be reinstated to the stored state when the checkpoint was created, except if elements are referenced, for example way nodes or relation members, that have been deleted in later edits. References to deleted objects will not be re-created, including relation memberships in later deleted relations.

Example: as the first operation you split a way, as the second step you delete one of the two halves of the way. Undoing the first operation will only add way nodes back that were not removed in the second step.

Simple tag changes in a checkpoint can in general be undone without negative consequences.

Review changes before upload

Entries on the list of the created, changed and deleted elements can now be selected and the actual changes inspected. Elements that have failed the validator tests will have a red icon displayed.

Auto-Presets from taginfo

The results in the preset search are now complemented by results from querying Jochen Tops taginfo service, currently this requires network connectivity to work and will only return results for tags that have at least one wiki page. In general so generated presets should not be taken as gospel and the tagging will typically need some manual work.

Selected search results from taginfo are added to a special preset file that is stored on device in the public Vespucci directory, this can be used as a starting point for creating a proper preset for the object in question.

This feature can be turned off in the Advanced preferences “Data and Editing settings”.

Miscellaneous

  • Support for icons in preset combos and multi-selects.
  • The info display for selected elements will now show the original state of the element side by side with the current state if the element has been modified.

The full change log is available here

Known problems

Upgrading from previous versions

Whats in the next (10.2) Vespucci update?

Posted by SimonPoole on 18 March 2018 in English (English)

First a note, it seems as if I never blogged about the 10.1 update, information on those changes can be found in the release notes on the Vespucci website.

The beta release of 10.2 that is now available in the beta channel on the google play store, or from the releases on github does not change an awful lot that is end user visible outside of a new upload UI, however there are two core changes that I want to touch on quickly.

Support for “network” location providers

Historically Vespucci has only supported using the on-device GPS location provider, or nothing at all. That meant that you were unable to get a rough location approximation on devices that didn’t have onboard GPS, or that had GPS disabled for example to reduce power requirements. The main reason for this is that on the one hand we wanted to avoid location information potentially tainted by your devices Android provider and avoid our users position being tracked by them.

We now support using so-called “network” location providers, that is location sources that derive your position from the mobile network, WLAN and other signals your device is receiving. If you’ve enabled such providers on your phone, more on that later, Vespucci will use all available providers for centering the map display on your position and for auto-downloads, tracks will still exclusively be generated from GPS data.

The change in opinion is mainly due to less and less people caring about such matters and at least google tracking in any case (see for example https://www.theverge.com/2017/11/21/16684818/google-location-tracking-cell-tower-data-android-os-firebase-privacy), further allowing such providers enables better indoor positioning which is a clear advantage.

If Vespucci detects that network positions can at least potentially be used, it will display this icon instead of the classic GPS icon on the screen and will alert you to which provider it is currently using via toasts (the short on-screen messages).

Modern devices running a google variant of Android have three location mode setting (besides turning location services completely off):

  • Device only - use only the on device GPS location information, does not require sharing your location data with google
  • Battery saving - doesn’t use GPS, instead uses mobile network, WLAN and other signals to determine your location, requires sharing of your location data with google
  • High accuracy - uses GPS and other signals to determine your location, requires sharing of your location data with google, this is typically only more “accurate” than Device only if receiving GPS signals is seriously impaired

Vespucci does not use the Google play servers “fused” location service and remains usable independent of if you are running it in a Google sanctioned environment or not.

Better https support

Given the push for more and more services on the Internet to be accessible only via encrypted transport (https) Android apps are faced with two challenges:

  • the standard Java API for accessing http services does not support protocol level redirects, that is http to https or the other way around this is not difficult to work around but would still require codes changes at every impacted place in the code
  • more and more sites are turning off TLS 1.0 support for security reasons, unluckily TLS 1.1 and 1.2 are only supported from Android 4.1 on and are only enabled by default since 4.4, again addressing this requires touching all the same code as above

In the end I decided to address these issues by migrating all the networking code to OkHttp that we’ve already been using for some things, for example for map tile retrieval since 10.1. As OkHttp exposes a different programming model and it didn’t require massive changes, it wasn’t a drop in replacement and we appreciate all feedback on the changes as some aspects of the networking code are difficult to test automatically.

Two notes:

  • yes this means that users with devices running Android 4.0 and older are not able to access any services that have turned off TLS 1.0, for example you will not be able to update the imagery configuration on the fly from github.
  • modern Android versions actually use OkHttp under the hood wrapped in code that emulates the standard Java API, so we are not doing anything particularly exotic.

Updated contributor stats

Posted by SimonPoole on 7 January 2018 in English (English)

I’ve updated, as every quarter, the statistics on our wiki.

One thing we can see on

is what looks like growth flattening out.

Splitting out the new contributors from maps.me and other sources gives us the following graph:

Which seems to show maps.me as a source of new editors declining (and other sources continuing to grow).

If we look at just the maps.me part, it becomes very clear:

The trend is not really a surprise, maps.me had a large installed base when it first announced the editing feature and that was bound to lead to a burst of people trying it out in the beginning. Even though the rate of new contributors has dropped it is still at a very respectable ~20% of the total.

The overall graph without maps.me looks like:

which continues to show a reassuring upward trend.

PS: yes we passed through 1’000’000 actual contributors to our map data on December the 14th, but explaining what that means and why there are numerous different metrics that can be considered that all differ a bit is a complicated thing.

Announcing Vespucci "X"

Posted by SimonPoole on 19 December 2017 in English (English)

Just in time for “X*mas we’ve started rolling out Vespucci 10 “X”-

I discussed the new C-Mode and configurable validator previously in this diary entry, but there are quite a few other changes:

Version number change

Even though Vespucci has been around for over 8 years, we used version numbers below 1 in a rather nerdy understatement fashion. I’ve been guilty of slightly weird numbering before .

The problem with this is that these days nobody understands if you are not at at least at version 3 after a couple of months and people may actually think something is wrong, so we’ve decided to do a Mozilla and jump to version 10.

While semantic versioning doesn’t really make a lot of sense for applications that don’t expose an API, we will be sticking with a major . minor . patch system for the internal numbering for now.

The preset search will now use the same list of synonyms that the iD editor does additionally to the internal preset search index. The new functionality is independent of preset translations and uses the same fuzzy matching as the index search. Additional synonyms should be added on transifex to the iD translations.

One of the reasons I based this on existing technology, even though the current system has its warts (we should probably simply gather synonyms completely separately from our translation platform) , is that OSM development already suffers enough from rampant NIH and there is no real added value in asking translators to add synonyms to two different systems.

Support for custom tasks

You can now load (and save) tasks in a simplified Osmose JSON format. The format is not particularly forgiving and must follow the following example:

{
	"description": [
		"lat",
		"lon",
		"error_id",
		"elems",
		"subtitle",
		"title",
		"level",
		"update"
	],
	"errors": [
		[
			"47.3050383",
			"8.3702817",
			"11187837418",
			"way396965872",
			"This is a silly error of type 1",
			"Silly Errors",
			"2",
			"2017-03-26 20:30:16+02:00"
		],
		[
			"47.2930434",
			"8.3615897",
			"11187837446",
			"way397334779",
			"This is a silly error of type 2",
			"Silly Errors",
			"2",
			"2017-03-26 20:30:16+02:00"
		]
	]
}

The value for “error_id” should be unique in the file. Note the custom tasks can’t be uploaded, however you can save the open tasks to a file for later reuse.

Miscellaneous

Conditional restriction editor

  • The icon for splitting a selected way is now a pair of scissors.
  • Improved label rendering.
  • If a PresetFilter is filtering just on one preset item, apply that automatically when creating new elements.
  • Support i18n attribute in presets to indicate that a text field can have i18n suffixes
  • Improved tile download behavior when zooming or switching layers.
  • Support for object_keys and value_type attributes in presets (current value_type opening_hours, conditional, and website are supported). See the list of JOSM preset extensions for more information.
  • Support for using patterns for way rendering, added styling for natural=cliff, natural=coastline and man_made=embankment.
  • Improved conditional restriction editor with opening_hours editor support.

What is coming in Vespucci 0.9.10?

Posted by SimonPoole on 15 November 2017 in English (English)

With the last release 0.9.9 running nice and stable and the monthly updates to it contain mostly updated presets and imagery configurations, it is time to give you a preview of what I’ve been working on for 0.9.10.

Vespucci has long highlighted objects that had fixme tags and streets that are missing name tags, the code for this goes back to 2010, long before that was considered award worthy.

Over the years we have added support for warnings from Osmose and OSM Notes, making it easy to spot issues that might need work. In 0.9.9 I added preliminary support for highlighting objects that haven’t been checked for a longer time.

But it is undoubtedly true that with OSM data becoming denser and denser and in many areas near complete it is non-trivial to find objects that need your attention..

Enter C-Mode

The solution to this has two components:

A mode that only shows elements that have warnings.

And re-factored validation code that adds user configurable tests for missing tags and makes the resurvey warning time fully configurable (additional tests are easy to add in the new code and more will be added before release).

The missing tag check works together with the preset system, to generate a warning an object needs to be both missing the tag, and the matching preset needs to contain it too.

This means that if, for example, you want to highlight missing “name” tags you don’t need to specify a long list of which objects really need that tag, it is enough that the preset suggests that a name tag should be added to the object.

This does have a slight downside in that the quality of the presets is key to this working well, and one of the reasons that the presets have received a lot of attention over the last couple of months (and that extends to JOSM too, see for example https://josm.openstreetmap.de/ticket/15143 ).

But the key point is that you can configure which keys you consider important and which not.

An early beta build of 0.9.10 is available https://drive.google.com/drive/folders/0B9pKLmh8s1h8bFI5bGd4VnhYWkk?usp=sharing a version signed with our release key will likely be available from google play store in one to two weeks.

[Update 2017-11-22]

Users that are receiving beta releases from the google play store should be getting the first beta release now. Release notes can be found on device and here.

Update name suggestion index

Posted by SimonPoole on 17 September 2017 in English (English)

A couple of weeks back I regenerated the data in the “Name Suggestion Index” from a current planet dump, adding a largish number of new entries. The index is used by iD and Vespucci to generate canonical spellings for well-known brands and to apply the correct presets at the same time.

Naturally the raw list contains a lot of nonsense, and that’s why there are two ways to reduce noise to an acceptable level: one, a list of (mis)spellings that are mapped to a canonical value https://github.com/osmlab/name-suggestion-index/blob/master/canonical.json and, two, a filter https://github.com/osmlab/name-suggestion-index/blob/master/filter.json that removes names that we don’t want, for example Bank for banks.

Previously you could drop names only globally. Now you can drop them specifically for a type of object. For example, in older versions anything with the name “Casino” was dropped. Now only casinos with the name “Casino” are. (That was the reason why, in earlier versions, the suggestions didn’t work for the French supermarket chain of that name.)

The index is not perfect, mainly because it is not country-specific (and creating such and index would be, IMHO, too much work). But, it works quite well, even given its limitations.

Now, why am I writing this: The update added a lot of names in non-Latin scripts and other new entries that need to be checked for whether or not they are actually useful. Considering that iD is used by the majority of new mappers, improving the index has a direct effect on the quality of their contributions.

"was lange währt wird endlich gut" Vespucci 0.9.9 released

Posted by SimonPoole on 25 August 2017 in English (English)

You may already have it on your device and this is old news, but yesterday we pushed the release version of Vespucci 0.9.9 to the google and amazon app stores.

screenshot 0.9.9

The release notes list most of the changes including preliminary documentation on the opening hours editor.

I would specifically like to thank Mateusz Konieczny and Holger Jeromin for beta testing (Mateusz particularly for finding a last minute issue).

0.9.9 took a long time to get to this state, mainly due to a lot of under the hood work which should make upcoming releases much easier. I’ll be trying to step up the release frequency again and expect 0.9.10 including C-mode to be available by the end of the next quarter.

Somebody ....

Posted by SimonPoole on 12 August 2017 in English (English)

out there is trying to run Vespucci 0.7.0 which dates back to April 2011 on a Xiaomi Redmi Note 4 with Android 6.0.

While the app will run, 0.7.0 was released a while before 64bits for OSM element ids became necessary (early 2013), and trying to parse current data will lead to a crash.

With other words: please upgrade to a current version!

Thank you