Last week we released version 0.9.8 of Vespucci, see http://vespucci.io/help/en/0.9.8%20Release%20notes/ for what is new.

This was a bit earlier than expected and does not yet include the opening hours editor (but does include a form editor for conditional restrictions) because we couldn’t update the 0.9.7 version on googles play store any more.

The versions available from google and amazon have been updated, we are in contact with the person responsible for the F-DROID build but we cannot provide an ETA for release there yet.

As essentially every month (and as every month WeeklyOSM will ignore us) we’ve updated the current production (0.9.7 change log) and beta (0.9.8 change log) releases with some fixes and improvements.

Both APKs are available from github https://github.com/MarcusWolschon/osmeditor4android/releases signed with the same key as the versions distributed via googles play store so you can download and replace your existing installation without issues (you will have to, at least temporally, allow side loading).

The production version available via the play store will however, thanks to googles divine intervention, not be able to be updated till we release 0.9.8. It is however, besides github, available from Amazons app store .

Like a broken record (hmm can youngsters actually relate to that?), at the end of every quarter, I’ve updated the contributor statistics on our wiki.

As you may remember there was an unexplained big drop in December of last year, the new numbers show some recovery from that, but not to the 2nd and 3rd quarters of 2015 levels.

We’re well on the way for the next Vespucci release, a rough list of what I’m working on can be found on github. Numerous changes have already been implemented, here I just want to touch on the most interesting one for now.

Long time users may remember when the tag editing UI looked like:

The image is from version 0.7.0 which dates back to 2011/2012 (before I even knew it existed), I haven’t been able to find older screen shots, but I suspect it looked similar in 2009 in the first version. Mid last year I refactored the tag editing code to support a multi-page view and lots of other improvements, resulting in:

One of the goals of that effort was to make extending the tag editing UI easier and that has now beared fruit in allowing a simple preset-driven form based editing interface to supplement the old key-value editing:

This might sound straightforward, however there is no guaranteed one-to-one matching of presets to OSM elements which may have attributes for multiple real-world objects tagged (finding the correct preset is key to determining the correct values for each tag). I’ve taken an iterative approach to addressing this by trying to find the best match, adding all tags that belong to that preset, then adding all tags from linked presets and then repeating this for the remaining tags and so on. So far the results look good.

There are still a couple of rough edges which need to be smoothed out, but I expect a test build with this feature to be available in a couple of days.

Update: first beta with new UI https://github.com/MarcusWolschon/osmeditor4android/releases/tag/0.9.8-beta-1

As every quarter I’ve updated the contributor stats here

http://wiki.openstreetmap.org/wiki/Stats#Contributor_statistics_reports

• ~580’000 contributors total to our map data
• ~163’000 contributors in 2015

The two most newsworthy points are likely that we manged to hit exactly 9999 new contributors in November and the large drop in new contributors in December (which is supported by pnormans recently published numbers too).

Yesterday evening the release version of Vespucci 0.9.7 hit the google and amazon app stores. While not quite as fast as I intended, we did manage to get this out just 5 months after the release of 0.9.6 in July, my expectation is that we will continue to increase the frequency of releases going forward.

The most user visible difference in 0.9.7 is the reworked Notes and “Bugs” support. This can now work offline and supports, besides OSM Notes, warning and error reports from OSMOSE. One of the nicer aspects is that you can “auto-download” Notes and OSMOSE data (this naturally requires that you are online) independently of OSM data proper and with two “clicks” inspect a report, download the element in question, zoom to its position and select it:

Further the short cut to the “Tag only” editing mode has now been made official (long press on the lock icon) and we have a greatly improved documentation site. For more information see the release notes.

Happy mapping!

For a long time I’ve been wanting to produce some numbers detailing the size and growth of our national contributor communities. While a lot of things are sort of assumed to be true for example that the D-A-CH region is by far the largest community we’ve been missing some hard numbers.

Given the awful weather this weekend I at last had some time to finish off what I had started on a couple of weeks back.

So that you can play around with the numbers yourself and have a look at what interests you I’ve dumped the output in to a LibreOffice spreadsheet.

Some interesting things that I produced for myself:

The above graph shows new contributors per month for the US and Germany, it clearly shows that the adoption of OpenStreetMap in Germany was very strong early on in the projects life and has essentially continued at that level since 2008. The US on the other hand has a completely different history with the growth rates picking up substantially in 2014. It should be noted that the US is nearly three times more populous than Germany so the US is still adopting substantially slower than Germany at this point in time.

Naturally inquiring minds want to know which community is the largest, no surprise there, it is Germany at nearly double the size of the runner up. But a large population, which is one of the major drivers for the absolute size, is naturally not an achievement. So lets have a look at the contributor number per head:

Again no surprise Austria leads the pack. Note this is a bit unfair in that Vatican City actually has 4 contributors per 1000 population and a couple of the other very small states have higher numbers too, but cutting off at half a million population seems to be reasonably sensible.

Finally how are our contributors distributed over the world:

Two thirds of all contributors are in Europe, again no surprise.

Methodology and Caveats (lots of them)

The numbers are generated from geo-coded (country level) changesets and are current as of September 2015. For each changeset the centroid was determined and that compared to a set of OSM derived country boundaries. Of the roughly 35 million changesets 2 million didn’t return a result, aka the centroid was not located in a country (likely over water). The first non-HOT or Missing Maps changeset was used to determine which country a mapper belongs to. Filtering out the identifiable HOT and Missing Maps changesets reduced the overall count of contributors by 8000.

• using the location of the first changeset is naturally a very rough indicator of where a mapper is located and further it naturally doesn’t take migration in any form in to account.
• along borders using the bounding box centroid will potentially lead to miscounts
• very large bounding boxes will cause miscounts
• 6000 accounts couldn’t be associated with a country at all (aka the initial changeset was in the 2 million that couldn’t be geo-referenced)
• prior to the use of hashtags by the HOT community it wasn’t possible to filter out such changesets, as a result the contributor number in targets of HOT activations prior to 2013 should be taken with large grains of salt
• country borders tend to be a bit volatile and you can argue a lot about which continent certain countries belong to.

UID 133:

http://www.openstreetmap.org/user/TimCorio/history#map=13/42.9408/-77.6196

It is a truism that software has bugs and while as a developer it would be nice to say that they are all somebodies else’s fault, but we are probably are all just as bad as each other with respect to slip-ups. In the case of Vespucci the additional complication is that we are dealing with at least 26 different Android versions and 100s of different devices, each with its own manufacturer tweaks to hard- and software.

So unluckily now and then you might experience a crash. I do have to say outside of provoked crashes and early dev builds I haven’t had one for ages, but then I use a relative mainstream device for mapping and don’t try to edit gigantic areas, The important thing is to either get the issue fixed, or at least find out how to avoid it in the future.

Those readers that have had the unpleasant experience know that post-crash you will be offered the chance to submit a crash report. We do this with ACRA http://www.acra.ch/ and store the reports on a private acralyser server. ACRA gives us a lot of information on the HW and SW configuration of the device, a short excerpt of the system logfile and a stacktrace indicating what the immediate cause of the crash was and where in the Vespucci or Android code it happened. It does not store any personal information of the user in question, in principle we could ask for an e-mail address, but I would prefer not to for data protection reasons. Note on the side: there are some situations in which we’ll produce a stack trace just to document an unusual situation, if you don’t get an additional warning in general there is no reason to be concerned.

Now if you experienced a real crash (real: as in Vespucci restarted or stopped completely) please submit the crash report, but further please check our issue tracker and our twitter account. If your problem doesn’t seem to be known or not already fixed, please open a new issue pointing out that you submitted a report, this is the only way we can get in direct contact with you. Complaining on googles play store or continuously sending the same crash (don’t forget it might be related to your device or your editing habits) wont help.

Happy mapping!

No, this is not news. Vespucci has had a tag-only mode since its original creation in 2009, however since the old very modal user interface hasn’t been the default UI after the work by Jan Schejbal in 2012 and the start of the 0.9 releases, it hasn’t really been easily accessible.

Starting with 0.9.6 tag editing only mode can be switched on with a long press on the lock icon.

vespucci can still be locked/unlocked with a normal single touch on the lock and a long touch will get you back to normal editing mode.

Now while I personally don’t quite see the utility of such a mode, users have asked for it and it is the only remains of the old modes that will survive in 0.9.7. In any case it is light years better than the many apps of different kinds that claim to offer simple editing but in general are survey apps on steroids that mess up existing data.

I’ll be missing this weeks hack weekend in London with the topic of “mobile development”, unluckily it turned out to be on the one weekend this year on which I really coudn’t find a way to attend. In any case I wanted to at least give an update on how far I’ve progressed with the work on vespucci 0.9.7 which is scheduled for release early in Q4.

From a development perspective I try to get stuff that involves major internal changes done as early as possible in the ramp up to a release, for the obvious reason that we then have enough time to find issues in completely new code.

For 0.9.7 the big refactoring has been in the Notes code which in prior versions has been online only (you needed to have a working network connection and any changes to the Notes had to be committed immediately). I’ve reworked the whole subsystem to work more like the way vespucci handles OSM data: you can download Notes for an area manually or using a auto-download setting, work on them, save them locally for uploading at home or upload them immediately. And on top of that the same mechanisms support Osmose bug reports.

A further long time enhancement suggestions by our users was to support translated presets, for both technical and licence reasons we can’t fall back on the work that has already been done for JOSM (vespucci uses JOSM format presets) and I ended up doing it rather different way and using the GNU gettext format for holding the translated strings (there is a long and boring story why the Android i8n system can’t be used). The other preset related thing requested was a search function, which with the translated presets will work with those languages too.

As you can see the search seems to be a bit too fuzzy right now and will need some further tuning (some of the fuzziness is due to searching both in the original preset names and translated ones).

Back to the hack weekend and more interesting stuff. The current build of 0.9.7 contains substantially better support for generating notifications (on your phone and on any attached android wear devices, see my last diary entry) and includes support for Osmose bugs. This is fine where I live but in high error regions will potentially swamp you with notifications, some of this can be avoided by filtering the Osmose bugs, but some kind of distance vs. seriousness filter would likely work best.

The other hot topic is voice support: the current build has an experimental “pure voice” interface that IMHO doesn’t really make a lot of sense and goes against the basic philosophy of vespucci that you should nearly never need to fix data after entering it on the road. What seems to work far better is a hybrid approach that I’ve built: mark the position where you want the new object (as always in vespucci with a long press) and then use a voice command to actually select what you want to add.

Long press:

Press the microphone:

“Mcdonalds”

Result:

This works better than expected, particularly for names that are in the name suggestion index. Naturally likely one reason this works well is because googles voice recognition works better with well known name and terms.

If you want to play around with the current build (and please: feedback is welcome), as always it is located on google drive

Now that 0.9.6 is out, my focus is, naturally, on the next version. Hopefully 0.9.7 wont take quite as long to be ready, I’m fairly optimistic that we will be able to hit the planned release time frame of end of September. The larger operations on the guts of the app have already happened and should have enough time to stabilize till then.

If you are interested in what is planned see https://github.com/MarcusWolschon/osmeditor4android/issues?q=is%3Aopen+is%3Aissue+milestone%3A0.9.7 Two of the items have already progressed pretty far: putting the code in place for translating the presets and the integration of notification support.

The later was mainly written as a proof of concept early this year at the Karlsruhe hack weekend but needed some work to be really useful. Some images from Karlsruhe:

Alert on a watch (naturally works on phone too) from an error detected in the nearby OSM data

the same for a Note

and navigating to the location with OSMand

A test build with the inital (still needs a bit or work for production) alert support is available here: https://drive.google.com/folderview?id=0B9pKLmh8s1h8bFI5bGd4VnhYWkk&usp=sharing . You will need to turn on alert support in the advanced preferences and have “auto download” enabled in the GPS menu (if auto-download doesn’t initially work try toggling the setting).

While I have your attention: one of the work items is redoing the Notes code to work more like the rest of Vespucci and allow offline use. Are there any specific wishes for how it should behave (I will likely leave a way to immediately save and close a note in the app, but the normal mode will be a bulk upload at the end of your surveying/editing session). If you have some suggestions please add them to: https://github.com/MarcusWolschon/osmeditor4android/issues/228

In case you didn’t notice we have pushed 0.9.6 to the regular google play store. Release notes can be found on device and in the repository. There is likely to be a small maintenance release soon that will address a handful of minor issues.

If you do experience an issue or even a crash, please submit the crash dump if any and check our issue tracker for known issues and if appropriate open a new one. Vespucci supports a large number of different devices over a wide range of Android versions (2..2 to 5.1.1) and the issue you are expierencing maybe not be repeatable on other devices or circumstances. A post on the google play store does not get problems resolved.

In all the noise about MapBox’s Series B offering and their successful bid to replace MapQuest’s in house map rendering capability, it seems that our dear trade rags missed something.

Likely the most important medium term aspect of the successful bid is that it removed funding and support for a competing vector tile rendering stack that MQ was developing internally.

Anybody that has been following the developments knows that while open source and in principle freely available, the MapBox vector tile stack doesn’t work “out of the box” in any reasonable meaning of the words. It follows the trend of the bits and pieces of MapBox’s technology becoming increasingly more difficult to use in practice by the community. The other well known example is MapBox studio, the follow up to the widely independently used TileMill.

A recent article actually points to parts that are closed source, a not completly unexpected change of direction.

Now I think we all realize that it is just a matter of time till the open source community catches up on the vector tile front. This will address some of the issues the OSM community has been having with its map rendering and even the playing field a bit. But MapBox has clearly bought themselves some more breathing space for now.

As an experiment we have made the current release candidate available via the google play store beta program. This allows you to replace the release version from the play store with a pre-release version and back again.

To participate you need to:

• be signed up for our discussion group https://groups.google.com/forum/#!forum/osmeditor4android
• visit https://play.google.com/apps/testing/de.blau.android (at least for me this has only worked with a browser on a mobile device) and join the beta program
• you should then be able to access the beta version via the normal play store app

The underlying reason for trying this out is that we currently do not get enough feedback from the testing period and this should make it easier for users that don’t want to use alternative ways of installing apps to participate.

Simon

A release candidate of 0.9.6 is available https://drive.google.com/folderview?id=0B9pKLmh8s1h8bFI5bGd4VnhYWkk&usp=sharing Note the “signed” APK is signed using the same key as used for the play store and should not need an uninstall if you are replacing a 0.9.5 install.

The only change that is forseen for release, outside off resolving any last minute issues, is inlcuding the release notes.

Simon

The last new version of Vespucci with major changes was at the end of 2014 and it has taken a lot more time than I wanted to get near to completing the next version. But now a lot of things are coming together and I’m expecting that we are only a single digit number of weeks away from a stable release.

Just some of the major features:

• multi-select mode
• restyled tag editing with split pane layout on tablets.
• selective tag copy, cut and paste.
• lots of relation editing improvements
• start a camera app directly
• go directly to preset screen when creating new elements.

Besides the multi-select support most changes have been geared towards making the user interface more consistent and easier to use.

Given that the new property editor is so much better, I’ve added a way to quickly and easily switch in to the “tag editing-only” mode that has been hidden in the deprecated modes preference for the last two years. This should not be confused with the current raft of “lets add a node regardless of what is already there” pseudo editors, all available OSM objects can be modified, just the geometry is locked.

In other news Vespucci now has a twitter account and we have moved the source code repository and issue tracking to github.

For developers: one of the major hills that had to be climbed was refactoring the old “TagEditor” activity, likely some of the oldest code in Vespucci. Written by the original Vespucci author in the first months of 2009, by both OSM and Android standards it was pre-historic. I added a significant amount of code two years ago for relation support, but hadn’t touched the general structure and monolithic mass of code that it had grown to, and that had started to be a big road-block to improving and adding functionality. One immediate benefit of the refactoring is that Vespucci is much improved on tablets and uses the available screen real estate a lot better.

Watch this space or twitter for the announcement of release candidates, as always any help in testing and coding is welcome.

I spent some time today producing some numbers to show how large the effect of old accounts returning and starting to edit actually is and determined the annual numbers of new mappers that created their account at least two years back. For 2014 this would be accounts created in 2011 and earlier.

Year  New mappers
      with old accounts
2012       1'817
2013       2'241
2014       2'848

To give some perspective: 2014 missing maps and HOT together attracted roughly 1’500 new mappers. The total number of new contributors in 2014 was 105’612..

Thanks to TomH for providing some of the data required for the stats.

People who have read Gary Gales blog post on Geohipster will have noticed that one of his claims is that OSM is “business unfriendly”. It is a reoccurring theme in discussion with people from the geo-industry however in many many discussions and contacts with companies outside of geo** it has never been an issue, and so the question should probably be reformulated as:

Is OSM geo-business unfriendly?

Well, my answer is, you expected this: no.

It is obvious simply by observing the many thriving businesses that would not exist without OSM and the way OSMs “business-model” is structured.

By positioning itself as a data collection project OSM has left lots of space to build businesses using OSM data and providing services on top of it. This is in stark contrast to say Wikipedia, which has always positioned itself as the one-stop shop for WP content and services.

Would a MapBox exist if OSM had chosen a more Wikipedia like model? Naturally not. Would MapBox cease to exist if OSM changed its mind today? Probably not, given that they have moved away from being a one-trick pony, but it would be the death knell for a number of other players.

But no fear, a further reason that OSM is extremely business friendly is that we have held a steady course over the decade the project has existed. Major changes have taken place over a long period of time with lots of time to adapt.

Now there is a certain slow feature creep with respect to services provided on the central OSM site which will continue to raise the bar of the minimal functionality for a viable online map portal, but anybody endangered by this should likely rethink their business model in any case.

Community run and developed software and services will likely have more impact. OSM provides a level playing field and your business model should take competing with non-commercial services in to account (note how OSMand in continuously improving).

On top of the above OSM is extremely cheap for business. Not going down the Wikipedia route has enabled OSM to produce all this good stuff with a minimum of fixed costs. The annual budget of the OpenStreetMap Foundation (the formal body behind OSM) is roughly a 1/500th of its Wikipedia counterpart, the Wikimedia Foundation. There is no obligation for a business to donate to the OSMF and a major part of its income has been from donations by individuals.

A further common complaint from the geo-business pundits is that the OSM community is business hostile, however occurrences of this can essentially always be traced back to the company in question trying to force something on the community, or trying to telling the community what to do instead of being part of it.

Matter of fact the OSM community has an extremely laissez faire attitude towards business involvement in OSM. It is difficult to find somebody opposed to building businesses on the volunteer work, and the boards of two of the most influential OSM related organisations (OpenStreetMap US and the OSMF) are dominated by industry representatives. Dominated as in: a single token non-industry involved member in each.

Garys superficial main beef with OSM is however the current distribution licence, the ODbL.

Over many decades essentially all businesses dealing with geo-data have had business models where they would obtain data from various sources, add in some self-surveyed information and sell the result either directly or services built on top of the resulting dataset., negotiating contracts at every step in the process. This goes for essentially every player from Tomtom, Nokia and google at the top, down to smaller players, excluding essentially only the national and regional government operated GIS departments. Doing business this way is deeply ingrained in the thought patterns and culture of the whole industry.

The advent of open data, mainly open government data, has not changed this. What it has changed is that the cost structures of the businesses have improved. You shouldn’t be fooled by the marketing, many of the “disruptive” geo-businesses are simply using the tired, age old business model with lower costs. The benefit is that they seem to have a bit more lee way to do cool things right now, but that will change when everybody has caught up.

Enter OSM.

OSM is the odd kid on the block- Now I don’t want to dive in to yet another licence discussion. As has been pointed out many times, many of the issues with the licence are make believe, the real main issue the geo-industry has with OSM is that we don’t conform to their traditional business model and they are having problems to adapt. You can’t simply haggle a contract with OSM, everybody gets the same.

With other words: their problem is that OSM is truly disruptive and different.

Is this “business-unfriendly”? No, it opens up lots of opportunities for geo-businesses that are willing to adapt, for the others: its capitalism, bye bye.

** it should be noted that non-geo businesses don’t seem to have problems googling for the OSMF and sending mail or e-mail with inquiries and questions. The geo-industry on the other hand seems to be so ripe with google-challenged people that it was possible for MapBox to fill a full “legal” marketing piece with complaints by them.

It is time that we lay the geocoding related licence discussion to rest by forming consensus on a guideline.

It is well known that I support the concept that the results of bulk geocoding form a derived database and support the corresponding conclusions on the Geocoding Guideline page .

However Example 7 glosses over a point that has been raised for example by Steve Coast in the past: are failed geocoding results really free of OSM intellectual property? For clarity: we are not discussing on the fly gecoding as there is no database created and nothing to share.

We need to resolve this to move forward on the matter.

I don’t believe there is a clear and conclusive answer to the above and there is a certain danger of getting in to “how many angels can dance on the head of a pin” type of discussions, so I believe that it boils down to: with what is the OSM community happy? Naturally with the backdrop of the ODbL in mind.

I suggest something very simple: that the set of failed addresses (or more general: input data) should be shared with the OSM community. I am not saying that the failed addresses are subject to the ODbL SA clauses, just that we should treat them as if they are.

Now you might ask why would we be interested in failed addresses? On the one hand these can be mined, just as the successfully geocoded ones, for additional information, for example for house number -> post codes relationships and on the other hand the list of failed addresses is obviously helpful for quality assurance.

And I believe that this, particularly the later point, creates a win-win situation for the organisation doing the geocoding and for OSM. The win for the geocoding organisation is that more of its addresses will be found in OSM and the reliance on third party datasets will be reduced.

Now assuming that a consensus forms around the above, there is still a slightly touchy issue in that companies may not want to be identified as the source of specific addresses. To resolve this I propose providing a facility by which such input datasets can be provided to the community and published anonymously (there is at least one system in existence that could simply be cloned to provide this facility).

Note: all of the above only applies to datasets that are being publicly used so there can’t be an expectation of a high level of data privacy to start with.