OpenStreetMap

SimonPoole's diary

Recent diary entries

Post scriptum to "long time - no edit"

Posted by SimonPoole on 7 May 2015 in English (English)

I spent some time today producing some numbers to show how large the effect of old accounts returning and starting to edit actually is and determined the annual numbers of new mappers that created their account at least two years back. For 2014 this would be accounts created in 2011 and earlier.

Year  New mappers
      with old accounts
2012       1'817
2013       2'241
2014       2'848

To give some perspective: 2014 missing maps and HOT together attracted roughly 1'500 new mappers. The total number of new contributors in 2014 was 105'612..

Thanks to TomH for providing some of the data required for the stats.

Is OSM business unfriendly?

Posted by SimonPoole on 30 April 2015 in English (English)

People who have read Gary Gales blog post on Geohipster will have noticed that one of his claims is that OSM is "business unfriendly". It is a reoccurring theme in discussion with people from the geo-industry however in many many discussions and contacts with companies outside of geo** it has never been an issue, and so the question should probably be reformulated as:

Is OSM geo-business unfriendly?

Well, my answer is, you expected this: no.

It is obvious simply by observing the many thriving businesses that would not exist without OSM and the way OSMs "business-model" is structured.

By positioning itself as a data collection project OSM has left lots of space to build businesses using OSM data and providing services on top of it. This is in stark contrast to say Wikipedia, which has always positioned itself as the one-stop shop for WP content and services.

Would a MapBox exist if OSM had chosen a more Wikipedia like model? Naturally not. Would MapBox cease to exist if OSM changed its mind today? Probably not, given that they have moved away from being a one-trick pony, but it would be the death knell for a number of other players.

But no fear, a further reason that OSM is extremely business friendly is that we have held a steady course over the decade the project has existed. Major changes have taken place over a long period of time with lots of time to adapt.

Now there is a certain slow feature creep with respect to services provided on the central OSM site which will continue to raise the bar of the minimal functionality for a viable online map portal, but anybody endangered by this should likely rethink their business model in any case.

Community run and developed software and services will likely have more impact. OSM provides a level playing field and your business model should take competing with non-commercial services in to account (note how OSmand in continuously improving).

On top of the above OSM is extremely cheap for business. Not going down the Wikipedia route has enabled OSM to produce all this good stuff with a minimum of fixed costs. The annual budget of the OpenStreetMap Foundation (the formal body behind OSM) is roughly a 1/500th of its Wikipedia counterpart, the Wikimedia Foundation. There is no obligation for a business to donate to the OSMF and a major part of its income has been from donations by individuals.

A further common complaint from the geo-business pundits is that the OSM community is business hostile, however occurrences of this can essentially always be traced back to the company in question trying to force something on the community, or trying to telling the community what to do instead of being part of it.

Matter of fact the OSM community has an extremely laissez faire attitude towards business involvement in OSM. It is difficult to find somebody opposed to building businesses on the volunteer work, and the boards of two of the most influential OSM related organisations (OpenStreetMap US and the OSMF) are dominated by industry representatives. Dominated as in: a single token non-industry involved member in each.

Garys superficial main beef with OSM is however the current distribution licence, the ODbL.

Over many decades essentially all businesses dealing with geo-data have had business models where they would obtain data from various sources, add in some self-surveyed information and sell the result either directly or services built on top of the resulting dataset., negotiating contracts at every step in the process. This goes for essentially every player from Tomtom, Nokia and google at the top, down to smaller players, excluding essentially only the national and regional government operated GIS departments. Doing business this way is deeply ingrained in the thought patterns and culture of the whole industry.

The advent of open data, mainly open government data, has not changed this. What it has changed is that the cost structures of the businesses have improved. You shouldn't be fooled by the marketing, many of the “disruptive” geo-businesses are simply using the tired, age old business model with lower costs. The benefit is that they seem to have a bit more lee way to do cool things right now, but that will change when everybody has caught up.

Enter OSM.

OSM is the odd kid on the block- Now I don't want to dive in to yet another licence discussion. As has been pointed out many times, many of the issues with the licence are make believe, the real main issue the geo-industry has with OSM is that we don't conform to their traditional business model and they are having problems to adapt. You can't simply haggle a contract with OSM, everybody gets the same.

With other words: their problem is that OSM is truly disruptive and different.

Is this “business-unfriendly”? No, it opens up lots of opportunities for geo-businesses that are willing to adapt, for the others: its capitalism, bye bye.



** it should be noted that non-geo businesses don't seem to have problems googling for the OSMF and sending mail or e-mail with inquiries and questions. The geo-industry on the other hand seems to be so ripe with google-challenged people that it was possible for MapBox to fill a full “legal” marketing piece with complaints by them.

The Failover Issue and Publishing Derived Datasets

Posted by SimonPoole on 28 April 2015 in English (English)

It is time that we lay the geocoding related licence discussion to rest by forming consensus on a guideline.

It is well known that I support the concept that the results of bulk geocoding form a derived database and support the corresponding conclusions on the Geocoding Guideline page .

However Example 7 glosses over a point that has been raised for example by Steve Coast in the past: are failed geocoding results really free of OSM intellectual property? For clarity: we are not discussing on the fly gecoding as there is no database created and nothing to share.

We need to resolve this to move forward on the matter.

I don't believe there is a clear and conclusive answer to the above and there is a certain danger of getting in to "how many angels can dance on the head of a pin" type of discussions, so I believe that it boils down to: with what is the OSM community happy? Naturally with the backdrop of the ODbL in mind.

I suggest something very simple: that the set of failed addresses (or more general: input data) should be shared with the OSM community. I am not saying that the failed addresses are subject to the ODbL SA clauses, just that we should treat them as if they are.

Now you might ask why would we be interested in failed addresses? On the one hand these can be mined, just as the successfully geocoded ones, for additional information, for example for house number -> post codes relationships and on the other hand the list of failed addresses is obviously helpful for quality assurance.

And I believe that this, particularly the later point, creates a win-win situation for the organisation doing the geocoding and for OSM. The win for the geocoding organisation is that more of its addresses will be found in OSM and the reliance on third party datasets will be reduced.

Now assuming that a consensus forms around the above, there is still a slightly touchy issue in that companies may not want to be identified as the source of specific addresses. To resolve this I propose providing a facility by which such input datasets can be provided to the community and published anonymously (there is at least one system in existence that could simply be cloned to provide this facility).

Note: all of the above only applies to datasets that are being publicly used so there can't be an expectation of a high level of data privacy to start with.

Searching (in) OSM on a PI2

Posted by SimonPoole on 28 April 2015 in English (English)

Two days back there was a longish discussion on the OSM IRC channel about supporting OSM on XSCE, aka on offline, potentially slow devices. Currently it seems if they (XSCE) distribute a pre-rendered set of tiles, and during the discussion (which was mainly about alternatives to distributing tiles) it was mentioned that it would be nice if they, besides a slippy map, could provide a search function.

Now given that disk space idoes not seem to be an issue in the project and keeping in sync with OSM central is not a requirement, it occured to me that Photon might be a viable way of providing a global search function.

Installing it on my PI2 (running Ubuntu 14.04) was surprisingly easy and the only painful part was downloading 30GB of search index over my not particularly fast Internet connection.

Photon on a PI2

Now perfomance is not great, something like 7 to 10 seconds for a query, ruling out using "search as you type", but bearable for individual queries. Potentially single language search indices would be faster, but that would needed to be tested.

Naturally alternative approaches for example using mapsforge might be better and would get around the requirement for prerendered tiles, however it is not clear if a global map would actually be feasible.

long time - no edit

Posted by SimonPoole on 28 April 2015 in English (English)

Anybody who has any interest in the growth of OpenStreetMap has probably read at least one paper or blog that has moaned about "only" a couple of 100'000 users actually having contributed out of the 2 million + that have signed up for an account.

The over 500'000 contributors are a good 25% of the total registered accounts and I'm not sure if that really counts as "only" given that we don't really have any comparative numbers, I would suspect it is actually very good.

In any case there have been calls to simply delete the "inactive" accounts as they inflate the numbers and in general do no good. I'm very much against that for two reasons: on the one hand we don't know why inactive members have joined, maybe they simply wanted to show support, maybe they wanted an OSM account for autentication in uMap or any of many other possible reasons. On the other hand they are a reservoir of new mappers, and every day we likely have dozens of old accounts starting to map.

As an holdover from the licence change I'm still running a script that produces a daily list of old accounts that have newly accepted the contributor terms and typically there are a dozen or so each day. These accounts have not been active since at least April 2011, when we had roughly 350'000 accounts total. The majority tends to be accounts that hadn't previously edited but there are always a couple that somehow didn't get the message during the licence change and had actually edited more than 4 years ago.

In any case the tl;dr version: deleting the 1.5 million inactive accounts would deny us the pleasure and fun of welcoming new contributors like https://www.openstreetmap.org/user/Geocurioius to the active mappers.

Addresses Revisited

Posted by SimonPoole on 11 April 2015 in English (English)

Two years ago I produced some statistics on addresses in OSM. While I did regularly re-run the scripts, I never really revisted the topic to see how things are developing,

Global Address Counts as of 2015-04-09

                2013-03-27     2015-04-09    Increase since March 2013
Total           20'168'470     51'903'310            257%
Germany          3'659'043      8'470'716            232%
USA              2'090'893      5'229'243            250%

The complete set of numbers can be found here, According to official statistics there are 18.5 million residential buildings in Germany so I would expect that we are roughly 1/3 to 1/2 of the way to complete coverage there.

The large global increase in early 2014 is mainly due to the import of 8.7 million addresses in the Netherlands which due to peculiarities in building and numbering in NL led to a far larger increase in the counts than you would normally expect of a country with a similar population size.

When I first produced the address stats the openaddresses.io project didn't exist and it it is now interesting to compare the two projects. It should be noted at this point that openaddresses.io does not make any representations wrt licences of their individual datasets and there is no guarantee that all or any of the data could actually be used in OSM.

OA currently claims to contain 115 million address, slightly more than twice the 52 million in OSM. However how large is the overlap? A very rough (and unfair) estimate based on the numbers for the US, ES, NL, PL, DK, NO and JP in OSM, would indicate that 24 million address likely turn up in both projects, but, more interesting that OSM contains 28 million addresses not in OA. Or put differently a combined dataset would contain 143 million addresses (see caveat above).

A mad dash to Haarlem and back

Posted by SimonPoole on 18 March 2015 in English (English)

SOSM operates a fair number of services mainly for the Swiss community that up to now have been hosted on a single small server leased from Hetzner. While this is a very cost worthy solution, it doesn't have any redundancy built in and further requires cross border Internet traffic to access, which raises privacy concerns at least with parts of our community. SInce we started operating the server a bit over a year ago it was clear that a local solution would be preferable.

Due to the good relations between SOSM and Wikimedia Switzerland late last year we were pointed to the fact that Wikimedia Germany was throwing away their old hardware which used to host the "toolserver" and that while most of it was clearly not worth continuing to use, there was three couple of years old Dell R520 servers that might still be useful. While we needed to find an affordable hosting location to go with the machines (more on that soon), it was clear, if at all possible, we would like to secure the machines for SOSM.

Wikimedia operates a larger site in the Netherlands in a commercial hosting facility just outside of Amsterdam in Haarlem (yes Haarlem and nearby Breuklen gave the names to the Harlem and Brooklyn in NYC). They don't have direct on site staff and it was clear that we would have to arrange pickup when somebody was there dismantling the hardware. This turned out to be bit of a challenge and in the end, after determining that we wouldn't have to commercially import the hardware (no financial difference, just a procedural one), we decided that the easiest and most efficient, perhaps not most ecological, solution would be simply to drive up to Amsterdam short term when Wikimedia was ready and pick up the servers in person.

Last Friday was the day, Michael (user datendelphin) and myself started on our mad dash to Haarlem and back, a total of over 1'700km. To at least get some more out of it we decided to record as much as possible of the drive with Mapillary and naturally navigate with OSM only.

An old S2 for Mapillary with two empty 32 GB and OsmAnd for navigation.

I initially tried to run RTKGPS+ on the S2 with an external GPS device with the antenna beneath the sun roof (GPS signal reception is really lousy in that car), which however proved to be a bit much for the phone together with Mapillary and from Freiburg im Breisgau on we switched to using a normal bluetooth connected GPS device.

The set-up worked very well fillng a 32GB card for each direction, our longest continuous segment had something over 6200 images. Currently there are still roughly 8000 images from the drive back uploading (of a total of roughly 23'000), in any case we did cover a lot of new territory.

Routing with OsmAnd was quite painless too, and while it did take a couple of minutes to calculate the 850+ km routes that didn't really cause an issue. Very noticeable was the quite new support of destination, lanes and turn:lanes, lots of the intersections and on/off ramps were already completely tagged and only a handful had some errors. Matter of fact the only serious routing error we had along the way was due to a typo

Here exactly the same issue in OSRM:

which was caused by a short stretch of road with a wrong speed limit. OsmAnd does have an annoying tendency to try and short cut over off/on-ramps now and then, this may however be indicative of simply not penalizing these enough compared to continuing straight on. In summary the actual driving was uneventful and quite relaxing.

In the end, as we have already blogged about on sosm.ch, we picked up the servers (at 23:00 on Friday!) and had a quick and uneventful trip back on a different route on Saturday.

Special thanks to Wikimedia Germany and Switzerland and to everybody that helped getting this done. Michael is now working on getting the servers ready for deployment which we expect in a couple of weeks time.

Please consider our SOSM 2015 donation drive which should cover the initial costs we incurred acquiring the servers and any replacement and additional hardware we will need to purchase to get everything up and running.

Contributor Statistics 2014

Posted by SimonPoole on 9 January 2015 in English (English)

I've updated the statistics that I regularly produce from our changeset dump, see stats page on wiki.openstreetmap.org, to the end of 2014 numbers.

The well known trends continue with growth in every category with a total of 472'925 contributors since 2005.

Looking back on the 10th anniversary celebrations in 2014 it is important to note that was celebrating the birth of the idea, not the start of success.

Success slowly started in 2007 with the project taking off in 2008, for a Web 2.0 undertaking prior to that it was a flop. Perhaps even better than a graph Martijn van Exel's "then and now" map comparision with 2007 illustrated just how much of nothing OSM was back then (unluckily the comparision is no longer available).

It is diffcult to nail down why the sudden change in course happened, likely it was more the pieces of a puzzle falling in place than just one cause. However if asked to point to a single person and event, I would clearly point to Richard Fairhurst and the Potlatch editor being the pivotal person and thing that changed the course of history for OSM.

OSMF Members: Please Vote at the General Meeting this Weekend

Posted by SimonPoole on 3 December 2014 in English (English)

Dear OSMF members

You have likely already seen the announcement for the upcoming general meeting on the 7th of December. I would ask you to make the small effort and either participate directly in the meeting on Sunday or, better, vote via proxy now.

As I pointed out in the original proposal for this general meeting, actually achieving the 75% votes to carry the special resolutions is a very high hurdle and unlikely to be achieved without support by the board. But even if we do not pass the 75% mark, a high as possible support for the term limits will send a clear signal to the board that we want the issues to be addressed.

It has been suggested that the main problem with the proposed term limits is that they actually affect some of the board members ability to continue to stand for re-election. Without us sending that clear signal to the board, it is very likely that the board will implement placebo limits that only serve to pacify their electorate and have no real effect.

Please make the small effort to participate in this vote and make your voice heard!

Simon

Money!

Posted by SimonPoole on 27 October 2014 in English (English)

One of Steves favourite sound bites is “we have the money, lets employ somebody” in a couple of variants, used again in his current manifesto to convince people in to electing him back to the OSMF board.

During my time I simply ignored them, not wanting to derail things more than they typically were, but the obvious response would have been “how on earth would you know?” (well that is actually the polite version).

The OSMF has limited financial reporting (it has improved over the last two years) and essentially no planning, also known as a budget. It is currently simply not possible to know what the financial status of the OSMF is or should be at point in time in the future. And while the OSMF has roughly £80'000 of cash available we shouldn't be eating too much in to that given that substantial amounts of it would be needed in case of larger hardware problems (imagine one of our hosting locations catching fire) or other shortfalls and unexpected costs. Not even mentioning any kind of planning for short term cash needs.

It is one of the things that I consider a personal fail that we didn't manage to get something resembling a conventional budget in place over the last two years. It is not difficult, but it would have required cooperation from multiple board members to be actually meaningful and that was not forthcoming and so I concentrated on other pressing issues.

One of the larger planning issues is the largest line item, SOTM. Right now a couple of days before this years event, the OSMF board has no idea what has been signed in its name (if anything has been signed that is), what the potential risks are, nothing. And this was the same the previous year, the year before, the year before that and so on ….

Last year the board faced a potential GAU (German expression for the largest conceivable accident) when the local organizers for Birmingham threatened to walk out, we patched that up, but till this day we don't quite know how much we would have been in the red had the worst case happened. [I would like to point out that the local team was doing the right thing. I was simply showing that, yes, even "safe" things go wrong, and without planning you can't even do "what if"s in such a situation]

Now there is quite a lot of potential sources of funding available for OSM, but we are kidding ourselves if we think it is going to rain manna if we don't get our act together, get our financials under control and actually have projects worth funding (a separate topic).

To be clear, from a legal point of view there is nothing wrong with the OSMF finances and the way we have financed hardware purchases in the past has been low risk, leaving SOTM and our G&A as the larger variables. But it has trapped us in a vicious circle.

Rebooting (with an emphasis on boot) the OSMF board might be a way of breaking out, installing a Junta with the person mainly responsible for the current state of affairs in charge, not.

Vision?

Posted by SimonPoole on 27 October 2014 in English (English)

In 2011 the then current board had a face to face meeting in Seattle and produced a set of "audacious goals" for OSM that were

  • The World’s Most Used Map
  • More Than Just Streets
  • Cultivating Leadership of Mappers
  • Easier Contribution for Non-Geeks

Now some of these would have been directly actionable, but the first two are really "visions" under a different name. And as visions goes they are quite good, far out goal posts that nobody expects to be achieved soon.

And we are still far away from reaching the first goal . There are lots of pieces missing before we can remotely hope to achieve it, but we are on the way.

Steve has long harped about adding more addresses to OSM, matter of fact I have too. From a pure usability point of view if you want to produce a competitive device or app with door to door navigation support you are going to need address data. To be more precise this boils down to adding hose/building numbers with their associated streets (intentional pun) to our dataset in some form.

Note on the side: there are lots of regions that don't have a conventional western way of describing locations, we don't have good support for that yet in OSM, solving the “address problem” tends to centred around 1st world countries.

The OSM community has lots of experience with adding addresses by both on the ground surveying and imports of suitable open data, matter of fact we've had a full country “complete” for half a decade now. What is however undeniable is that it is slow progress, even importing a couple of million addresses (we don't have good numbers, but it is likely that there is something between half and one billion address conventional house addresses out there) takes a lot of time if you want to provide some minimum quality and the preferable on the ground surveying tends to be even slower.

But we are making progress and looking at one of the larger countries, Germany, with 81 million population, we are now at roughly 1/3 coverage with a combination of on the ground survey supported by open data and smaller imports, completion likely in 2 years at the current rate.

There are some places where we don't have a good handle on the issue, for example in the “original OSM country”, the UK, due to the addressing system revolving mainly around non-surveyable proprietary schemes and house numbers playing a secondary role. But we are not the only group feeling the pain there and I'm optimistic that we will find a way out, and if it is simply by replacing such proprietary systems.

To summarize: addresses are important and yes it is something that the community is working on with dedication. There is really nothing visionary about it at all at this point in time, not more than “lets map all roads in Germany”.

Hammers

Steve has a legitimate commercial itch that he wants scratched and he wants that fast. This is not unique, readers following the “imports” mailing list have seen similar requests from HOT, matter of fact one of the HOT requests had that other problematic attribute “non-editable”.

Luckily HOT hasn't tried to stage a coup d’état to get their way. But what it does tell us is that the OSM data format, its tool chains, its editors and other applications have become extremely popular and people prefer our tools over others. It is a great testimonial to OSM.

OSM was built around the notion of mappers collecting or curating data and then adding it to our central repository, iteratively improving the quality and completeness over time. “fast” and “non-editable” are the antithesis to what normal OSM is about.

Now I'm sure ESRI doesn't get many complaints about them not providing the ultimate multi-100 GB all free geo-data of the world shape file that includes everything, so why do we? I can only guess that it is because people are feeling left out and think that if their favourite dataset is not included that they are not part of this great community.

But, really, it is just a superficial marketing and packaging issue.

If Steve or better his employer want to create the

ultimate_free_address_superpack_US_edition.osm

containing a quickly thrown together conversion of all the open data address sets in the US, great, the OSMF might even be convinced to distribute it in the same place as the planet dumps. And we have more than enough id space to support conflict free merging.

There is simply no need to compromise the normal OSM community process for a short term gain.

We don't only have hammers, please stop just seeing nails.

Thank you for your trust!

Posted by SimonPoole on 26 October 2014 in English (English)

Rewind two years, September 2012. I'm rather fuzzy on why I stood for the OSMF board in the first place, but I remember at least being annoyed by the treatment of the working groups by the board, experiencing first hand a dress down of one our most respected community members just because he didn't jump when the then chairperson said jump. And yes something didn't feel quite right about the organisation.

To make a long story short, I wasn't expecting to end up being elected the chairperson and further was expecting the fault lines inside the board to be in other places than they turned out to be. Naive me assumed that we would be having heated discussions about the scope of services the OSMF provides and how we delineate what the OSMF does from commercial and other services providers, how to grow the OSMF membership base and how we could best support the wider OSM community. Alas things were not quite ready for that.

My approach was let the past be the past, fix the issues and then move on. The net result was an uphill battle every foot of the way over the last two years and for the moving on part, well that might have happened now.

Why did I step down?

No, I didn't step down because of some flak on the mailing lists. I'm a big boy (well I am kind of small actually) and can easily take the heat. A consequence of exposing yourself and engaging with the community is that you tend to function as a focal point for everything real or just perceived wrong about the organisation you are representing, it simply comes with the territory. But I still think it is necessary out of respect to the OSMF members to be open about what I think and where I believe the sticky issues are, instead of spewing safe political correct canned sound bites or hiding away somewhere.

Steve Coast has decided to stage a come back after two years as chairman emeritus, as board member and chairperson that put me between a rock and a hard place. On the one hand I have to respect our democratic processes and not interfere with the ongoing election, on the other hand I do hold strong opinions on where we should be heading and in what fashion.

To make things clear: I consider it a very silly thing for Steve to do, bad for him, for the OSMF and for the whole project.

As the board member that has driven most of the change over the last two years, I see the danger of old alliances undoing the baby steps towards a more mature organisation we have achieved. An organisation that could actually engage in all the things the current board candidates envision. An organisation that holds a stable course and can function as a trustworthy intermediary between our fantastic community and the rest of the world.

I stepped down to make clear this is not about power, or my personal position, but about what is best for OSM.

Somebody who has been to a recent talk of mine on OSM might remember a slide representing OSMspace, showing some of our fairly complicated interactions and overlaps with the outside world, the OSMF a tiny speck somewhere off centre. Yes the OSMF is just a tiny piece of the puzzle with limited influence on the project as a whole. Most of our contributors will likely never know that it exists, no different than the Wikimedia Foundation. What is different is that OSM is a significant player in a multi-billion dollar market and the outside forces are of a very different nature.

While the OSMF is just that tiny speck in the OSM ocean it does hold the keys to our core product and with control over the foundation it is completely possible to misuse that. Yes, even handed, community control of the OSMF is important for everybody in OSMspace, and it scares me when somebody seriously proposes handing those keys to a Junta, even if it is “only for a year”.

I would like to end for now with thanking everybody that was kind enough to show me support over the last few days it is much appreciated.

More to come .....

1000

Posted by SimonPoole on 25 October 2014 in English (English)

A bit over 10 months ago the SOSM board decided that we would try and systematically send a welcome note to every new contributor in Switzerland http://sosm.ch/how-many-mappers-are-there-in-switzerland/. Today I just sent off the 1000th such message, not a big deal, takes a couple of seconds per mail and I typically send them every 2nd day or so. The mail (in 4 languages, really should be in five), tries to be as low key as possible and simply welcomes the new mapper and points out our local and international resources.

Observations

  • there hasn't been a noticeable increase in SOSM membership signups, on the one hand 10 months is too short term to expect that to happen, and on the other hand we don't actively solicit that in the mail anyway.
  • just as above direct feedback is quite rare.
  • the "one edit user" seems to be on the way out, most of the users already have more than one changeset by the time we send the mail (nailing that down in hard numbers is a project for another day).
  • given that Switzerland is rather small (8 million population), well mapped (as all DACH countries) you wouldn't be surprised to see some indications of saturation after 9 years, but there are no such effects noticeable at all.

If you think this is a good idea and are considering doing something similar yourself, please note:

  • in most countries you will want to split the work up and coordinate, swamping new users with mail is likely not a good idea.
  • Pascal Neis provides a RSS with new users on a country by country base here: http://resultmaps.neis-one.org/newestosmlist
  • users signing up to OSM don't have to agree to receiving unsolicited mail, matter of fact there is currently no way that they can opt-in at all. This implies that you should consider the legal implications in whatever jurisdiction you are and be careful how you word your mails. I haven't heard of any problems, but you never know.

As a final note: most old timers know about or have participated in the reoccurring discussions on why people sign up for an OSM account and then never edit (there is currently no way of getting zero edits sign ups per region, so you can't contact those). I'm personally fairly sure that we have quite a good conversion rate, there are simply no comparable numbers from other projects available to prove that, others disagree. But as this example shows http://www.openstreetmap.org/user/user_4682 in the end we get them all :-).

PS: I should point out that by no way this is particularly original, we have had users doing this for a long time, it should be seen more as a reminder that this can be done.

Contributor numbers Q3 2014

Posted by SimonPoole on 12 October 2014 in English (English)

I've updated the statistics that I regularly produce from our changeset dump, see stats page on wiki.openstreetmap.org, to the end of Q3 2014 numbers.

The well known trends continue with continuous growth in every category with a total of 447'883 contributors. Of special note is

showing that at the end of the third quarter we have already had slightly more active contributors than in the full year 2013.

August 2014 showed the 2nd highest increase in contributors in the history of the project with over 10'700 new mappers joining OpenStreetMap. It is not unwarranted speculation that this was due to the tenth anniversary celebrations.

Upcoming Vespucci 0.9.5 release

Posted by SimonPoole on 1 August 2014 in English (English)

Vespucci receiving GPS data from RTKLIB

Vespucci 0.9.5 is now very near the feature freeze for the 0.9.5 release, numerous tasks that have either been been requested by users for a long time or have been on my personal to do list have been completed and there are only one or two left to do.

Some of the more interesting new functionality

  • On device help and some usability improvements
  • JOSM compatible OSM file reading and saving
  • Auto download
  • Fast address tags adding with house number prediction
  • Import and upload of GPS tracks
  • Function to add node at current GPS position
  • Support for external GPS sources (for example RTKLIB)
  • Basic conflict resolution
  • and numerous "under the hood" changes

If you want to give the beta version a spin, it can be downloaded from

https://drive.google.com/folderview?id=0B9pKLmh8s1h8bFI5bGd4VnhYWkk&usp=sharing

Contributor numbers revisited and empty changesets galore

Posted by SimonPoole on 20 July 2014 in English (English)

Some of you may have seen my blog posts on contributor numbers in the past (May 2013 , December 2013) on our contributor growth based on analysing the regular changeset dumps.

While I was fairly sure that the numbers were correct, I did know that I hadn't taken one potential systematic problem in to account: users that had signed up and produced only empty changesets. Now it could be argued that such users at least tried to contribute and should be counted, but opinions may differ on that. It was clear that any effect on current trends would be minimal, all modern editors will typically only allow you to save if you have actually changed something, implying that an empty changeset is something that can only happen in an error situation (for example an editing conflict, or a crash of the application).

Here are the corrected graphs:

The effect of removing the empty changesets lowers the overall number by roughly 25'000 users, however as expected does not effect the general trend. End of June 2014 we had an accumulated total of 421'701 contributors, increasing at a rate of around 8'000 per month.

The following graphs compared to the previous ones show that most of the effect of the change is in 2009, which as we know is the year changesets were introduced in, so some issues there are not quite unexpected.

In mid-2014 we have already had more than 90'000 active contributors, we can expect a substantial increase in the year total if the trend continues.

Back to the empty changesets, when and how much of an effect did removing them from the numbers have:

and the same relative to the new contributors per month:

(note: the above is the difference between the old numbers with zero edit changesets and the new ones, in some cases this simply caused users to be counted one month later, which explains the single negative month)

As already mentioned all changesets before May 2009 were generated when changests were introduced. I haven't been able to determine what caused the large numbers from May to November 2009, however the changesets in question do not have a created_by tag and I suspect that they were created after the fact, by some kind of mechanical process. Nobody that I asked can remember, maybe a reader can shed some light on this.

After November 2009 most of the empty changesets were created by Potlatch 1 and a well founded suspicion is that they were caused by P1 live mode. This continues up to April 2011 when Potlatch 2 was made the default editor and the absolute numbers have remained stable in the 100-200 users per month range since then.

Now it is important to remind ourselves what we are looking at: the numbers are new users that signed up, tried to edit, failed and never had a successful edit after that.

In other words, users that wanted to participate that we lost (the total number of empty changesets is far higher, however for now I'll assume that regular contributors are more tolerant of things going wrong than first-timers).

Done deeds are done deeds and there is nothing we can do about users that we have already lost, however what we can do is try and improve the situation going forward.

I've produced some numbers on which editors the users were using and nearly all (yes including JOSM) turn up, however the majority of the users effected are using iD (which is not a surprise given that it is the default editor) and, big surprise, "Go Map!" at nearly the same level. Given that "Go Map!" has a far lower user base than iD (editor stats), this indicates that there may be a real problem with the app this has in the mean time been resolved. Naturally given that iD is what is usually used by potential new contributors, further investigation is warranted too.

Crazy TIGER

Posted by SimonPoole on 5 May 2014 in English (English)

Anybody that has spent any amount of time fixing TIGER data in the US has seen all the artifacts that we have all come to love, over- and underruns in corners, lost fixes after being inside, wiggles where the surveyors stopped to read road signs, crosscountry non-existing residenials and so on. Today I saw one thing I hadn't before:

(blue is the corrected version)

OpenStreetMap and the Open Data Movement

Posted by SimonPoole on 4 May 2014 in English (English)

[As always this is just my own, personal, opinion, and is no way an official statement by anybody]

Yesterday I had a short exchange of tweets with somebody that was surprised that http://opendata-hackday.de/ was using google maps instead of OSM. Given that it is rather a convoluted subject, explaining why this in fact is not surprising was a bit difficult in 140 letters and is what prompted me to create this post.

It is probably just natural that outsiders, even members of both the OSM community and the Open Data movement, simply assume that these are essentially the same and have a large overlap in motives and goals. Numerous OSM contributors are active in the Open Data movement and undoubtedly we are a very large consumer of open data in various forms.

However this apparent overlap shouldn't hide the fact that both our goals and motives are in large parts completely different. The Open Data movement is about liberating, accessing and exploiting data that is already there, and, please don't take this negative, about improving the bottom line of the companies involved. One of the major arguments used in prying data out of the hands of government is that it will have a beneficial net effect for our economies and the involved companies and I don't have an argument with that. On top of that, the “we have already paid for it” justification is surely, at least in some ways, correct.

The OpenStreetMap project is very different, it is all about producing free and open geo-data and while we do utilize open sources, we are clearly at our best when the data has been surveyed and curated by mappers on the ground. Our goal is, in the end, to produce the best “map” of the world that is at the same time freely usable and re-reusable. Yes, some of the economic arguments apply just as well to the OSM ecosystem as they do to the same in the Open Data movement. I think we should be all be proud of the enlightened view the OSM community has had on commercial re-use of our data from the beginning.

But it has to be very clear: the OSM community created and owns the OSM data, and we control the terms on which it can be used, we have not been “already paid for it”.

Hopping off the soap box, I believe it is now understandable why complete alignment of goals cannot be expected. For “joe open data” google is a just as valid member of the open data community as OSM. google may even fit the open data model better: consumes and produces non-open products from it. We are just the crazies that spend our own time producing something free.

FIX THE MAP

Posted by SimonPoole on 1 May 2014 in English (English)

It seems, even though it has been available for quite a while, that the fixthemap page on openstreetmap.org is still not particularly well known. If you have an OSM based project and you don't want to create your own page for this purpose (as for example Mapbox has done now too) please provide a link to the page with a suitable text. You can provide coordinates with the URL (best probably the center of the map the user is viewing) and if the user chooses to add a note it will be positioned correctly.

An example of how to do this is for example the SOSM run osm.ch site: http://www.osm.ch/#13/47.2095/8.5237

vespucci 0.9.4 available on google play store

Posted by SimonPoole on 24 March 2014 in English (English)

Vespucci Release 0.9.4 Highlights

This release contains a lot of “under the hood” improvements and some work on making the UI more consistent and easier to use. In particular the following changes have been made

  • selectable overlay layer.
  • support for multiple simultaneous presets.
  • added find action to lookup location with nominatim.
  • add per zoom level imagery offsets with support for querying and saving to the imagery offset database or manual entry.
  • added support for name suggestions and auto preset setting.
  • added goto current GPS location.
  • added action to arrange nodes of a closed way in a circle.
  • limited support for geo: URIs and JOSM style remote control.
  • add action to directly set position of node by entering coordinates.
  • major rework of imagery provider configuration, now based on https://github.com/osmlab/editor-imagery-index .
  • make https API default.
  • major refactoring of projection code.
  • lots of bug fixes and stability improvements.

The full change log is available here http://code.google.com/p/osmeditor4android/source/list

We will be updating the documentation to include the new features as soon as possible.

Upgrading from previous Versions

There are a few points that you may want to consider when upgrading from previous versions of Vespucci:

  1. some of the new features may cause degraded performance on older phones, see http://code.google.com/p/osmeditor4android/wiki/FAQ#Running_Vespucci_on_%22old_and_small%22_devices
  2. 0.9.4 uses a new configuration file format for imagery as mentioned above. As a consequence the internal identifiers for the background tile providers have changed with the exception of the “standard” mapnik tiles and Bing imagery. If you have extensively used any other sources you will be left with unused tiles using space on your device. You can delete unneeded directories and tiles by navigating to the andnav2/tiles directory on your system (where exactly the directory is located depends on the device and Android version, but it will be in the same place as the vespucci directory) and simply deleting the directories with the exception of BING and MAPNIK.
  3. some of the defaults for preferences have changed in 0.9.4, your old values will continue to remain the same, however the new defaults seem to make much more sense. See http://code.google.com/p/osmeditor4android/wiki/Tutorial
Older Entries | Newer Entries