Recent diary entries
I have talked publicly about improvements to walking papers since at least SotM 2013. Made a blog post here in 2014 with some thoughts. But all I've seen were new ways to print tiles or atlases. While I admire the Field Papers and MapOSMatic fork improvements over the past years, a good walking paper is more than that.
For a long time I have been using a 28-step process to prepare walking papers for my mapping parties. It involved using Maperitive, Inkscape and some proprietary software. This year I finally got fed up with reanimating that old renderer, which doesn't work perfectly on Linux, and tried something else. I had always been recommending QGIS for printing maps, and I decided to try it myself. Turned out, making walking papers with it is really simple and straightforward, albeit not without issues.
I started writing another guide with QGIS and GDAL and all the new tech, but it quickly grew to 22 steps. Still too many. Having discovered the Python Console in QGIS, I started experimenting with automating a few tasks. One thing after another, and now I have automated almost everything, fixing a few issues in QGIS on the way. I present to you...
Walking Papers QGIS Plugin
It is the simplest way to prepare good walking papers for your mapping party. All you have to do is sketch the pie, and the plugin does the rest. Here are the complete instructions:
- Install the "Walking Papers" plugin from the official QGIS repository.
- Click the button with blue rectangles and choose "Download OSM Data".
- On a layer it created draw a polygon around your mapping party area (click a pencil button, and then "area" something near it. Left mouse button adds a node, right button closes the area), and choose the same menu item again.
- Yay, we've got a map. Sketch the pie with lines in the "Pie Overview" layer.
- Having finalized the pie, activate the "Pie Sheets" layer and draw areas around quarters that go on each of the printed sheets. Usually it's 2-3 sheets per a pie piece. Name areas like "4-west", where 4 is a pie piece number, and "west" helps a mapper to locate themselves.
- Click the blue rectangles button and choose "Prepare Atlas". That's all, check out the sheets and print them or export them to a PDF file.
Amazing, right? For a regular mapping party this way of preparing walking papers gives you much more control, and you would need to do much less explaining when handing these sheets to participants. Here is why I prefer it to atlas-printing websites:
- The data is very recent. It is downloaded from Overpass API, and you don't have to wait for a server somewhere to catch up. Buildings missing? Ask mappers to help drawing them, and print the papers an hour before the party, with everything they managed to draw by that time.
- Custom map style. With online services you have basically one good choice: Stamen's Toner. It is not perfect for walking papers: labels are in English, lines are too thick and dark, buildings don't have numbers and are hatched, so you can't draw anything on top of them, and the water is awfully black.
- Vector maps. You are not limited by zoom levels, and thickness is specified in millimeters, not pixels on some maximum zoom level.
- Custom attributes. The bundled style prints house numbers and building heights on buildings. It is not easy to alter that at the moment, but by manually editing
wp_style.yamlfiles in the plugin directory (
$HOME/.qgis2/python/plugins/walking_papers) you can add any attributes and change the style however you want.
- Rotation. It is frustrating when the roads in your mapping area go in 45° angle on the map, which makes most of the space on walking papers sheets unusable. With this plugin, maps on your sheets are rotated so objects on the map are as big as possible, and you have plenty of space to put down POI names and house details.
- Speed. No more waiting for an hour while your task crawls through the queue. Click a button, get an atlas, that's all.
- Works offline. Download a map area in JOSM beforehand, or copy it with a flash drive from a connected computer, and use the "Open OSM Data" menu item.
I hope this plugin helps you with organizing a mapping party. We know these don't help with attracting new contributors, but parties are fun, you get to know your city or village better, and the amount of data you collect is unmatchable by any other data collection method.
The OSMF Board elections happened a month ago, but only now I've got my hands on the anonymized ballots. Which means we are getting some statistics.
First, let's look at numbers. By the 10th of November we had 457 members eligible to vote. That is much lower than the previous year, when we had around 500 eligible members. Still, that's just a tiny fraction of active OpenStreetMap editors, so the room for improvement is huge. Other numbers have also dropped: 4 candidates instead of 11 in 2015, and 253 voters (55%) instead of 272 (~53%).
It was definitely the fastest and simplest of OSMF elections: with only four candidates, two of whom were running for re-election and other two much less known in the community, 77% of first choices were either Frederik or Kate. That is, Kate got three times as many first votes than the runner-up. I doubt there is a system in which the outcome of the voting would be different.
Let's look for the third place. Guillaume got 27 first votes and Darafei got 29. But did Darafei take the third place? By now you should know the STV does not make the answer obvious: it depends on how many people have listed these two as their second choices. And we see that twice as many Frederik's voters had chosen Guillaume for their second vote: 38, and 19 for Darafei. Kate's voters, on the contrary, preferred Darafei: he got 23 vs 18 for Guillaume.
Since the weight of Frederik's extra votes was much higher (0.4 vs 0.3) and more people gave the second vote for Guillaume, he gained most after the two transferral rounds: 60.2 vs Darafei's 55.4. Which would make him the third winner, had we another seat to fill.
Five ballots with just one candidate listed had either Frederik's or Kate's names on them. One ballot was empty, and 184 (73%) had all four candidates. Just one ballot did not have Frederik or Kate listed. 17 ballots had Frederik and Kate for third and fourth choices.
Darafei would take the third place in “Approval Voting”: if we count all mentions of candidates, regardless of a position, then Frederik and Kate would get 234 votes each, Darafei gets 217 and Guillaume 207. Incidentally, Darafei gets the most “disapproval” votes, that is, he is listed fourth on 67 ballots out of 184 that have all four candidates. Frederik has the fourth line only on 29 ballots.
Here is the original ballot file, which you can feed to OpenSTV to check on numbers, or analyze to get more statistics.
The OSMF Board did not change this year, which means we don't have to learn new names, but also that we won't hear new voices. I really hope next time there will be a lot more candidates, especially from under-represented countries. You don't have to be fluent in English or know accounting to serve on the Board. An urge to make OpenStreetMap better and an understanding that you would need to move in small steps is all it takes. Do push yourself for the elections this Autumn, and thanks for choosing us, the current Board members.
The editor usage stats wiki page has two useful tables for comparing editors: by number of users and by number of edits. And while MAPS.ME reach is clearly visible in the first one (alas, it hasn't still surpassed iD in users), the second table has much lower numbers for the mobile editor. Which is reasonable: MAPS.ME does not allow editing geometry, thus a single user can make more edits in a day, than all MAPS.ME users in a month.
I was wondering if we could compare only the type of edits our mobile editor allows, that is, POI edits. And with daily replication diffs, I did:
This chart shows that while users of iD and JOSM editor create more points of interest than MAPS.ME users, it's not by a large margin.
I noticed days when 10k or 20k new POIs were uploaded with JOSM, and wondered if the picture would change if we took median values for days of each month. That is, from a sorted array of numbers we would take the middle number. Turns out, not much.
Most of us know how hard it is to update POI data, besides fixing geometry or fixing tags. Not only you have to go out and collect new opening hours or names, but also you must know that something has changed. Mobile editors usually excel at that, so why editing numbers are higher for desktop editors? That's because I could not separate meaningful tag edits from automated, or geometry changes from tag edits. Maybe if I could, MAPS.ME's share of POI edits would be much higher.
Last time it went pretty well, so let's try another long-standing issue. Namely, #1274.
You can register or log in to the OpenStreetMap website using a social account: Facebook, GitHub, Google. The thing is, most of us have many social accounts, but you can only link one to your OSM profile. If you click a wrong button, you will have to either go back or register a new account. You cannot log in to your account using different social buttons.
For me, that is a problem. I would like to not remember which of the buttons I clicked when I tested the social login, and which of these is linked to my main OSM account. For that I would like a social accounts management in the settings page. Alas, I don't have time and skills to add that, but I have a hope that somebody has. And for that I'm willing to pay a small grant: a hundred euro, like the last time.
The offer stands for two months: the pull request to openstreetmap-website should be submitted until 15th of January and merged until 15th of February. Right after merging I will transfer 100 € to any given credit card or bank account.
I've just counted some statistics on a planet file from 14th of October. Here it is:
This table shows a number of nodes, both tagged and untagged, that are referenced by ways and relations. You can see that nearly 97% of 3.5 billion nodes are untagged, and most of these — 88% — are part of exactly one way or relation. Like, when you trace a building, you add four untagged nodes that are part of that closed way.
98.4% of all nodes are part of something, but only 12% (424 million) have two or more parent objects. This could help with designing a data storage for nodes.
There are equal amount of tagged nodes that are not part of anything, and part of an element. Interesting are these 9 million tagged nodes that are part of two or more ways. The taginfo says there are 2.5 million crossings and 860 thousand traffic signals, so that's a ⅓ of that.
Finally, we have a million of nodes with no tags not being a part of anything. I wonder when someone puts on their OSM saviour cape and a programmer's hat and rids us of these.
Many mappers agree that simple and accessible editors are hazardous: the simpler editor is, the easier it is for a horde of newbies to submit wrong data. This was a main argument against Potlatch, and then iD. Now MAPS.ME built-in editor allows for changing tags and adding nodes with just a few clicks for any of our tens of millions of users. Which of course has led to a number of questionable edits.
The first field in any place card is name. When we released the editor in April, it was a single field for editing the "name" tag. You changed a name — the new tag value was uploaded to the map.
Complaints started coming almost immediately. Turns out, some tourists were renaming attractions to their language for easier navigation. If you look at the Questionable Edits wiki page at the time, you'll see that names in wrong languages are the most worrying kind of edits.
How do we fix that? Well, finding the language of the name from its characters could work for some languages (like Chinese vs. English), but not for most. Adding a warning that users should type only what's written on a plate is better, but it was there from the start, and nobody reads instructions. Removing the field completely, like some suggested (along with the rest of the editor) could solve the issue, at expense of the better map.
In August, we prepared a list of native languages for each country. For example, in Finland it's "fi" and "sv", in Estonia it's a single "et". India has 11 languages, though its regions have less. We took this from the Wikidata, which may be incomplete and sometimes wrong. If you have a minute, check this list for errors. Languages should be ordered from most-used to least-used.
And with that, we completely disabled editing of the "name" tag in the 6.3 release. Mappers were asking, and we delivered. Now users were presented with one or two native language name fields, plus an English name and a name in a user's language. For example, if you are a russian in Helsinki, you'd see editors for "name:fi", "name:sv", "name:en" and "name:ru". This way it was less likely Chinese names would be entered into e.g. name:en. And since the default style on osm.org uses only the default name, changes from maps.me wouldn't be shown there.
Except for new objects: when a user creates a POI and fills any of the native language fields, that name gets copied into the "name" tag. But not when editing. Which started causing another kind of error: when a shop had changed its name, we would get old name in the "name" tag and a new name in "name:lng". It displays properly in maps.me, since we favour localized names, but not on other maps. And some mappers started complaining about equal values for "name" and "name:lng".
With the 6.4 release, we adjusted the workflow again. Keep in mind that our goal is to prevent accidental mistakes by users, not by experienced mappers who know how the application works. For the latter, we added a special language: "Native for each country" at the very bottom of the languages list. That's right: it is a way to edit the "name" tag directly.
When creating a POI and filling a name in a local language, that name will be not copied, but moved into the "name" tag, so you won't see duplicated values in tags. In my opinion, that's a drawback, but still, that's what mappers requested.
Now the complicated part: when there is only one local language for a region, like in Estonia or US, a user has a chance to change the default name. First, all empty name fields for local languages and English are pre-filled from the "name" tag. If a user have edited names in both languages, this would mean the user knows what they are doing, and the app will put the local/English/any other (whichever is not empty) name into the "name" tag.
This still means you will get discrepancies between "name" and "name:lng" values for countries with more than one local language, or with users who don't have time to edit all the fields. Know how to make name editing more safe and effective? Please share it here in the comments: maybe we could make it more transparent or even more smarter.
Okay, I've got tired of this UI, and I'm swamped with other tasks, but there is some money left from my travel to SotM. So I am announcing a grant: 100€ for a merged pull request allowing people to subscribe to comments in OSM user diaries. (NB: 300€ now, thanks to Stereo and Mikel.)
There should be a checked by default checkbox near the "Save" button ("Receive notifications about further comments"), and a button to subscribe/unsubscribe. All notifications should go to e-mail, much like changeset comments now.
When you have your pull request merged, I'll transfer that money to your card or bank account. And of course I'll publish a big thank you :)
The offer is not indefinite: the PR must be submitted until the 1st of November and merged before the 1st of December. And yes, there might be a competition, in that case OWG will decide the winner by merging a pull request.
This Sunday, we will meet the nominees for the first OSM Award and learn who gained the most votes in each of the six categories, including Mapping and Blogging. 650 mappers have already voted, and if you have not, please head to the awards website and make your choices. All you need is an OpenStreetMap account.
The voting closes on the morning of September 22nd, when the Brussels Maptember begins with two great conferences: HOT Summit and FOSS4G.be. After that we will dive into OSM topics at the State of the Map, and on Sunday, before lightning talks and workshops, we will know the winners.
But for now, there are still ties in some categories, and your vote can decide who will get the award. Do vote now and meet us at the State of the Map!
Which popular editors do we have now? According to statistics, three of these: iD, JOSM and Potlatch 2. The next four editors are mobile: Go Map, OsmAnd, Vespucci and Pushpin each have a thousand of users. Today there is another one, which now has less than a hundred users, but aims to go for the first place: MAPS.ME.
As you might know, MAPS.ME is a popular app for using OpenStreetMap data on a phone or a tablet. It has geocoding, routing (using OSRM engine), bookmarks and 3d-buildings. It runs both on Android and iOS devices, and it is very fast. Obviously it works offline: you just have to download some countries. Besides speed, MAPS.ME is known for simplicity: even I can understand which buttons to press, without examining every control and menu item first.
Just now we have released the first major update this year. It has better geocoding (and reverse geocoding), smaller regions (no bigger than 70 MB, most are below 50) and, the most important change, now it can edit the map! In most POI cards (click on a POI and get one) there are two new options: «Edit place» and «Report a problem». The first one opens a simple (as in, easy to understand) editor for relevant fields: name, address, opening hours. The second one is for leaving OSM notes. Also there is an «Add a place» option on the menu. So yes, it's an editor.
We decided not to build proxy servers, but to connect each of our users directly to OpenStreetMap. Which means, there is a sign in / sign up option that pops up when you edit a feature. It uses OAuth, obviously, and for new users it promotes using Facebook / Google authorization. So, every MAPS.ME user that edits the map submits their confirmed e-mail, so you can contact them. It works, as I have contacted some of the users during the beta testing.
OpenStreetMap currently has 25-30 thousands of users editing monthly. MAPS.ME has more than 7 million monthly active users. So I assume that in time it would be the most popular way for adding and editing points of interest in OSM. You can monitor edits made with the application: that service will most likely break when there is more than 5 edits per minute, but for now it works. When you see a mistake, "Level0" button is the fastest way to fix it.
I've been using MAPS.ME editor for two months, and it is good. I have finally added and refined POIs around my house, and I add new objects each time I go somewhere. Because it is so easy, and because I use the application for navigating anyway. I have been saying since Autumn that mobile editing will be the next step of OpenStreetMap development, and now we can witness it in real time.
If (when) you encounter a bug, please write to email@example.com. The next release is planned for the end of the month, so with your help we can make the editor even better. We know some of drawbacks: incomplete type list, users can add the same feature multiple times, no notifications for sent changes, non-native facebook/google authorization. Most of issues will be fixed by the next version, but we would like to hear your opinions on how to make OSM better with this new editor.
Happy editing, and we're sorry you will have nothing to update around your house in a month :)
Почему-то информация была только на форуме, но на ближайшие дни, если хочется потренироваться в картировании города, который ещё не так надоел, как ваш собственный, загляните в тему http://forum.openstreetmap.org/viewtopic.php?id=53769
Alas, you cannot embed youtube videos in diary posts:
Did I mention to vote for me in the upcoming elections? :)
I am Ilya Zverev, a.k.a. Zverik, currently living in Russia. I've nominated myself to the OSMF Board, and now I have supposed to write a manifesto, touching on diversity, transparency and other serious topics. But the thing is, I've read all manifestos for past three years, and all of them (except Frederik's) are boring and didn't affect anything. So what if I show support for transparency — would it help? Nearly all candidates supported it, and look how verbose Board minutes are (they aren't).
Do you know what the Board is working on? I don't, and I read the minutes. There are two options: either the Board discusses a lot of things on their private mailing list, or they actually are working only on topics spotted in minutes. Both of these are not good: I am a member of OSMF, and I expect to know what's in store for OpenStreetMap. I hope it would change, and maybe I could help it — but after Frederik's revelations, I am not sure. I'll try.
I support diversity. The Russian community is severely under-represented in OpenStreetMap, despite being the fourth (occasionally the third) biggest in the project. Though I don't like needless «regionalizing» of some aspects, e.g. tagging. Obviously, we should promote OSM in more countries, though I don't see how it is the Board's task. Last year I learned what gender diversity actually means, how inequality is enforced by nearly every aspect of most cultures, and that IT has it worst. But also I was taught that, as a man, I can do nothing about it, other than properly teach my daughters. So I have nothing to offer on the topic of diversity.
What I am interested in, and what is one of the main Board's duties, is supporting the OSMF. I joined it in 2011, but never was sure what is the point of being there. Supporting the project monetarily — yes, but it is simpler with donations. The mailing list is sometimes entertaining, but mostly silent. I don't know who other russians in OSMF are, and cannot connect with them. I have no say in any of the decisions, except during the yearly AGM. Nothing in OSMF motivates me to do anything, though there is a lot of subtle shaming: you are not participating in any WG, you are not donating enough, you are not attracting more members to OSMF. Being an OSMF member is not fun, and I intend to change that.
And the money. Do you know what the last year WG grants were spent on? Only OWG seems to spend money on something useful. OSMF has a few pounds on its account, which doesn't seem to increase much, but also it doesn't spend it on anything outside the State of the Map and servers. There were a lot of promises and ideas in past years. I can remember the Ulf Award, software grants, mapping party funding. I doubt any of that was discussed by the Board: in my mind, it deferred these matters to working groups, and the groups weren't interested. I expect the Board to spend more, and to collect more. I'd like to have developer grants in the OSM, and not rely on big software companies to technically advance the project.
Corporate membership is the way to get more funds for the OSMF. Well, increasing the regular membership ten times, and that. My employer, MAPS.ME, is a corporate member, so I know how hard and frustrating it is to become one. This membership is not promoted in any way, which is one of the reasons we get only 4-5 companies every year. Maybe we should learn from Wikimedia: they are more visible and attract more big sponsors, and we could do with a couple.
I am not a good manager: I have no experience in talking with huge companies or managing teams (though I've organized quite a lot of mapping parties, and some conferences). But I won't be alone on the Board, and members there are good in different ways. I'd be glad if Mikel is elected, we need an experienced person like him. My expertise is in being an OSMer for a lot of years, in writing news and hoping OSMF would generate more of them, in being an OSMF member and not knowing why. I want to change that, to make OSMF more visible and meaningful, and that's why I want to be elected to the Board.
If you are not yet using MAPS.ME, you are missing out :) The most frequent complaint from the mappers was that official maps in that application get updated only once a month, along with new releases. I usually map stuff the day before I'm going out, so this update cycle does not suit me. And since I work for them now, I can fix this.
Since this month, there are daily updated map files for MAPS.ME. To install downloaded files on Android, find MapsWithMe directory on your device (you can check "Settings → Map storage" in the app), and put new files there. You should delete old maps and directories with same names (the latter is to clear caches). And probably restart the app. On iPhone and iPad, just use iTunes: find and open MAPS.ME application, delete old maps, upload new.
Maps are published every day at around 5am UTC. Mwm files are maps, routing files are needed for car routing (pedestrian routing doesn't need them). In a couple of days a new version would be released, and it will be required for daily maps to function.
These files are not official. The application may behave strangely (there will be notifications about outdated maps), data may be broken (it's OSM, it is always broken), and your application may crash. If you encounter anything strange, you can clean the MapsWithMe directory and/or move the app from SD to the device memory, which must fix most bugs. Daily maps are my initiative, and MAPS.ME company is in no way responsible for these. Of course, I'm ready to answer any questions.
Surely now is the moment for OpenStreetMap to accelerate adoption, usage and uptake? But why hasn’t this already happened? Why hasn’t the geospatial world run lovingly into OSM’s arms?
Gary Gale published an interesting article on removing SA clause from our license (actually, the major part was about business-friendly face, but you know the principle: want it? go do it). We've heard it before, from Mapbox. As Richard points out, that won't happen any time soon, because there is clearly less than 2/3 of active contributors supporting the idea.
And these opinions strike me as lacking an understanding of OpenStreetMap project. Are we mapping for PNDs? Yes. Are we mapping for commercial companies? Of course. Would we like a thousand more commercial users promoting OSM by simply using it? Yes, go ahead. What? They cannot do that right now?
Well, we can wait. That what distinguishes us from other map data providers: we can really wait. OpenStreetMap is slow, but unstoppable. Mapbox and other businesses have immediate tasks, and for that they need a fast reaction from OSM. But OSM isn't fast. The last license change took 3 years. That's just a bit less than Mapbox has existed. Some think that because we make maps for crisis areas so fast, we are very responsive – but we are not. And it is good.
What I like in OSM, is that it is not going anywhere. Businesses appear and go bankrupt or sold, new datasets are published and then forgotten, but nothing ever can happen with OpenStreetMap. The question is, what will happen if we wait 50 years? I can bet a hundred bucks I'll still be mapping my town in OSM when I'm retired, but will Mapbox, HERE, Google, Yandex, TomTom exist? If we are to act right now, what good will it do to our project in a scope of 50 years? On that scale, publishing a new tagging proposal seems more important that changing a license, just so that some more businesses and government organizations could use our data without having to change their ways. Tags will remain, organizations – not so sure.
And that's why I think a share-alike license is perfect for our project – at least until another popular mapping project appears with a more open license. If we are starting to look good for businesses, look ahead 50 years and think, why we should adapt to their needs, and not vice-versa.
Of course, we can start thinking about changing the license, but don't expect a reaction in the nearest 10 years. Not at least until we've updated our API.
Two weeks ago Steve Coast held an AmA (Ask Me Anything) on Reddit. I've selected and rearranged some of his answers for this post, since he rarely expresses his point of view on lists or anywhere else.
Also, he agreed to participate in next week's Russian OSM Radio (in English, of course). You can submit your questions during the broadcast (22nd of January, 20:00-21:00 UTC) on #osm-ru IRC channel.
mr_gila: What inspired you to start up OSM?
There's a few different answers to that question. On one level, it was just kind of obvious. Back then, in 2004, Wikipedia was hot new technology and the wiki idea in general was spreading. Why not apply it to maps?
On another level, I had an old laptop with Debian Linux on it and a USB GPS device. I tried to use some mapping software but there were no maps. So why not make them?
On another level, the maps that were available in the UK and Europe tended to be very proprietary and expensive. So why not open them up?
On another level, I was young and naive.
Let's not forget though that OSM is now many, many people from all over the world. It wouldn't have worked if I hadn't convinced a lot of people to join in and help.
mapsandmapsandmaps: How did you find your time studying at UCL, and how much of an impact do you think this lead into you founding OSM? Does it feel strange that it has become a big topic of academic research with people like Muki Haklay writing papers about it?
UCL. I was working in a couple of PhD research labs and not paying much attention to studies. That mean I had the time and resources (computers with direct access to the internet, no NAT!) to go do OSM and other things.
Muki was in one of those research labs (as was Paul Torrens, Martin Dodge, Sean Gorman and others), so it's not entirely strange.
ManAboutCouch: Half-jokingly, how has OSM managed to get this far without a properly defined Polygon feature type?
OSM has succeeded, I think and in part, precisely because the data model (and other things) are/is so simple. When I started it, there were various calls for OSM to use all kinds of complicated schemas (like WFS). You'd blow your brains out just reading the specification. OSM to me in many ways was a people problem not a technology problem, and it's easier to fit the technology to the people (e.g. OSMs simple models) than it is to convince people to go use WFS.
NorbitGorbit: If you had to redo the map project from scratch, what sort of system would you use or design to handle crowdsourcing map data?
I think I'd pretty much do it the same with some tweaks.
I'm trying to be careful to assign credit. The addition of change sets and relations for example. I had similar ideas but I didn't implement those, and they're critical.
I think exploring tags beyond just keys and values, since we hack in third values by doing things like "addr:housenumber=42" for example.
Beyond the data model itself, waze really nailed some aspects of crowd sourcing. The human element of getting people to contribute certain things.
dv7d: Do you map? What do you like to map the most?
Yes I still map under a variety of usernames. I've been attacking addressing to get a feel for the complexity of it. I used to spend a lot of time cleaning up TIGER data. Map roulette is a good way to find random things to fix in the map: http://maproulette.org
GregZorz: What would you say to someone responding that your use of multiple accounts waters down ability to check authenticity/reliability of edits?
i'd say in the general case you're right but as the founder I've had satirical fake blogs set up about me, people follow me and other internet weirdness. So I take a degree of anonymity.
ManAboutCouch: Hi Steve, you're on record saying that you think the next big challenge for OSM is address data. Given the myriad of address systems in use across the globe, and how this is often perceived as 'less fun' than adding other features to the map, how do you see this challenge being met?
Frankly it's hard to see it happen within OSM any time soon. Addressing requires some bold moves. For example, only show roads on the OSM website which have addresses. That would instantly make the world go blank, and create a lot of pressure to add address data, similar to how OSM was 5-7 years ago but with more people and resources. That kind of bold move is unfortunately hard to make happen these days.
edparsons: Looking forward to the book, but to preempt it - Are there any decisions you made in the early days you now regret ?
I'll split this in to two. Mistakes and regrets.
Mistakes abound. OSM could have had an exit like waze. Segments (a data model we had prior to ways) diverted energy away. Trying to run mapping parties by telling people where or what to map rather than letting them self-select. Calling it OpenStreetMap when it's much more than streets.
Defining "mistake" would take too long, but we should note that many of these things are only mistakes when viewed under a certain light. Mistakes of some kind are inevitable when doing something new. I'm happy making mistakes because it means I'm learning something. What I discovered is that this doesn't apply to most people, for whom mistakes or even trying something which has a chance of becoming a mistake is... not something you do.
Which brings me to my only regret: Giving up too much power. I thought that everyone in the world thinks like I do, and would also give up power and try new things like I did. That for the most part simply didn't happen. It's worked out very well, and the people are great, and OSM hums along... but the days of taking big bets and risks is over. That drives me nuts, since there's so much more out there to do with open mapping than just making the map slightly better every year and running another conference. For example, addressing.
We've done very well, as you know. We blazed a trail for others to follow too. I just have a much higher set of ambitions, including OSM being "done" by now (which would include addressing, of course).
dalek2point3: Steve -- are you saying that you wished that OSM had gone down the route of for-profit crowd sourcing a la Waze, rather than non profit a la Wikipedia? Have you thought about these two modes and pros and cons of each?
That's why I mentioned "certain light" above, there are tradeoffs here. Being able to monetize would speed things up. It doesn't have to be all or nothing, open or closed. You can make data open after two years, or something.
The downside is you don't get the same unexpected use cases like the Humanitarian OSM Team going and saving the day in Haiti.
GregZorz: Forget mental regrets... looking back 10 years, is there a physical item(s), or data from a specific trip, you wish you you kept/saved/rescued?
I tend to throw things away. I remember the anecdote that when Jobs went back to Apple in '97 they had an Apple Museum with all the old computers and stuff in it. He closed it down. Or as I think Gates said, he doesn't spend a lot of time looking in the rear-view mirror.
I find those items like mapping t-shirts, paper maps, conference pens, old GPS units tie me to a past that is gone anyway. I'm much more interested in the future.
smellsliketuna: Do you get a lot of people pitching you on ways to leverage your previous mapping project, for for-profit ventures? It would seem to be a logical choice, given your knowledge and experience.
Yes - I'm on a few different advisory boards now for example. Notably Auth0 and ParkNav, the others are stealth.
mapsandmapsandmaps: What's your opinion on the open/proprietary software situation in the mapping/GIS industry and do you think open tools and data will eventually take over?
I don't think open software will take over because it's always playing catch up and very rarely customer-focused or original. As an example, select a group of numbers in Excel and it takes two clicks to color the cells by value. That is, green for low numbers through orange and red. A simple visualization that's very valuable that I use all the time.
Go try that in libre/open office, apple numbers or google docs. It's essentially impossible by comparison. Everyone tries to copy Excel (and ESRI and so on) but they always end up copying the wrong thing. See my talk and the part about Dubai copying New York: https://www.youtube.com/watch?v=v2UqGUa2Tgk
alexandreleroux: Do you see Google ever moving to OSM for Google Maps/Earth data? Other major players have done it -- at least partially (Microsoft, Apple, MapQuest, Esri).
Google people have been super supportive of OSM including funding our conference and so on. I think OSM just moves too slowly for what they're trying to achieve, and that's fine. The world can support more than one map or one ideology.
I think it would be hard for Google for a couple of reasons. First is the investment. Who wants to be the guy to write off billions of dollars? Second, the map isn't actually good enough yet for them, and they're not done yet. They're trying to get cars to drive themselves which in part requires great maps, and they're not there yet.
Will it ever happen? Eventually. I think it depends how long Google (and OSM) lasts, which depends on them (G) finding more than one business model, which enters in to the realm of speculation.
Think about it like this: Would you bet people wouldn't use wikipedia? In the end, if OSM is good enough at zero price, why wouldn't you use it?
alexandreleroux: I heard lots of folk in the geospatial community claiming that it's OSM's ODbL license that reduces OSM data reuses.
The ODbL is a convenient thing to blame for not using OSM. I haven't found a use case yet where it wasn't really about something else, like a business decision. For example, some don't want to contribute addressing back to OSM and so "the license is bad". It's like saying wikipedia's license is "bad" because I have to credit wikipedia when I use it.
Is the license perfect? Absolutely not. But we're breaking new ground here. There isn't another large open data project close to the scope and size. Could we go public domain? Yes, but then it's an open question as to whether it would succeed without incentives to contribute anything to the pot. Hence discussion of Linux vs. BSD.
smellsliketuna: What future projects are you looking to get involved in?
I think there's a lot out there in the world that can be fixed. Search can be a lot better as an example. There are a lot of closed databases in the world that could be freed up. It feels like local businesses should have better services to help them with their online presence.
Then there are simple things. I'm noodling with this: http://www.my-evangelist.com
(Ed.) A pitch from another reddit post:
It feels like companies and startups need evangelists more than ever, but they're hard to find and retain. Why not contract that?
I've been an evangelist and hired them in the past. It's hard and expensive. Then when you have an evangelist, you have to pay a bunch of money to fly them around to conferences. They burn out. And then, when you go to a conference, most of the people manning booths aren't super excited or inviting.
So I figured, why not offer evangelism as a service? Help man your booth, plan great talks, run and attend meetups, help with online evangelism. Maybe you just want to know what's going on at conferences.
It's not a replacement if you want a full-time person who truly believes in your product, but it's a way to fill out your team or hire someone to do a conference for you here or there, far cheaper than hiring someone.
dv7d: How does the future of OSM look like?
Every day that goes by makes it harder to justify not using OSM in some way, because the map keeps getting better and the price is staying the same. I've said enough about addressing elsewhere here already, but it's the missing piece.
Hi everyone, I'll be visiting Porto in two days. Portugal is great, not as cold as Saint-Petersburg. We've already been to Lisboa, from where we recorded a holiday OSM podcast: English parts start at 2:00:30 (featuring Manuel Hohmann, Thomas "malenki" and Jerry Clough) and 3:01:15 (with Ian Lopez). The next issue will be recorded on 8th, 20:00-21:00 WET, and I very much hope to talk with a local, portugese mapper. Maybe even face to face, though usually I use Skype or Mumble.
So, if you map Portugal, or live in Portugal, or know someone from Portugal — contact me please. Also, I'm up for a osmers meeting in a cafe near Santo Ildefonso on 7th or 8th. I'm Ilya, my phone (for sms only) is +7 921 583-12-91.
update: radio recording today, 8th of January
We in Russia have a weekly podcast recorded live: OSM Radio (looks like RadioOSM, I know). Every Thursday we go online, discuss OpenStreetMap news and interview guests, either from OSM community or from OSM-related companies (Sputnik, Maps.me, OsmAnd, NextGIS etc).
The next broadcast will be on the 1st of January. Because of the holiday, that won't be an ordinary podcast, but an "open mic" event. From 17:00 to 21:00 UTC we'll be receiving calls from OSM members (and making calls ourselves), and will ask three questions:
- What was the most cool or satisfying thing you did in OSM last year?
- What was the biggest event for you in OSM in 2014?
- What do you expect of OpenStreetMap in 2015?
On that evening we expect to bring the community together and to make them hear each other, to really feel like a group united by the common goal. And I think it would be great to have English-speaking OSM members on air as well. Probably in the second half of the program. We are open till the last visitor, so if there are many osmers, we'll finish later. If you want to participate, call me by skype (zverikk) on the 1st after 19:00 UTC, or mail me (firstname.lastname@example.org) your skype id, so I will call you myself. Mumble server is also available, mail me for its address. Let's make it an event to remember.
Hi. I often drop hints about what our project misses, and now want to talk some bit more about a part I'm interested in. Since I joined OpenStreetMap, I have been interested in geodata collecting methods. I quickly grasped walking papers, put my GPS and camera to use, and struggled with georeferencing audio recordings. OSM allows for many types of sources, and in past years a lot were invented. But alas, not much in last years.
Walking papers (or field papers) are still produced from tiles. Fieldpapers.org, the "modern" service for generating atlases, is more than two years old and is a slight update to older walking-papers.org, built in 2009. It stiches tiles and produces a PDF file. You have no control over map style, you can't even use your own tiles. "Toner" style, which is the best option, are updated infrequently: you might have to wait a week before traced buildings appear on it. And some of them still won't, since it's hard to grasp how it works and why it hides some features unpredictably. Finally, pages of an atlas will be oriented by cardinal directions, in a grid with 90° angles. Of course, not many towns have such proper road network, so you will have to choose smaller scale, with less effective area for mapping.
I think I can fix this. It is easy, really: most of building blocks for a proper solution are already invented. First, an interactive map, on which you place rectangles for pages. Arbitrarily, not neccessary in a grid. Maybe draw some lines, which would be "pie segments": instead of using MS Paint for making a pie, use some advanced technology. Maybe integrate it with MapCraft. So, a bunch of rectangles on a map. Not 90°-aligned: rotate them as you like. Align with road network, with streams etc, so areas for filling in take as much space as possible, and scale is biggest you can get. When done, just save your work and close the website. Go trace some buildings and tracks.
When a morning of a mapping party comes, open the website and press "Create atlas". It will display the progress, but the atlas will be made on a server asynchronously. First, it downloads an area from OSM API. Yes, not from a local postgis database, only fresh data. An added bonus (for local installation) is that you can use cached osm data, or just bring it from another computer, if an internet connection is weak. Then it applies a MapCSS style (which you can customize, even upload some of your own or josm's) and renders each page, rotating data as needed. Then it joins pages into an atlas and provides a download link. Later the atlas can be rebuild, using fresh data, maybe a different style or with more pages.
I don't believe in scanning pages for using them as a layer in JOSM (Bing/MapBox imagery make for a better reference layer), but georeferencing marks probably can be included. That won't be the greatest feature, because I have some other thoughts. You know the weakest point of the OpenStreetMap mapping in 2014? Updating data. It is quite easy to collect and map new roads, new POIs, new restrictions. But updating it is very, very hard, almost impossible for rich regions. There are no tools. My theoretical walking papers can fix that issue. Since we have full control over data, we can put POIs and relevant tags right on pages. We have a second side: for example, on the map there would be dots with coordinates (A4, D9), and on the reverse side — tags for each dot. And the same for ways and maybe relations (didn't think it all through, obviously). So you can have not only a base map for collecting new data, but also a check list for updating the already rich map.
This solution will make mapping easier not only in thrid-world countries (where internet is sparse and you can't rely on external web services, or spend days installing tile rendering stacks), but in densely mapped cities, where data has not been updated for years, because it already seems well-mapped, why go there again.
The next step would be an Android application for mapping. Why android? Because I again keep in mind mappers in third-world countries, who can get an android phone for $30, but not an iPhone for $300. So, let's take Vespucci. It is a powerful editor, getting better every months thanks to Simon. It can download an area and let you edit it. But can it work without internet and GPS? Not likely. Can it be used on a mapping party, when you are passing 10 points of interest a minute? You'll be exhausted in half an hour. The ideal mapping solution will have to separate data collection and data processing. Step 1: go out and record everything. Take photos, record audio, type a hundred house numbers, draw some crude lines on a touch-screen, like you do on walking papers. The interface should allow for quick mapping, e.g. in a car: you see a sign — you press a button and leave moving a mark to a correct place and tagging it for later. KeypadMapper and OSM Tracker are examples of this approach, but it should be made more streamlined and consistent. You cannot rely on GPS, for it has low precision, and not available on cheaper phones (off go those two apps). You cannot rely on the internet to provide you with tiles (off goes OSMPad). But you can assume you'll have a chance to download 100k of osm data back at home (or 10-20k on the road), to use as a base map for further mapping.
Step 2: Upload collected data somewhere (e.g. on your computer), process it and update the map. Data format can be universal, which means some central server for storing all the information. Audio notes can (and should) be converted to text, GPS trace joined from broken segments, data split between days and so on. Since most of points would be in tags or other non-textual representation, so anybody can use it for mapping, you would have an option of upload it to a server (right from your phone), so someone else could map it in their spare time. Or it could be you, download the data pack in JOSM. And then — map it. Recorded points shown as icons, with copypasteable tags; photos already georeferenced with subsecond precision, notes written on a map, crude drawings also georeferenced as underlying layers, between OSM data and imagery. It would be so much better then trying to remember what you meant by these waypoint titles, or having to read your handwriting on walking papers. And technical requirements would be as low as they get.
I am not a great programmer, or a designer. But I know what and how should be done to drastically improve mapping in OSM. No commercial company would make it, because they make profit not from mapping, but from using already mapped data. Hence loads of geocoders, routers, renderers, data converters, but nothing really good and innovative on editing front (sans iD, which was really lucky and sponsored by a grant). I really really want all I wrote above implemented. I want to push a button and get up-to-date walking papers with POIs for my street, to go out and find new amenities and update opening hours. After a long drive I want to run JOSM and add lane information, cafes, hotels and petrol stations, house numbers and surfaces I collected just hours ago to the map. I want to make OSM better, but I see no way these tools could be made. We all know the main principle of OpenStreetMap: "You want it — you make it". Maybe some wonderful person would start on that, but given my experience with mentoring "OpenSurveyor" on GSoC, it's not an easy task for students or even novices to OSM. Turned out having a 10-page design document means three months won't be enough.
All I know I can program, I know some of these technologies and will have little trouble learning the rest. Eventually I can finish all that. But as a spare-time job, between hosting an osm radio, writing news posts for shtosm, organizing mapping parties, disputing on osmf-talk and so on, progress would be very, excruciatingly slow. Not to mention I have other challenging ideas like writing a proper changeset reverting web service. So can you recommend me any way to make creating these tools my full-time occupation?
update: if some company (Stamen? Mapzen? Mapbox? Enaikoon?) decides to commit to one of these projects, I'd be happy to translate and update specifications.
Richard is now counting some stats on anonymous ballots from the Board voting, and he persuaded me to publish some other, more complex stats I did on that Saturday. So, here comes.
Out of 219 ballots...
- 79 (36%) have all 8 candidates ranked
- 56 (26%) have 3 candidates
- 23 (11%) have 4 candidates
Richard makes a smart assumption that some people didn't quite understand that one can submit any number of candidates, not one, not 3 (for number of seats) and not all 8. I submitted 4 candidates, because I had strong preference for Board members, and I believe that's the case for most of 3/4 votes. And people who filled all 8 positions maybe are not happy with a tiny chance their vote will be burned otherwise.
Every candidate has been listed at each of 8 positions in ballots (that is, there is no candidate that haven't been assigned e.g. #6 in at least one ballot).
For ballots with less than 8 positions, some of the candidates were not mentioned. Let's count number of ballots for a candidate, where he/she is not included:
- 102 (46.6%) — Steve Coast
- 93 (42.5%) — Ethan Nelson
- 80 (36.5%) — Randy Meech
- 75 (34.2%) — Marek Strassenburg-Kleciak
- 73 (33.3%) — Paul Norman
- 64 (29.2%) — Peter Barth
- 54 (24.7%) — Kathleen Danielson
- 39 (17.8%) — Frederik Ramm
So, nearly half of voters skipped Steve (I wonder why) and Ethan (probably because he is less known than others). Frederik and Kathleen wrote a lot of good, thought-provoking posts in osmf-talk, so I hope that's why everybody were voting for them.
We know Frederik Ramm got 78 first-rank votes, and 23 of them were distributed among other candidates. Whom?
- 27 (35%) have chosen Peter Barth as the second candidate, so he got 8 of these extra votes
- 19 (24%) have chosen Paul Norman, so he got 5.6 votes
(The rest was skipped because numbers are too small). Some of those who gave the first preference to other candidates had very strong preferences for the second place:
- 11 of those who voted for Paul (55%) have chosen Frederik as the second
- 12 of those who voted for Randy (52%) have chosen Kathleen as the second alternative
So it's quite clear why when Randy Meech was eliminated at round 5, Kathleen was immediately elected for the second seat at the Board.
Now, what if we didn't use STV, but went for the first preference? Here are [some of] first candidates ranks:
- 78 (35.6%) — Frederik Ramm
- 32 (14.6%) — Steve Coast
- 30 (13.7%) — Kathleen Danielson
- 23 (10.5%) — Randy Meech
- 20 ( 9.1%) — Paul Norman
You see a bit of difference with final results (Frederik, Kathleen, Paul). So, if there were 4 seats to fill, who would take it — Steve or Randy? Guess again:
- 4: Peter Barth
- 5: Steve Coast
- 6: Randy Meech
- 7: Marek Strassenburg-Kleciak
- 8: Ethan Nelson
The magic of STV! Of course, next preferences played a big role in the actual result. Let's see who was most often mentioned as a backup candidate:
- 43 (19.6%) — Kathleen Danielson
- 41 (18.7%) — Frederik Ramm
- 35 (16.0%) — Paul Norman
- 32 (14.6%) — Peter Barth
Here they are. Let's invent another voting system, where the first candidate gets ½ of a vote, the second — ¼ of a vote, the third — 1/8 and so on. What the results would be?
- 52.5 — Frederik Ramm
- 31.5 — Kathleen Danielson
- 24.9 — Paul Norman
- 22.9 — Peter Barth
- 21.6 — Randy Meech
- 20.6 — Steve Coast
This is where it gets complicated
Why the 5th and 6th candidates differ from STV results for 6 seats? That stems from placements variation. Let's find a most assigned rank for each candidate. For example, Frederik was placed first in 78 ballots, and second in 41.
- Frederik Ramm: 1:78, 2:41
- Peter Barth: 3:41, 2:32
- Marek Strassenburg-Kleciak: 3:28, 4:23
- Randy Meech: 3:24, 4:24, 1:23
- Kathleen Danielson: 2:43, 1:30, 3:29
- Paul Norman: 2:35, 3:34
- Steve Coast: 8:44, 1:32
- Ethan Nelson: 5:28, 4:25
That list is hard to read, so let's erase the ballot count:
- Frederik Ramm: 1, 2
- Peter Barth: 3, 2
- Marek Strassenburg-Kleciak: 3, 4
- Randy Meech: 3, 4, 1
- Kathleen Danielson: 2, 1, 3
- Paul Norman: 2, 3
- Steve Coast: 8, 1
- Ethan Nelson: 5, 4
Steve's positions are interesting: mostly first and last ranks. Frederik is most favored, so 1st and 2nd ranks mostly. Others float around their ranks, not deviating much. Voting results basically reflect that: Frederik, Paul and Kathleen are closest to the top ranks, with Peter following.
We haven't got permission to publish number of normal and associated OSMF members who voted and who were eligible to vote, but we can make some assumptions from the statistics and number of ballots.
- Members: normal + associate = total
- By October: 182 + 336 = 518
- Eligible: ? + ? = around 300 (518 minus 218)
- Voted: 106 + 113 = 219
So if we take number of voters for 73% of those eligible, we get following numbers (again, they are very approximate):
- Eligible: 145 (of 182) + 155 (of 336)
Which means that nearly half of associate members are behind on their membership payment, while most of normal members pay on time. The question is, are they behind because of financial troubles, or because they don't care. If it's the former, then the new policy on associated membership fees would help a lot.
Have you ever printed a map? Clicked a hundred times on "Export" button on osm.org? Installed mapnik or maperitive and spent days configuring a database and customizing a style? Did you wish for a simple web service that lets you select a bounding box and produces a hi-dpi raster or vector image? Well, there is one now. It is called Get Veloroad, for a style it was created for.
On the side panel you choose paper format and margins, add a GPX trace if needed, select style ("veloroad" and "osm.org" are available), image format, and press "Submit". If the server is not overloaded, you'll get your image in a minute, in glorious 300 dpi. SVG files are postprocessed, so you can easily move labels as a whole, instead of separate letters (the most annoying trait of mapnik-generated maps).
Alas, my server cannot fit all the planet, so there are only Baltic countries and parts of Russia and Finland. Everything is open-sourced though, so I hope soon we'll see a worldwide service for producing high-resolution images.