OpenStreetMap

What I want

Posted by Zverik on 12 November 2014 in English (English)

Hi. I often drop hints about what our project misses, and now want to talk some bit more about a part I'm interested in. Since I joined OpenStreetMap, I have been interested in geodata collecting methods. I quickly grasped walking papers, put my GPS and camera to use, and struggled with georeferencing audio recordings. OSM allows for many types of sources, and in past years a lot were invented. But alas, not much in last years.

Walking papers (or field papers) are still produced from tiles. Fieldpapers.org, the "modern" service for generating atlases, is more than two years old and is a slight update to older walking-papers.org, built in 2009. It stiches tiles and produces a PDF file. You have no control over map style, you can't even use your own tiles. "Toner" style, which is the best option, are updated infrequently: you might have to wait a week before traced buildings appear on it. And some of them still won't, since it's hard to grasp how it works and why it hides some features unpredictably. Finally, pages of an atlas will be oriented by cardinal directions, in a grid with 90° angles. Of course, not many towns have such proper road network, so you will have to choose smaller scale, with less effective area for mapping.

I think I can fix this. It is easy, really: most of building blocks for a proper solution are already invented. First, an interactive map, on which you place rectangles for pages. Arbitrarily, not neccessary in a grid. Maybe draw some lines, which would be "pie segments": instead of using MS Paint for making a pie, use some advanced technology. Maybe integrate it with MapCraft. So, a bunch of rectangles on a map. Not 90°-aligned: rotate them as you like. Align with road network, with streams etc, so areas for filling in take as much space as possible, and scale is biggest you can get. When done, just save your work and close the website. Go trace some buildings and tracks.

When a morning of a mapping party comes, open the website and press "Create atlas". It will display the progress, but the atlas will be made on a server asynchronously. First, it downloads an area from OSM API. Yes, not from a local postgis database, only fresh data. An added bonus (for local installation) is that you can use cached osm data, or just bring it from another computer, if an internet connection is weak. Then it applies a MapCSS style (which you can customize, even upload some of your own or josm's) and renders each page, rotating data as needed. Then it joins pages into an atlas and provides a download link. Later the atlas can be rebuild, using fresh data, maybe a different style or with more pages.

I don't believe in scanning pages for using them as a layer in JOSM (Bing/MapBox imagery make for a better reference layer), but georeferencing marks probably can be included. That won't be the greatest feature, because I have some other thoughts. You know the weakest point of the OpenStreetMap mapping in 2014? Updating data. It is quite easy to collect and map new roads, new POIs, new restrictions. But updating it is very, very hard, almost impossible for rich regions. There are no tools. My theoretical walking papers can fix that issue. Since we have full control over data, we can put POIs and relevant tags right on pages. We have a second side: for example, on the map there would be dots with coordinates (A4, D9), and on the reverse side — tags for each dot. And the same for ways and maybe relations (didn't think it all through, obviously). So you can have not only a base map for collecting new data, but also a check list for updating the already rich map.

This solution will make mapping easier not only in thrid-world countries (where internet is sparse and you can't rely on external web services, or spend days installing tile rendering stacks), but in densely mapped cities, where data has not been updated for years, because it already seems well-mapped, why go there again.

The next step would be an Android application for mapping. Why android? Because I again keep in mind mappers in third-world countries, who can get an android phone for $30, but not an iPhone for $300. So, let's take Vespucci. It is a powerful editor, getting better every months thanks to Simon. It can download an area and let you edit it. But can it work without internet and GPS? Not likely. Can it be used on a mapping party, when you are passing 10 points of interest a minute? You'll be exhausted in half an hour. The ideal mapping solution will have to separate data collection and data processing. Step 1: go out and record everything. Take photos, record audio, type a hundred house numbers, draw some crude lines on a touch-screen, like you do on walking papers. The interface should allow for quick mapping, e.g. in a car: you see a sign — you press a button and leave moving a mark to a correct place and tagging it for later. KeypadMapper and OSM Tracker are examples of this approach, but it should be made more streamlined and consistent. You cannot rely on GPS, for it has low precision, and not available on cheaper phones (off go those two apps). You cannot rely on the internet to provide you with tiles (off goes OSMPad). But you can assume you'll have a chance to download 100k of osm data back at home (or 10-20k on the road), to use as a base map for further mapping.

Step 2: Upload collected data somewhere (e.g. on your computer), process it and update the map. Data format can be universal, which means some central server for storing all the information. Audio notes can (and should) be converted to text, GPS trace joined from broken segments, data split between days and so on. Since most of points would be in tags or other non-textual representation, so anybody can use it for mapping, you would have an option of upload it to a server (right from your phone), so someone else could map it in their spare time. Or it could be you, download the data pack in JOSM. And then — map it. Recorded points shown as icons, with copypasteable tags; photos already georeferenced with subsecond precision, notes written on a map, crude drawings also georeferenced as underlying layers, between OSM data and imagery. It would be so much better then trying to remember what you meant by these waypoint titles, or having to read your handwriting on walking papers. And technical requirements would be as low as they get.

I am not a great programmer, or a designer. But I know what and how should be done to drastically improve mapping in OSM. No commercial company would make it, because they make profit not from mapping, but from using already mapped data. Hence loads of geocoders, routers, renderers, data converters, but nothing really good and innovative on editing front (sans iD, which was really lucky and sponsored by a grant). I really really want all I wrote above implemented. I want to push a button and get up-to-date walking papers with POIs for my street, to go out and find new amenities and update opening hours. After a long drive I want to run JOSM and add lane information, cafes, hotels and petrol stations, house numbers and surfaces I collected just hours ago to the map. I want to make OSM better, but I see no way these tools could be made. We all know the main principle of OpenStreetMap: "You want it — you make it". Maybe some wonderful person would start on that, but given my experience with mentoring "OpenSurveyor" on GSoC, it's not an easy task for students or even novices to OSM. Turned out having a 10-page design document means three months won't be enough.

All I know I can program, I know some of these technologies and will have little trouble learning the rest. Eventually I can finish all that. But as a spare-time job, between hosting an osm radio, writing news posts for shtosm, organizing mapping parties, disputing on osmf-talk and so on, progress would be very, excruciatingly slow. Not to mention I have other challenging ideas like writing a proper changeset reverting web service. So can you recommend me any way to make creating these tools my full-time occupation?

update: if some company (Stamen? Mapzen? Mapbox? Enaikoon?) decides to commit to one of these projects, I'd be happy to translate and update specifications.

Comment from guanchzhou on 13 November 2014 at 11:54

Wish you all the best to find a full-time OSM employment. By my opinion, it will be very good not only for you but for OpenStreetMap as well.

Hide this comment

Comment from escada on 13 November 2014 at 16:51

good ideas. hopefully someone with the time and resources to develop it will pick it up.

Hide this comment

Comment from Randy Meech on 14 November 2014 at 03:16

Mapzen might be into this. Let's chat: randy@mapzen.com

Hide this comment

Comment from SimonPoole on 14 November 2014 at 15:31

Just a note on the side: vespucci can be used fully offline (it can save and load JOSM compatible .xml files on the mobile device, and tiles can easily be preloaded) and 10 POIs per minute might be stretching it, depending on the level of detail one wants to add, but is in principle possible.

Hide this comment

Comment from flohoff on 22 November 2014 at 16:18

I'd vote for using the MapOSMatic stack for producing the FieldPaper PDFs. Lots of stuff has been solved and its easy to set up. Only the rectifying markers + QRCode needs to be put in there but then you'll get a 99% Vector PDF which is smaller and scalable to any paper size you'll decide to print on.

Hide this comment

Comment from Mark_Cupitt on 25 November 2014 at 05:00

Excellent set of thoughts.

I have been considering a Mobile App exactly as you describe it. I live in the Philippines and am in total agreement regarding the use of Android. My super Duper unit cost be less than $100.00 and is way smarter than me :)

Am now looking at vespucci, first I have heard of it. and what a wonderful effort on the developers part. Congratulations.

I bought my thoughts on this type of tool up for a potential RHoK effort next year with some level of interest.

here is what I think is needed.

I feel at a full on editor is not needed, but an easy to use Android app that knows you are moving on a road, can tell you you are on an unnamed road and ask for a name, knows you are walking alongside a road and can offset the track to the road centerline, asks for the type of road to tag correctly, grabs Poi's on the correct side of the road, based on where you are standing and adjusts the position if needed, and so on.

I want to hide the use of tags from a users perspective, every day, non technical users are a HUGE source of data, especially in developing countries. I think an app can be set to work in different modes, tap for road mode, tap for poi mode. The app should know you are close to an existing node or way, check for data that is already available and prompt for needed basic data, like one way, speed, nbr of lanes, road type, all using simple and quick option buttons. I know this limits the amount of data, but these could be preset into Different Modes, Initial Mapping Mode and advanced Mode that could ask for more detailed data for power users.

I know for a fact that kids now days would gladly use a non technical app to supply data. Heck, even make it fun with Points awarded for effort and amount of data and build a community around the effort.

The idea is that the data would be dumped straight as a change-set into OSM, no intermediate editing. required

Wishful thinking, maybe, but I intend to have a shot at it next year. I was looking at adapting OsmAnd but maybe vespucci might be a better candidate.

Hide this comment

Comment from ImreSamu on 25 November 2014 at 14:32

for our Budapest Wheelchair mapping party ( 2014.August ) , I have created a basic prototype for Field Paper for city center ... but we don't used

  • because we have to translate ( to hungarian) the key-value parameters for new mappers
  • and it was too complex for the beginners

--- Sample what I did ---


What I have done :

What I have learned:

  • We need special carto rendering style - because not easy to find a POI with a similar name. ( like metro entrances with same name )

  • we need translate key-values - to beginner - and not english - friendly . My tipp : using similar view like "iD Editor" POI view ( because it is translated every language and beginner friendly )

Hide this comment

Comment from Alan on 27 November 2014 at 17:23

These are very interesting ideas. At Stamen we're assembling a wishlist of things we'd like to add to Field Papers if/when we get another grant to improve it. There is a lot more work we want to do on it. Many of your ideas would be great as extensions to Field Papers, while others sound like they might be stand-alone projects.

For now I encourage you to add some of these ideas as issues on the Field Papers github repo: https://github.com/stamen/fieldpapers/issues

Also, if there are any small improvements you would like to make to the Field Papers code, we will happily accept pull requests and can deploy changes to the live site once they have been tested.

Hide this comment

Leave a comment

Parsed with Markdown

  • Headings

    # Heading
    ## Subheading

  • Unordered list

    * First item
    * Second item

  • Ordered list

    1. First item
    2. Second item

  • Link

    [Text](URL)
  • Image

    ![Alt text](URL)

Login to leave a comment