Recent diary entries
TIGER! 'DaveHansenTiger' originally imported TIGER, but 'woodpeck-fixbot' (noderank #1) subsequently touched nearly every node. Because TIGER is such a mess, it may be possible to estimate how quickly it is getting cleaned up based on the last-modified count of woodpeck-fixbot. Currently it's 136 M, going down at around 12 K/day, so at this rate it will take 32 years to clean up all the TIGER in the USA.
TIGER ways: between 'DaveHansenTiger' and 'bot-mode', there are around 8 M imported TIGER ways that haven't been touched since import. At the current rate of 1800/day, it's going to take 12 years to clean it all.
NHD! (USA national hydrographic dataset). A lot of NHD was imported without any decimation at all, resulting in >90% of the nodes being redundant, effectively noise. There are at least 6 accounts involved in NHD import, including 'jumbanho' (noderank #2) and 'nmixter' (noderank #5). I've tried manually cleaning up this NHD mess manually, but it takes several hours to do 100 K nodes in JOSM. At that rate, it would take me 8 months of editing every night to clean up all 46 M nodes.
Canada! The CanvecIimports account (noderank #3) is at 45 Mnodes and still rising, and there are several more accounts that appear to import Canvec like azub (noderank #11), bgamberg (noderank #13). Some areas are neatly decimated and tidy, some are not.
Netherlands: There are two huge imports, 3dShapes (noderank #4) and BAG, which is spread across 16 accounts which all nicely have BAG in their name (Sander H_BAG, Commodoortje_BAG, etc.) All 16 are in the top 200 of noderank.
Massachusetts: The state GIS was a massive import, by account jremillard-massgis (noderank #10) and a few others. Amazingly, the road data is actually of high quality and needs very little cleanup; the wetland hydrography is a bit messier.
Some highly ranked accounts appear to be national imports (?) that I found harder to learn about, such as Tom_G3X (noderank #7, 19 Mnodes in Japan) and Petr1868 (noderank #9, who has apparently added 23 Mnodes to the Czech Republic using "Tracer Using RUIAN and LPIS")
France has many accounts importing from its national cadastre database, but it is very hard to tell which. One might guess that ËdzëronK (noderank #12) and the 15 other massive contributors to France in the top 100 are importing cadastre, but perhaps some of them are actually just amazing, really active mappers.
In my next post I'll talk about some non-import, real cool mappers I discovered.
My name is Arushi Vashist and I joined the HOT community as an intern through the Outreachy program 2015. I will be working on the project "Improve HOT Export Tool Help Documentation" with my mentor Mhairi O’Hara. I am looking forward to having a great experience and collaborating with the community.
In reply to RichardF's call for stories, I shared a few of my mapping stories. Marking the link here, so I get back to it later: http://www.openstreetmap.org/user/Richard/diary/35107#comment30756
Everyone has their own reason for contributing to OpenStreetMap. Maybe you wanted a better map for your favourite activity - hiking, cycling, skiing. Maybe you were using a site or device with OSM data, and found it lacking. Maybe a friend got you involved. Maybe you believe in our aims as an open project. Or maybe you just thought it was cool.
What was your reason?
I'm giving a talk at State of the Map US in just over a week, in which I'd like to share people's stories as to why they contribute. I'd love to hear yours.
I have been actively participating in LearnOSM for a few months now and here is my first report of the community work on this project.
Status of material in English
Sections are being added, reorganised and updated, in particular:
- Coordination: http://learnosm.org/en/coordination/
- Note to new volunteers: http://learnosm.org/en/coordination/nepal-disaster/
- Smartphone app: http://learnosm.org/en/mobile-mapping/osmtracker/
Thank you Nick @Nick-Tallguy for your sustained hard work to improve the materials!
Special thanks to Win @gitGOwin for his various corrections and his full update of:
- Smartphone app v2: http://learnosm.org/en/mobile-mapping/osmand2/
More thanks to Daniel @CloCkWeRX, Jessica @jmarlena, Brad @neuhausr, Manuel @humitos, Blake @bgirardot for all contributions big and small in our GitHub repository.
I also would like to salute people contributing behind the scenes or participating in discussions to improve form and contents globally: @pyrog for the input, Nick @Nick-Tallguy for the cleanup, Jessica @jmarlena, Alex @lxbarth, Mikel @mikelmaron and members of the HOT Training Working Group.
Take a tour at https://github.com/hotosm/learnosm/issues
Improvements in several languages
- Spanish: Translation of Coordination by the great team of Jorge @jsanz, Pedro-Juan @vehrka and @shiguera.
- German: Translation of Detailed Editing and review of Beginner's Guide by Michael @michael63-osm
- Japanese: Care by Satoshi @nyampire
- Ukrainian and Russian Beginner's Guides BOTH by Andrey @Andygol.
- Norwegian: Work recently started by Anders @andersar.
- Swahili: Work recently started by Geoffrey @kateregga1.
Thank you to all our translators!
Stay tuned for what's next and how you can get involved.
Starting in 1996, Seznam.cz is nowadays the biggest web portal on Czech internet. The portal was originally search engine (today number2 in the Czech republic with share rate similar to Google), in 1998 the freemail function was added and since 2001, another functions were added, including maps.
Mapy.cz originally used data from Seznam for the Czech republic and from Navteq for the rest of the Europe. Another parts of the world were not rendered on usable zoom level.
As of the end of the year 2014 they switched from Navteq to OSM in the Europe and added zoom level 15 and 16. They earned 1.5milions of POIs (originally 1milion, 2.5 with OSM).
In the Q1 2015 they started to test in beta rendering of the whole world with OSM data and today their product left beta stage and now is available on their homepage.
OSM data are used everywhere except Czech republic and Slovakia (two countries Seznam.cz is focusing mostly - they are collecting data by themselves and have some business plan with them). The only render is the default map, the other ones are available in Czech republic and Slovakia only (with promises of tourist map for favorite tourist destinations). There is also no Panorama (StreetView like) functionfor the rest of the world nor aerial images.
Seznam.cz promises not only taking the OSM data but also actively contribute: if someone mark a mistake on the map (outside Czech republic and Slovakia), they will repair it in OSM. I have no information if the bugreport needs to be done in Czech or other languages are also acceptable.
Round 10 of Outreachy has started and I will be working as an intern during the summer on the Tasking Manager of Openstreetmap with Blake Girardot as my mentor.
In a nutshell, Outreachy is a program that encourages women (cis and trans), trans men and genderqueer people to make their first steps in free and open source software. For three months an intern is paired with one or more mentors for a project. This internship not only provides space for people from underrepresented backgrounds, it introduces new and enthusiastic helping hands to community-driven projects.
Openstreetmap is a worlwide community of mappers that collaborate on collecting data for a digital map. It is user-friendly software that respects the user's data. It has Open Database License and is community driven and in that way is setting an example of what self-initiative and collective collaboration can offer back to our communities.
Usability testing for the Humanitarian Openstreetmap Team's Tasking Manager
Humanitarian Openstreetmap Team's (HOT) Tasking Manager is a tool used for dividing mapping projects for humanitarian causes. In June 2014 (update: corrected release date as suggested from the comment) Tasking Manager 2 was released. There are currently attempts made to improve the Tasking Manager Usability Interface.
The Outreachy internship started yesterday, Monday 25th of May. This is the schedule for the first week of the project:
- Introduction to the community
- Understand workflows, participate in online meetings
- Review open Tasking Manager issues
- Have the local repo up and running
- Make a pull request
- Populate a list of tasks a user does in order to map
- Get more familiar with the Tasking Manager front page (priority status, project selection etc)
The last new version of Vespucci with major changes was at the end of 2014 and it has taken a lot more time than I wanted to get near to completing the next version. But now a lot of things are coming together and I'm expecting that we are only a single digit number of weeks away from a stable release.
Just some of the major features:
- multi-select mode
- restyled tag editing with split pane layout on tablets.
- selective tag copy, cut and paste.
- lots of relation editing improvements
- start a camera app directly
- go directly to preset screen when creating new elements.
Besides the multi-select support most changes have been geared towards making the user interface more consistent and easier to use.
Given that the new property editor is so much better, I've added a way to quickly and easily switch in to the "tag editing-only" mode that has been hidden in the deprecated modes preference for the last two years. This should not be confused with the current raft of "lets add a node regardless of what is already there" pseudo editors, all available OSM objects can be modified, just the geometry is locked.
For developers: one of the major hills that had to be climbed was refactoring the old "TagEditor" activity, likely some of the oldest code in Vespucci. Written by the original Vespucci author in the first months of 2009, by both OSM and Android standards it was pre-historic. I added a significant amount of code two years ago for relation support, but hadn't touched the general structure and monolithic mass of code that it had grown to, and that had started to be a big road-block to improving and adding functionality. One immediate benefit of the refactoring is that Vespucci is much improved on tablets and uses the available screen real estate a lot better.
Watch this space or twitter for the announcement of release candidates, as always any help in testing and coding is welcome.
Use favorites in OSM-IQ to mark tags you are interested in and export all your favorite tags to a different layer in a single DXF file.
For quite a while I'm using oocalc to correct errors in osm.
With a few macros you can jump from a table cell to josm, osm webpage and other services.
Read more in the wiki: http://wiki.openstreetmap.org/wiki/LibreOffice_Calc
In humanitarian Map, Jammu and Kashmir you are making as part of INDIA. According to UNO, It is part of Pakistan. Such propaganda is against Pakistan.
Please correct the map according to UN.
After reading a diary entry about noexit=* Tags a few days ago I check out my area and...
I started to clean up my area using Osmose (which recently became available in Germany and really is an awesome QA tool!) and it's slowly getting better - have to wait for Osmose to update what I already fixed.
Many people seem to put it on ways, which is bad for routing as I found out when fixing some stuff for leitstellenspiel.de. Many tags also just seem to be very old and somehow just ended up in the wrong places by splitting streets and others are there even thought there is a foot way at the end of the road.
Use the noexit=yes tag only on the very last node Node at the end of a highway=* or on the last way Way to indicate when doubtful that the impossibility to travel further by any transport mode is perfectly normal, due to otherwise existing road layout. * http://wiki.openstreetmap.org/wiki/Key:noexit
Fix the issues in your areas with Osmose!
Last Update: 27th May 16:45
- 49A / 49B
Now I have to once again go through all the lines, andmark the individual stations as stops with all lines ... In a few lines, there are bus stops in the middle of the street ...
2: Regional Bus lines
[EN]: I just discovered how to adjust the bus lines for Overpass with Josm! The relations (stations) must be in the correct order. Then everything works. With the knowledge I will (after I adjusted the individual stations of the lines to the new schema) customize all lines (1A-99B) for Overpass. If there is still time I will also adjust the lines 106, 124, 125. The line 50B I will draw about 2 weeks prior to 1.7 to the then valid, new route, and the line 150 will be renamed in 50A. The line 49E I will also put onto the map, and will delete it again at the end of the rail replacement bus service... Also the "Bäderbus" Gänsehäufel is now in the system ...
Thoughts on the import of address nodes in JOSM using MassGIS L3 Parcel data to aid in adding addresses to buildings in MAPosted by xunilOS on 23 May 2015 in English (English)
One major downside to fully using Open Street Map as a complete alternative to other map services (at least in the US) is the lack of address data. I strongly believe this is tied to the slow tedious process of adding addresses. I myself have spent many hours adding addresses a variety of methods beginning with driving by with a dash camera in my car and reviewing video feeds later. I then discovered the massGIS L3 parcel data overlay in JOSM and began
- Selecting all buildings in a town adding city, state, postcode to them
- Selecting the houses on a street , adding the street
- Going through the street to add the housenumbers
- And repeat for other streets uploading changes at the end of the day
- When I was done for each town, I would remove the city, state, and postcode from buildings that had no housenumber associated with them (I.E. garages or sheds on peoples properties that got captured in the first step).
While this last method was considerably faster, I still thought at a minimum there must be a way to add all these addresses to OSM as individual address nodes. After some research and a preliminary testing I was able to create a few scripts to convert the data into an osm file that could be opened with JOSM.
[UPDATE] As I was rightfully pointed to the import guidelines wiki, I am rewording this post as a proposed method and proof of concept to gather general thoughts before implementing.
High level overview
This section lists the high level overview of the process I would like to gather thoughts on. For more details see below
- Download the scripts L3ToOSM scripts
- Download MassGIS ESRI shapefile for the town
- Export shapefile database to csv
- Run script to convert to OSM importable file
- Import into JOSM for editing and merge address data with buildings
- Remove temprary address nodes
- Uploading to OSM
- Validation and error checking
Below are detailed steps describing the process for importing address nodes into JOSM that I would like to gather thoughts on.
Download the scripts L3ToOSM scripts
Head over to my github project to download the project and read the README for software dependencies. [UPDATE] Perhaps an OSM file containing the address data as nodes could be provided instead for aiding in address additions for local folks. This would allow for skipping down to the import into JOSM step.
Download MassGIS ESRI shapefile for the town
- Download L3 parcel data from the MassGIS website - the zip file to download is under the ESRI Shapefiles column named something like L3_SHP_MXXX_.zip
- Unzip the file to a temporary location on your computer.
Export shapefile database to csv
- With Libre Office Calc (or Microsoft Excel - untested) open the unzipped directory and look for a file named MXXXAssess.dbf (Should be the largest filesize)
- A popup will ask what character set to use - i selected the first one Western Europe (DOS/OS2-850/International) though you can select Unicode(UTF-8) as well.
- Delete the columns so you are left with state_plane(LOC_ID),Housenumber(Addr_Num),street,city,zipcode
- Click file -> save as
- Choose a directory to save the csv file (it may be best to use the same directory the scripts are in)
- Make sure to select the Text CSV (.csv) file format
- Click save -> In Libre Office you may get a warning asking if you want to use ODF format, just click "Use text CSV Format"
- IMPORTANT: In the "Export Text File" box, make sure to change the "Field delimiter" to a colon ":" and click ok to save it.
Run script to convert to OSM importable file
- Open a terminal and cd into the directory where your script and csv file is saved.
- run the script with the following arguments: l3ToOSM.sh "" Where STRING_TO_SEARCH is captured in quotes and is data that will be added to a note for each address to be used for deleting address nodes after they have been merged with buildings. This is to prevent duplication of data and it is better to have address data tagged with a building than floating around as a single node. Make sure this is unique (ex: xunilOSMassAddressUploadDeleteMe) NOTE: For now this script only works for Massachusetts L3 Parcel data and in the mainland, not the islands like Marthas Vineyard etc.. For more information read about the State Plane Coordinate System specifically the state plane zones.
- The script is very fast and should complete within a second or less. Please note that the street data in the L3 database shortens the names. Ex Road would be RD and Street would be ST. I have added logic into the script to handle this, however it may be possible that I could have missed an abbreviation or two. See the below section on Validation and error checking on how to handle this.
Import into JOSM for editing
- The script should have created a new file named MXXXAssess.csv.osm. This file will contain all the address nodes from the L3 parcel database file.
- Download a section of the town from OSM
- Open JOSM and click File -> Open
- Select the file and click open
- You should now see all the address nodes for the town in question.
- Click on the address layer and make it the visible layer
- Select the address nodes that correspond to the section of town you downloaded from OSM
- Press CTRL+Shift+M to merge the selection to Layer1 or whatever layer contains the OSM downloaded data.
- Add the L3 parcel data as a background as this makes it easier to tell what buildings to merge the address nodes with
- Optional: Add a filter to hide any buldings that have addresses associated with them. I use "addr:housenumber"=* building=yes
- Select an address node and press CTRL+C to copy the node
- Select a building to merge the address data with and press CTRL+Shift+V to merge the two
- Repeat the last two steps until done for the day and ready to upload
- I occasionally check the address data to make sure it is in line with the L3 parcel background data and street names as good habbit.
Uploading to OSM
- Prior to uploading to OSM it is very important to remove the address nodes to prevent them from being duplicated on the map
- IMPORTANT: If you created a filter to hide building with addresses, temporarily deactivate it
- Click edit -> search
- In the search field look for type:node note="xunilOS ImportAddrPleaseDeleteMe" Where note="" is what you eneterd in the STRING_TO_SEARCH argument.
- Verify only the address nodes are selected and press delete.
- Click edit -> search again to remove the note field that was also merged with all the buildings you added address data to.
- In the searchfield enter note="xunilOS ImportAddrPleaseDeleteMe" Where note="" is what you eneterd in the STRING_TO_SEARCH argument.
- In the properties window on the right select the note property and delete all the notes containing the STRING_TO_SEARCH argument.
- Finally upload the addresses and resolve any errors that the validator may find.
Validation and error checking
After uploading to OSM, it is important to review the work to see if address data could be incorrect in case it was not caught prior to uploading to OSM. Ex a street name was not found etc.. The OSM inspector website is a great tool for such a task.
- Go to OSMInspector and click addresses under the view dropdown.
- Zoom to the town you would have been working on and resolve any errors that can be found. Typically this may include a misspelled street name etc..
I appreciate any comments or concerns.
As a bloody OSM newbie I quickly realized, the strength of this crowdsourced project is not (only) freely available geometries, but the database in the background. As far as I'm aware, OSM is quite influencal in the mapping business and we may take our chances for pioneering tasks.
With the last OSM carto release (v2.30.0), bicycle_parking has been finally made available for display by default on the main map. Great! But why omitting charging stations? (such as this here) Those are not even available on the cycling layer, why? This is something I really, really would love to see added sooner than later! Cities want to get rid of cars, there are lots of efforts/incentives around the world to achieve this. E-bikes are quite a thing by now - but I can imagine people fear the lack of appropriate charging stations (same applies to car charging stations).
Here in Vienna alone are around 300 charging stations available - why not show people: hey, go ahead, get your E-bike...there are plenty of charging stations around you! This wouldn't congest the map by no means, but would be really far-sighted in terms of social responsibility. At least, this should be added to the cycling layer. (I'm aware of things suchs as Overpass API for creating own specialized maps...but I think this might be of general interest)
However, as I'm new here, I have no idea how to address/forward such suggestions to persons in charge - but I hope this is also on the agenda for future updates!
I made this viz. It really made me think about the amount of influence the casual editor has in OSM. Sometimes I feel bad that I don't have as much time as I'd like to have to contribute.
However, after seeing this, I know all contributions are valued - no matter how small and no matter how occasional.
Keep it up community.
I have been making some good progress to date in my tram stop tagging adventure. Three tram routes have been completed (109, 48 and 16) and a fourth (72) has been started.
Doing the route 16 (Kew - Melbourne University) was an interesting experience. Despite living relatively near the Kew terminus, I had never taken the tram down towards St Kilda. I was surprised when adding stop tags at its convoluted (and incredibly long) route.
This strange route is a relic of history. The present-day route 16 is an amalgamation of two old tram routes: the old route 16 and route 69. The 69 ran from the present Kew terminus to Luna Park in St Kilda, while the old route 16 ran from Melbourne University to St Kilda Junction (Acland St) via St Kilda Road.
In any case, I managed to cover all stops along the route, although again I was surprised at some of the errors and omissions along the route. Some stops, even in the middle of the CBD, were missing from the map entirely (e.g. the Swanston/Bourke stops), while some were just in the wrong place (e.g. Havelock St, St Kilda).
- Route 72
- Route 16
- Route 109
- Route 48
How to use Notes?
Whenever we tag something there are a lot of key=value pairs we can choose from. One of them is the note key. With openpoimap I was investigating the use of this tag. Below you see a screenshot of Berlin with the note's that were made on a great number of nodes.
IMHO the use of a note is twofold:
- The note is used to clarify something that cannot be displayed with any other key=value. E.g. This note tells me what I need to know about Fritz Schloss.
- The note is used as a reminder to the mapper himself (or to others) that one is not sure about what is displayed with the other keys. E.g. some more research is needed. This use of a note is more or less the same as the fixme/FIXME key (which for mysterious reasons is available both in lower- and uppercase). Here is an example:
But now have a look at this: The note is exactly the same as the name tag for that node. Why is that? Probably the mapper first wrote down that note as a reminder and later decided to use the note text as the value for the name, but why not remove the note? It is completely without any use!
So, mappers, whenever you make a note to yourself, remove it once the case is clear!