Recent diary entries
Today is another being-part-of-the-solution kind of day. We (a few colleages at Telenav were involved in this) took the output of the Battle Grid comparison we use to generate the Battle Grid cells, divided them into more and less severe cases based on the road class and amount of difference between TIGER and OSM, and created a MapRoulette challenge out of the more interesting cases. This will make it easier for anyone to help fix antiquated, bad TIGER data anywhere.
This is what a typical case would look like in MapRoulette:
Loading this case in iD, it's plain to see that this road is mis-aligned pretty badly:
Easy to fix! Just drag the nodes that make up the misaligned way to match the aerial image:
To point out these errors, we compare TIGER 2014 against OSM data. This gets us mostly good results, but it's not perfect. In most cases where MapRoulette will point out a non-error, the TIGER data is still wrong. This is the case in Alaska more than anywhere else, it seems.
TIGER is wrong here, even though it's hard to see with the low resolution satellite imagery...
The number of non-errors is small enough to not be too annoying, I hope. If it is let me know, and I can see what more we can do to improve this challenge.
Not all at once
There are a lot of Mismatched Ways. A lot. Most of them, way more than 90%, are local and low speed roads, mapped to
residential in OSM when TIGER was imported. Of course, these all need to be fixed eventually, but let's focus on the more important roads first. Based on how it goes, I will gradually add more cases.
I look forward to hearing from you about this new challenge! I am pretty excited about it!
First day doing this. Added some trails that are very important to me (and my dog) in a patch of very ecologically rich woodland/wetland. Hope to indicate that this is not waste land, that it is a place that is used both by people and by wild organisms. Looking forward to more exploration of mapping techniques.
Try to c what's going on
The way in question, 55984002, had been tagged with the height restriction of 15'-3" since UrbanRambler added it on 20 April 2010...
...Yet Google doesn't even have a facility to put in height restrictions...
...Another reason to switch to OSM.
As I write this, I am wrapping up a year on the OpenStreetMap US Board of Directors. The Ada Initiative has been a huge asset to me in my work on the board. Without the anti-harassment policies and other resources found on the Geek Feminism wiki, I'd have had a much harder time pushing to make State of the Map US, and the OSM community in general more welcoming to women. If you've been following the OSMF-talk mailing list at all over the past two weeks, you know our work is far from over.
I have donated $256 to the Ada Initiative, as a token of my appreciation for they work they have done in making my own job easier-- I challenge you to follow suit and help me raise $2048 for The Ada Initiative from the OpenStreetMap community. Their annual fundraising campaign is ending on Wednesday-- donate today to make a difference for women in FOSS and Open Culture, especially OSM!
A tough fix to the Kennet centre in Newbury (begun). the offset problem on the structure is quite severe and contributers don't seem to agree on it I picked the roads and mapped the whole thing in one offest setting a lot of features need to be shifted and converted to indoor types. I've pdf's of the new shopping and residentual area to the north but rasteriseing and importing is delay behind other things and not being able to use the latest josm anymore...
OpenStreetMap Carto v2.22.0 has been released. This release focuses on labels.
The biggest change is a rewrite of landcover labelling. A landcover label is text connected to a background colour or pattern rendering, and not connected to an icon. This has been demoed extensively, and well received. It was also sent to the mailing list. The big changes are making colours better connected to the background, rendering labels on some features where they weren't before, and sizing labels based on area.
The last deserves a better explanation. Previously, the selection and choice of what labels to render didn't include area. It now does, avoiding placing labels on features that are only a few pixels in area at low and middle zooms, and selecting font size based on feature area. This results in a much more sensible label placement, more readable labels, better selection of what labels to place, and in many cases, more labels without impairing readability.
There remain some minor issues that can be followed up on (e.g. Glacier label colours)
Other changes were
- Ordering fixes
- Concurrent ferry rendering
- Line wrapping improvements
- Road label improvements in complex areas
- Small island improvements
A full list of changes can be found on GitHub.
Yes, these house numbers in Portland really have leading zeros. As the numbered avenues increase from east to west, they increment by 100 for every block. For example, buildings between SW 19th Ave and SW 20th Ave are between 1900 and 2000. In the city center, SW 1st Ave is very close to the Willamette River. However, as you head south the river bends leaving a bunch of streets that are east of SW 1st Ave, but still west of the Willamette river. Mathematically speaking, it might make more sense for them to have negative house numbers, but I guess just adding a leading zero has the same effect.
Looking at various maps providers, these are often not really handled correctly and the leading zero is often not preserved. It will be interesting to find out if there are any addresses where the only differentiating factor is the leading zero. The hilly terrain prevents building houses on both side of the SW 1st Ave axis in most cases.
Hi, I am trying to map a course travelled on a long trip, does anyone know how to do it? Andrew
Keeping OSM updated and in sync with the real (changing) world is in my opinion one of the key challenges for OSM. Adding new roads and POIs is fun - keeping them up to date is less so!
Have you ever come across POIs or roads and made a mental note to yourself to check later in e.g. JOSM if they are on OSM and tagged correctly? I have, and unless I document in detail what I am seeing with photos, there is a good probability that I miss some essential details. Also with photos, I often have lots of photos recording stuff that is already correctly tagged in OSM. Thus the hit/miss ratio is becoming smaller and this will continue as OSM mature.
To help myself spotting differences between the OSM database and the real world I wrote a small Android app that can display OSM data for nearby objects. With this its easy and efficient to be on the lookout for OSM updates.
The app looks like this:
and it can be found on google Play: OSMfocus
I'm hoping this will be useful for others and that it will results in more OSM updates.
I am just reposting a blog post I just published to make it easier for people to just add their business to OSM.
Together, we resolved more than 100 thousand suspected 'sharp angle' errors in the Ways Needing Smoothing challenge, and just under 25 thousand crossing ways in the Crossing Ways challenge.
So at least for the United States, we now have much less of this:
.. and many less situations where two ways cross but should really have an intersection node. (Sorry, in this case an image does not say more than a thousand words..)
Not all suspected cases turned out to be real errors. For the crossing ways, about two thirds of the cases were eventually marked as 'fixed' by MapRoulette users:
For the Ways Needing Smoothing, the fixed rate was much lower, about 40%:
This low rate can be explained in part by MapRoulette users fixing more than they are asked to fix! What I heard a lot is that mappers would load a smoothing task in their browser, and ending up spending a lot of time cleaning up the area around the error - often resolving other MapRoulette tasks without even knowing it. So it's not as bad as it looks, and I received mostly really positive feedback about this challenge.
I have a new challenge in the pipeline that also has to do with TIGER fixup - a separate diary is forthcoming about that!
As always, if you have a good idea for a challenge, don't hesitate to get in touch with me so I can help you get started! Better still, subscribe to the MapRoulette mailing list and share your ideas with the MapRoulette community!
tried to add some sense of features that are in the north west harbour area of gilbraltar and there effect on what seems at least on OSM as a disputed border.I hope I managed to get the relationships right though it would be good for someone with more experiance of these to check it went in the right order.
As an active firefighter I of course know about OpenFireMap. And I knew of highway=emergency_access_point way before the Weekly Task caused many more of them appearing in the database. But I always wanted to have something that is not a website, and something where I can customize what is shown.
Using the awesome library of Marble I was able to hack together OSMhyd, a Qt-based application that shows fire hydrants, water tanks and emergency access points. Of course this is far from complete, but maybe it serves someone else as inspiration.
If you are interested a bit more in the technical details and future directions I've written a bit more about that in my KDE related blog post.
During four weeks of which I hiked about three I used mapillary as described in a previous blog post.
The result were 113.384 pictures with 596 GB size (on ext4 – on exfat it was 623 GB).
The maximum of pictures recorded during a day was 9000, the average per day 4500.
After review 101.261 pictures with 561 GB size (ext4) remained.
Hardware used (lent from the Mapillary company):
- 2x 128 GB microSD card
- 2x deltaco Powerbank 10 Ah
- 2x short cables USB-A–µUSB- ~20 cm
- 1x long cable USB-A–µUSB- ~100 cm
- 1x Sony Xperia Z1 Compact (w/o SIM card)
- 1x external 2,5" USB 3.0 drive 2TB
- 1x OTG-Ycable (purchased on my own)
- Anker 40W charging station (Amazon OSM affiliate link) with 5xUSB with 2A output each. My property, can really recommend it.
As of today I finally sent all the stuff back to Mapillary including the images which they will add to the database on-site. I'd need about 90 days to upload them…
On the way
I used the phone with the app blindly: Start the app and start recording, put it front-to-back inside a phone cover which gets attached to the left shoulder strap of the backpack. One of the Powerbank batteries I put into the lid of the backpack, attached the long USB-A–µUSB to it and fastened it at the left shoulder strap. So the phone could be easily charged while walking. I mounted it upside down to connect it more easily to the cable but that doesn't matter since the app writes a orientation flag into the photos EXIF data which tells how the picture is to show.
Due to the cumbersomeness of disconnecting, dismounting and peeling the phone out of the cover to switch to panorama mode, take some pictures, switching to walking mode, start recording, putting it in the cover, mounting and connecting it to the charging cable again I mostly did not do so.
At viewpoints or when making short rests I usually just stood there, had a look at the landscape and turned myself slowly around my own axis to let the app make images of the panorama. Only rarely (at big rests on mountain peaks or when walking a city on mapping purposes with the phone in the hand) I bothered to take pictures in panorama mode.
I just hope that there will be software intelligent enough which can detect little to no movement (based on GPS data) and overlapping image contents to stitch panoramas automagically.
At the hotel
After four days both powerbanks and the phone would be discharged so I had to spend a night at a place with electric power supply. Once I was five days "in the wild" so the last day went without Mapillary pictures.
Since I left the magnet charging cable at home due to bad charging performance and for reducing weight I had to calculate when to charge the phone and when to copy data since both at the same time doesn't work when using an OTG cable. When I intended to leave the hotel after one night it was a good idea to have the phone's battery charged in the morning so I tried to reach the hotel with a still quite charged phone, start copying as soon as I was in my room and before going to bed started charging the phone. Since I had two 128GB µSD cards with me capacity was the smaller problem. It really would have been easier if I had taken the magnet charging cable with me.
Nice side effect: I was also able to copy photos from my camera to the hard disk.
What I experienced so far on
I was lent the phone with two 10Ah batteries. With them it was possible to cover four days of hiking. But all the time the phone's statistics say 24% of the energy are consumed by the display. In my case without any use because the phone is mostly operated blindly – I could have done without display.
I installed the app BlackouT which dims the display but does not reduce the energy it needs. The app "screen standby" I found too, but to get it working one has to have the phone rooted and Busybox installed. Though the latter I managed I wasn't able to get root and the app working (during the trip in a hotel room).
Several times I had to reboot the phone since the app told me it could not access the camera. Not so often I had to reboot it due to the lack of recognized GPS satellites. Switching off the phone during the night reduces occasional reboots during the day, but does not eliminate them.
Although the camera is said to be a good one, it may only be a good one for smartphones. If you don't walk in the bright blazing direct sun there is a good chance the image is blurred due to slow shutter while moving.
Sometimes the phone got so hot it showed irreproducible behaviour. The only possibility was to reboot it.
The cover of the micro USB connector I attached and removed quite often. Soon the fitting didn't fit anymore and a little later went to pieces. Anyway I didn't dare to check if the phone is really waterproof. I preferred to have pictures during all the holiday to a drowned phone. The "worst" weather I used it was very light rain where I attached it to my rain coat in which I had glued a strip of loop fastener, too.
Software (Mapillary app)
Since I am no smartphone user at all I am not sure if this issue depends on Mapillary or a dedicated photo viewing app. Displaying the pictures taken is fine as long as there aren't too much of them and you want to see the most recent taken pictures (remember I took per average 4,5k pictures a day). You have to scroll down all the way by hand, you cannot grab the scroll bar and pull it down. And after you had a view at an image and went back to the list you are at its very top again. Even switching from displaying horizontal to vertical lets the list go back to the first image.
As mentioned above, sometimes the app could not access the camera and crashed.
When I experienced this live it was not too bad, but when it happens during recording I didn'tt realize this and walked on taking no more pictures. Even when I set the app to "make shutter noise when taking picture" it didn't not help much because after a while one becomes accustomed to the sound. Usually it took me a while to realize that the sound (and the recording) had stopped - especially in cases were the app takes pictures only now and then.
The same goes for "warn at $(certain_amount_of_memory_left)%". I am not sure if the phone did warn before, but once I discovered it showed up a message "picture could not be saved" after each picture taken. The SD card was full… Give the app a warning sound!
Due to the way I used the phone (described above) it was not easy to know the state it and the app were in. Often it happened that I set the app to recording, put it inside the cover and attached it to the shoulder strap – and there was no shutter sound. So to have a look what the issue is one needs to take the cover off again and peel the phone out. By the way most of the time the phone was connected to a spare battery to charge while walking which made "have a look at the display" still more complicated. It also happened that by pushing the plastic cover the touchscreen received input - and when this happened at the wrong place recording could stop. As "invincible mode" I suggest a kind of lock mode in which one has to enter a kind of unlock "code" – touching the buttons in a certain order – to be able to access the app.
Some speech commands to start/stop recording and switch the modes (walking/panorama) would be helpful.
Multi level panorama
Being on a mountain top one has a great view. So great even that making a panorama with only one row of images is not sufficient. With my "good" camera I usually take three rows for a big panorama:
- Sky with a little horizon
- Sky and horizon in equal parts
- Horizon and below
Write level to Exif
Since it is very likely the phone is out of level while walking it could be useful to store the degree the camera is out of level to be able to correct the image easily.
Good sequences (not shaky, not blurred and well matching images) Mapillary could present as time lapse movies.
To review this amount of pictures I struggled a little to find the best way. Just putting all the images in one folder and looking at them one by one is a slow method to sort them out. A lot of picture viewers are really slow on such a task, especially if they create a file list. The viewer ''feh'' is fast at displaying images from a folder containing zillions of them. But it still would take a lot of time to watch picture by picture. I finally chose to create a folders per each day and hour which is easy because of the naming of the images: 2014_08_14_18_25_30_374.jpg (year, month, day of month, hour, minute, second, splitsecond) Inside a folder containing images this line creates folders for each day and hour and moves the regarding images there: for i in $(ls -1 | cut -d "_" -f 1-4| uniq); do mkdir "$i"; mv "$i"*g "$i"; done With the picture viewer geeqie and fast preview settings it doesn't take too long to sort the bad images out. I mostly scrolled fast the preview thumbnails to find them. For me "bad images" are unintended selfies, shaky, over- and underexposed ones or where the biggest part of the images is obscured by stuff in front of the lens (trekking poles, arm, hand etc.).
Duplicates, triplets and so on I mostly didn't remove since some intelligent automagism could create a panorama from several files. After reviewing some thousand images I started to name these and "real " panorama pictures by adding "pano" to their name. For the little parts of the journey I used transportation I also named the images accordingly.
When the way goes steep uphill the pictures show a close up of the path without much landscape, often with the trekking poles. When descending the opposite happens: A lot of images seem to show the same piece of the landscape but not the path on which one walks downhill slowly.
Due to the walking movements it happens a lot that the phone goes alternating out of level to the right and to the left or shows alternating more of the left side of the path and then more of the right.
Really bad roads Mapillary cannot depict when the phone is mounted to a cars front screen. Either the car is too slow or it shakes too much. And when you hold the phone with your hands you sway it around a lot… In some days you can see how this looks on Mapillary – with the sequence from Kroi Bardhë to Kumbull.
As I wrote at the hardware section already: It is very hard to get unshaky pictures when walking around without bright sunlight. Using Mapillary in the early morning or late in the evening? No way you get good pictures with your smartphone.
When it comes to mapping it is of course not bad to have as much data available as possible. But loading all the images of just one day with JOSM requires a potent computer – and looking at each image for mapping gets soon tiring. You should still collect POI with your GPS/a matching app or make photographs of interesting POI with another camera to know, where the interesting parts to map are.
When you have uploaded all the images to Mapillary without local backup and want to map using them it gets difficult. You have the choice of either look at the images on the Mapillary site, draw your conclusions and use the editor of your choice. But for me being dedicated to JOSM this is an awful way to map with photos. Also the script ubahnverleih wrote to download images from Mapillary and get them into JOSM gives results which don't satisfy me. It creates a gpx file with the images embedded. Contrary to loading images directly into JOSM where you are able to scroll back and forward through them you have to click on each single icon to see the image thumbnail - and click once more to see it in high resolution. Afterwards you have to close two windows, too.
My recommendation: keep a local backup of the images until you are done with mapping.
- Mapillary is a nice way to collect images during (holiday) activities without bothering too much about technology.
- Due to hardware limitations of smartphones you need a lot of light to get good images.
- While walking it is not easy to make good, leveled and not blurred images.
- For mapping purposes (with JOSM): keep a local backup and still collect POI with another app/device to easily find the highlights of the journey.
Despite all worries, the 1kg extra load and some occasions I really cursed the stupid hardware¹ I want to thank the company Mapillary to enable me to collect that amount of data in remote regions of Albania. In some way every photo is historic but I think since Albania is a fast changing country and little known to most Europeans these images may get some attention.
¹ "Great! I walked that breakneck path, didn't fall down – and the camera didn't work and I have to spend some minutes for rebooting the phone and restarting the app!!1 %$§£Ω¤"
Since publishing this post I added some paragraphs, the image and fixed some minor errors
My name is Eleanor Tutt and I live in St. Louis, Missouri. I’m passionate about maps, open data, community action, and (of course!) OpenStreetMap, which luckily combines all three. Over the past year, I’ve been actively contributing to and growing a committed St. Louis OSM community. As a board member, I would help out in whatever way possible, and I would especially like to:
1) Encourage participation of neighborhood organizations/block leaders
My day job is serving as a one person “data and maps shop” for Rise, a community development non-profit serving a four county area. In this role, I partner with neighborhood leaders to help them access, map, and interpret a wide range of neighborhood-level data. Our work emphasizes identifying and mapping assets such as schools, playgrounds, restaurants, gathering spaces - essentially, ideal OSM points of interest. I see a great opportunity for OSM US to work more closely with neighborhood leaders who are residents of and experts on under-mapped areas. These leaders already advocate for their neighborhoods at public meetings, in the media, in conversations over dinner - I want to encourage them to advocate for their neighborhoods on OpenStreetMap as well. One of the things I emphasize when I explain OSM to these new editors is that OSM gives individual people the power to shape the narrative of their city. I encourage them to map what matters to their community.
2) Continue support for editathons/mapathons
We’ve had great luck growing our OSM editor base in St. Louis through editathons, several of which I have organized. Our Cherokee Street editathon had two locations with different activities on opposite ends of a diverse commercial street. We partnered with a local art gallery and used Field Papers to increase participation from residents walking by (who may not have been carrying laptops or smartphones). For our Build for STL editathon, I encouraged editors (including nine new editors who had not heard of OpenStreetMap previously) through a friendly competition, with a score board and a trophy made out of an old globe, mouse, and LED lights. For these events and others, support from the national level in terms of organizing, publicity, and education/tutorials about available tools has been key, and I’d love to contribute to that work.
As I add my name to the list, I see that there are already many other highly qualified candidates. If I am not elected, I still look forward to working with them and becoming more involved at the national level in 2015. I thank the OSM US community for giving me the opportunity to run!
assorted detail adds near yeasterdays work. reinstating steadily removed details removed on "stylistic grounds" ok some parralax and offset problems exsisted but they only needed shifting to fit a good alignment. Walter's Row is to have the gates moved to the back the front hedgeing added soon.
I just reactivated my stone-old osm livetracker: http://datenkueche.com/osmlive/
What an evening!
60+ willing volunteers with laptops in a room (Big thanks to Skills Matter) with a team of helpers & great things happened. We talked, they worked & occasionaly we answered questions or helped in some other way. Some of our team were elsewhere in the world giving feedback to our new mappers, and some more were learning & helping using web links. There are quite a few photographs on Facebook - search for Missing Maps Project.
Well, I've never seen anything so good so quickly. Our mappers coped with the squares magnificently, checking what had already been done, adding missing details, and marking the squares as complete. Validators checked the squares, corrected any slight errors, gave feedback (messaging system in the TM works extremely well for this) & marked the squares as validated - we do need more validators though - might have to see if we can train some of our more regular volunteers in this.
You were terrific. They wouldn't have done it without you - the whole package is needed & I don't know the names to thank everyone. There will be many unsung hero's and you should have a warm glow inside from knowing that you helped to make it work. Advertising, contacts, paperwork, ordering pizzas, standing at the door, making the technical bits work, admin - more admin. There's a lot of work needed, and I for one am grateful that you helped to make it happen, On the night I was really pleased to see many helpers giving advice - Dan, Harry, Nataly, Ralph, Richard, to name just some who willingly gave their time & expertise.
Validating / Feedback - to come
I'll try to make sure that everyone who was at the session gets feedback on at least one of the squares they compete - it may take me a while though. Don't wait for the feedback, keep mapping & keep learning. Mistakes happen, when you find out you've been making one, see if you can go back over the squares you've already completed & correct your previous work. If the square has been validated, the validator has probably already corrected any small errors. Personally I spend part of my time researching & part of my time mapping - have a read through http://learnosm.org/en/ or any of the links from http://wiki.openstreetmap.org/wiki/Main_Page
I'd recommend that you sign up for HOT emails - this link for a howto: https://lists.openstreetmap.org/listinfo/hot
If IRC is your thing then http://wiki.openstreetmap.org/wiki/Irc (obviously I'd recommend the HOT group!)
More to come
Mapping - there is plenty of scope - the Task Manager contains many worthwhile tasks / projects. We trained mainly in iD, but there are other map editing programmes and future mapathons will cover options. If this was your first mapathon you only saw the tip of the ice berg - there are other HOT volunteers adding names to villages & towns (have to be careful of the source, so there are special instructions on this), mappers who validate, and volunteers who write software - boredom is not an option!
Thanks for reading, and I hope to see more of you with HOT and the Missing Maps Project - let's get it mapped before the disaster happens!