Recent diary entries
... this coming weekend, and now i'm hearing that there may actually be 2 or 3 people in the room for my presentation. i guess i better put a few more minutes into it so it doesn't suck so much.
Today I tried to edit my first entry in the OpenStreetMap Wiki by adding a page for Mapillary. However, big thanks to Alexandre Magno who started even before that with the Portuguese version. This community is pretty amazing!
I woke up one morning, and realized I needed a reusable dataset of all the communities in the world. Not just X-Y, but administrative areas. Obviously, I started looking on OSM. With a bit of playing around (and a little help from my friends), I had a nice set of admin areas of various levels from OSM in a shapefile. Then I started noticing holes. If a country is mostly made of holes, you know there is no data. But if there are a few holes, well, something is fishy. What happens is, there are no extraction tool that can make an admin area if there are gaps. A line is not an area, only a closed line is. Borders in OSM are relations. These are collections of lines, joined together in virtual union. Often, someone deletes one of these lines, and replaces it with a more detailed version. That's sort of OK, but then you have to add the new line to -all- the relations the old one was part of. This is of course very exotic to new mappers, and even experienced mappers don't always seem to care.
Data use = data cleaning
Data will only get fixed, if they are used. And even on the forum, I saw people referring end data users to other sources to get their admin areas. It is complicated to extract a dataset with borders from OSM. So why care about this data? It shows up okay on OSM even if it's broken. BUT, user Wambacher made this great tool to download shapefiles by country with all available admin areas. So now that it's easy to use the data, please help maintain it.
Fixing things up
I tried several tools to help fixing things. I have a global focus, so I've mostly been doing fixups on the admin level right below the country. Often states, sometimes departments, etc. If you're going to fix a certain area, you're going to need other tools (see below) - choose a level to fix using layers.openstreetmap.fr - check which are missing - find the relation which is broken (or rarely, missing). Search doesn't always work. Zoom to a frontier which at one side is ok, at he other isn't. Click on the frontier and find the relations it is part of. Copy the ID. - Vizualize the relation with an url like this: http://www.openstreetmap.org/relation/3624100 This shows obvious defects. If defects are more subtle (little holes, almost-junctions), go to http://analyser.openstreetmap.fr/cgi-bin/index.py and paste the ID.
Causes of trouble
During the fixups I found many different types of errors. Borders are basically always the result of imports. Duh. Messing around with borders are a good way to understand why imports are controversial. It's easy to do it wrong, and hard to do it right. Sometimes there is data from the original shapefiles that were used, like area=123. In some cases, the original polygons are still there. And where it gets really messy, is when you add detailed borders from source X on top of general borders from source Y. It takes a lot of time and effort to clean that up, and importers don't always get around to finish it all up. Once the data are there, information may be redacted, because the original data wasn't compatible with our licence. Most often: no commercial reuse, the menace of all open data users. And redaction leaves a mess. Another source of trouble is including both seafront and a maritime border. These should be separated. Apart from that, most errors come from beginners or experienced users alike who delete a line and replace it with something else. Simple solution: use shift-click to improve geometry instead. That makes understanding the history of an area much easier too.
In depth error checking
If you want to go in depth in a certain area, these tools will come in handy: http://keepright.at/report_map.php > Click "none" (bottom left), then only activate the "Boundaries" checks
This even vizualizes all the broken relations, but it's just plain depressing to use: http://tools.geofabrik.de/osmi/?view=multipolygon
Today I started using mappilary, the crowd sourced street view based on OSM. The android app worked very well and it is incredible simple!
Check this footway here in Brasilia. Abraços, Linhares
Re my previous diary entry the mapping party is happening!
I've also blogged a bit about the new section that needs to be mapped (Section 2 on my cake map)
Finally, have a look at these pictures to get an idea of what needs to be mapped - again, mainly section 2:
I've been trying to find an efficient method for adding street numbers, but so far every workflow I've found has been rather horrific.
I like vespucci, but it's a lot of clicks to create each node and type in each number. I thought I'd found a possible solution when I came across keypad mapper 3 for Android. Trouble is, for something designed specifically for this purpose, the interface is still pretty unwieldy.
I put quite a bit of time, thought and effort into mocking up a better user interface and posted it here almost a month ago to no response at all: http://wiki.openstreetmap.org/wiki/Talk:Keypad-Mapper_3#Faster_GUI_Mockup_for_Most_Common_Use_Cases
Worse than that, Keypad Mapper 3 positions the nodes all over the place. On the street it's a neat , regular 10 meters or so between each house and each number goes up by 2. In the OSM file it makes, Often the node for one house might be 50 meters away from each other or more. When importing into JOSM sometimes it's hard to even tell which street the node is supposed to be on!
The GPS on my Nexus 5 phone's other apps, seem to function with good accuracy.
So does anyone here map street numbers regularly and have a decent workflow for this?
Decided to take some time to myself today for active meditation. Mapping the Daybreak area for the rest of the day.
Away on my honeymoon in Thailand, am I doing any mapping? Not much no! Firstly because my wife wouldn't consider it a very honeymoony thing to do, but secondly everywhere we've been to has been pretty well mapped! The resort I stayed in Krabi is mapped despite being under a cloud in bing imagery.
Now we're up in the north of Thailand in the historical city of Chaing Mai. Amazingly well mapped! All the temples and every other POI I could ever need within this distinctive square old town area, mapped in great detail.
Of course Chiang Mai is also a popular holiday hotspot, so this probably doesn't mean the whole of Thailand is well mapped. I remember Matt commenting on this thing when he went on holiday to Mexico a five years ago. OpenStreetMap develops its coverage on an "interest first" basis. Interesting places get mapped, and this means holiday hotspots get mapped a lot quicker and more thoroughly, sometimes while massive cities nearby remain unmapped. Places which aren't mapped are maybe rather uninteresting (and if you live in an unmapped place, and find this insulting... time to make it mapped place!)
Looks like the mappers in Chiang Mai are other holiday-makers rather than locals, judging by the not-very-Thai-sounding user names in this top mappers display.
(ITO OSM Mapper view ranking top mappers in Chiang Mai based on last touched objects)
Whoever it was, they've done a great job. I'm using the MapsWithMe app to view all of this without roaming charges. I can watch where the tuk tuk is driving in great detail.
I was also watching where speedboats were taking us on snorkelling trips to various islands in Krabi. This seemed to reveal a few missing islands actually. On investigation I see a bunch of islands which have ended up without a natural=coastline way. Instead they have natural=wood turning them green on the map. e.g. look at this bunch. I added a missing little one here. Harry's island. I've put the coastline tag on it so that one will show up on MapsWithMe. So seems like there's some tag fiddling fix-up reconciliation needed on lots of islands. I reconciled things on Ko Ma Phai ("Bamboo Island") by removing the wood tag, since the whole island is not covered in wood. For starters it has a BIG beach with coral on it.
It's pretty tough having to survey these things. While I work my ass off on these troubling issues, my mapping friends back in London are busy...
Kicking off the London mapping season TONIGHT! Matt's organising a walking talking mapping tour. Great for new people who want to learn more about how OpenStreetMap works, so if that appeals to you, or if you know anyone, please let them know.
There's also an Olympics Park Mapping party (Yes! Two scheduled mapping parties!) happening next Wednesday evening.
All the details: wiki.osm.org/London
Vadivelu did tamil new songs everything like a movie playing in place of, say, drop cards torpparkale renegotiation silent.. it seems that the idea of knowing everything Ekambaram Mr. Cabral said that things. Need more information - http://sanjeevkanti.simplesite.com/
Debian is a free operating system, but it is a bit more than that as it also provides lots of packages, most of which are precompiled software. There is also lots of software in Debian, that relates to OSM (JOSM, mapnik, osm2pgsql, ...), and some software that is not (mod_tile, tilemill, ...).
One of my long term goals is for it to be possible to install an OpenStreetMap tileserver on a Debian system with a single command. The user would install a metapackage via the package manager, which then depends on all the components needed (mod_tile, postgresql, apache, ...), and then be asked the relevent questions (what data, ...) by debconf (the Debian Configuration system).
Unfortunately, this is a long term goal, as several not insignificant things need to happen first. The first probably being getting mod_tile packaged for Debian, as I currently understand, this is blocking on iniparser (which is not in Debian) being used in mod_tile.
In the short term however, there is now some hopefully comprehensive documentation on building a tileserver with Debian up on the Debian wiki. It should not have too many bugs, as I ran through the whole guide twice while writing it. Some bits still need doing, for example, this guide only covers jessie (the current testing release), and it does not cover getting a slippy map (with leaflet, openlayers, ...) setup.
Just read it at the tagging ML and cannot help but have to share it with you: Tag:natural=cloud
Hopefully some interested people will comment and vote on it – and the uninterested will use it for mapping.
Just a quick entry here:
I recently added a new postcodes from my recent surveys and corrected some format errors, so I was wondering if any one is still maintaining the OSM source layer in the Postcode map so that I can see how my edits have corrected the errors. Thanks.
I have an Igotu GT-120 GPS datalogger, for GPS-photo correlation. By combining gps-correlate, and any camera with a timer (or timestamps on files, see my previous diary entry for the code...) I can now do correlation. The device has no built-in display of any kind, so it won't be any use for anyone who needs on-the-fly GPS, with no external connections.
It works, but the connector is a bit worrying. As a simple mapping tool, it should be enough for those who are happy to do photo-correlation, but don't have/want or for some other reason can't use a mobile phone based GPS. It wont be super-high accuracy though!
The chipset in the device is apparently a "SiRF StarIII low-power" device. The usb A->proprietary connector is a bit dodgy (insertion/removal) at the proprietary end. Not sure why they didn't use USB mini...
There is a CHM file available on the manufacturer's website, which give the LED codes. I've reproduced them here for posterity
Warm-up times are claimed to be 35 seconds, but my first power on (albeit without a clear view of the sky) took several minutes after being outdoors before it woke up and started logging. I assume this has something to do with GPS almanac data, and that subsequent power-ons will be better.
- Turning on the device : press button for 1.5 seconds, until blue LED flashes once.
- Turning off the device : press button for 1.5 seconds, red LED switches off.
- If red LED blinks twice : memory low
- If red LED blinks once: battery low
- Charging : red LED on until charging complete (possibly 2 hrs?)
- Logging individual waypoints : push button once -> log acknowledge.
The manufacturers claim 65,000 records, so at 1/second should give ~18 hrs. The battery only lasts for 10 hours, according to the specs.
After using the device for a very short while (cycling), it seems to work, and I can load the tracks under linux (debian testing) using the igotu2gpx program. The capabilities include setting the logging time, and downloading tracks, as well as an on-screen display for maps. Most importantly, gpx export seems to work.
A deb file can be built by obtaining the .dsc and .tar.gz files from launchpad. This was quite easy, after
- tar -zxf igotu*gz
- dpkg-source -x igotu*dsc
- ln -s igotu*gz igotu2gpx_0.4.0.orig.tar.gz
- cd igotu2gpx-0.4.0; sed -i 's@native@quilt@' debian/source/format
- dpkg-buildpackage -j2 -uc -us;
dpkg complained about missing dependencies. After installing them, everything went smoothly.
I haven't tried realtime logging with gspd; though gpsd seems to recognise the device, but I have no lock indoors. As a cheap entry-level USB device, so far, so good.
Hopefully some good mapping can be done with it!
On this day last year I announced the Imagery Offset Database: a centralized storage for imagery offsets. It was planned as a way to provide every mapper, especially beginners, with a confidence that they are tracing correctly aligned imagery. And for those not knowing imagery can be misaligned, a way to not ruin a map. After the announce, tens of mappers started entering their offsets into the database, and I've never made a local offset bookmark ever since.
Aerial Imagery cannot be ideally georeferenced. Depending on precision of your measurement tools (GPX traces give 1 to 10 meter precision), you may notice that the imagery layer is positioned incorrectly, and use your editor's controls to shift it to the right place, so GPS traces follow roads and paths on the imagery. This is common knowledge among experienced mappers, and I hope beginner mappers learn that soon enough.
Sadly, looking at new editors I can't but conclude than either there is no misaligned imagery (which is probably false), or many mappers, including those who work on those editors, don't bother with aligning imagery to GPS traces. How many of you pressed those little arrows for shifting a background layer in iD editor? Does your favourite mobile editor, with which you upload POI nodes to the database, account for shifted imagery? Do you make bookmarks of such offsets so you can quickly restore them later, or on a different computer?
A week ago Simon Poole added IODB support to Vespucci, making it the second editor supporting it. This is great news, number of editors supporting the database has doubled overnight. Support in iD has been stalled, and I didn't expect Potlatch or iOS apps to support it. So basically, pay attention and don't be suprised when buildings are misaligned with imagery.
Now some statistics. As of now, there are 5180 non-deprecated entries in the database, of which 4078 (70%) are in ex-USSR countries (3180 in Russia). Mappers add on average 14 entries a day and have not skipped a day yet (see the graph above).
Of 5644 imagery offsets 4606 (82%) are of Bing layer, 714 of Russian and Ukrainian regional imagery, and around 150 offsets refer to some HOT-related imagery. Strangely, there are 44 offset of Google Maps, and 30 — of Yandex's. I assume those users were just testing the database and not uploading anything to OpenStreetMap.
I hope someday we won't need that database, but for now, when only a handful of coutries have precisely aligned imagery, it is one of the most important tools for mappers. It will probably be included in JOSM core later this year, and I'd be glad to hear of more editors supporting it.
And by the way, I always planned to publish the server's source code, but got to it only recently: see it on github.
If this article is true (http://mac.gov.pl/aktualnosci/dostep-do-panstwowego-rejestru-granic-bedzie-powszechny-i-nieodplatny), why can't I find the data anywhere!? Any users have any idea where this data is?
There's another forum for communication - an OpenStreetMap Subreddit on the reddit website - It's got 1,400 subscribers already!
When I first began tracking my travels, I needed a program that worked with my MBair. I'd looked at a few GPX Import programs and decided on using MyTracks. The program is pretty easy to use and it allows me to view shots over different maps. It also gives pretty fluid tracking data when I import to Fog of the World.
Here are a few Screen Shots. You can find the GPX files here, or see MyTrack shots of my travels at doubleDang.co.uk
The March release of JOSM is now available as version 6950 :)
Main improvements this month concern circle/alignment functions (thanks to Balaitous, a new contributor), HTTPS support, bugfixes and beginning of Java 7 migration.
This release is still compatible with Java 6, as will be also the next release at the end of April.
After that we plan to migrate to Java 7, so if you're still using Java 6, now's a good time to switch.
- Enhancement of Align Nodes in Circle, Create Circle and Align Nodes in Line actions
- Access to OSM and JOSM websites in https by default
- Remote control: listen also in https on port 8112
- Make UI messages copy-able
- Ask Mac/Debian/Ubuntu users to update to Java 7
- Add "Add/Edit/Delete" entries to tags/memberships contextual menus
- Add support for
- Add CRC32 checksum
- Add support for
- Check addresses interpolation range/values
- Check wrong multiple values
- WMS: filter unsupported image formats, preselect jpeg or png