Recent diary entries
Some more cross-stitch to complete but here's the early view of a globe based on OpenStreetMap data :-)
Globes, gores and geojson (topojson)
Been thinking about this for a while. Like maybe two years. How do you make a globe using openstreetmap data? So when I say think, I mean a quick way to hack this together using lots of other peoples hard won code.
According to what I could find, the main thing was to create 'gores' these are the name of the thin oval shapes that make up the outside edge of the classic 'orange' segments.
So I googled it a while back and found a perl script for Gimp that appeared to do the trick. At the time, it seemed very involved and I hardly knew much about writing scripts etc (although reading about again - it now seems more straight forward). So I put the idea on the back burner for a while
The next approach seemed to be using Generic Mapping tools (GMT). Which also has a way to stitch gore projections together. But the first time I attempted using GMT, I had issues installing it. Once I got it running it has a bit of a learning curve and lots of interesting file formats...After a short battle, I abandoned my attempt at trying to get it to work.
So it would be nice to do in QGIS using the proj4 library. But unfortunately, it doesn't work for gores like projections. In the business they are called 'interrupted' projections because the map is not just stretched but split into pieces. It is possible I believe to project one gore at a time and stitch together (sounds like a python script waiting to happen).
Anyway, the d3-geo has lots of examples and it was easy to use or maybe my knowledge is somewhat more advanced now. It was possible to tweak the examples and see how they work. Although the core projection mathematics is still way out of my league. But thanks to Jason Davies and Mike Bostock who created them - absolutely amazing.
The next stage was to get some OpenStreetMap data. But that was no problem, using openstreetmapdata.com (thanks Jochen Topf). This gave a shapefile of the basic global land polygon using OSM. I decided to simplify this using QGIS - but this step is not really necessary. After simplifying I was left with a file called simpleworld2.shp but D3 required topojson format. So on a linux type system, first you need to installed Node.js (I had this already), then install TopoJSON:
npm install -g topojson
then using the topojson guidance I used this command
topojson -o output2.json simpleworld2.shp
Converting to topojson greatly reduces the file size. You can now reference the output2.json file within the example given on the d3-geo site.
Now get busy with staples and glue!
p.s. looks like there are other ways using Wolfram / Mathematica/ Matlab examples.
Mapping parties are great fun and it's nice to show how much work the people involved put in.
So let's try it, stick this in your web browser:
I found this tool was easy to download and gave the option for meta data:
Install this at command line:
$ npm install -g osmtogeojson
and try something like:
$ osmtogeojson Anniesland.osm > Anniesland.geojson --flatProperties
Brilliant. We can now stick the .geojson in QGIS or Tilemill and make a map:
...looks like overpass api can be used for some nifty stuff, maybe animated snapshots by hour or days....go play.
Can't help thinking John Muir, the eminent 19th Century (Scottish-American) naturalist would have been a huge fan of OpenStreetMap. He was a bit of a hacker/maker in his day and his love of a good wander (up to a 1000 miles or more) would have made him a handy mapper.
Anyway, the John Muir Way has recently been officially opened for hiking and cycling across the central belt of Scotland. It a coast to coast route (with slightly different routes if you are cycling or on foot).
There's still a little bit of work required on mapping the hiking route. But the bicycle route is one continuous section. You can view the route and get some analysis of it here. What a neat tool that is, Kudos to GrundID.
But how do you work out how many people have contributed to mapping this? All the ways, nodes etc?
Well, I couldn't find a service that did this for me. But I knew it's pretty easy, so I used osmar package for R. Although, it doesn't look through the whole history of all the ways making up the relation/route it's a pretty good estimate. You can download R easily and I recommend Rstudio as a nice interface but plain R is all you really need.
Here's the code.
But if you can't wait the answer is approx 106 people have helped to edit the ways and nodes that make up the route.
If anyone can recommend other tools etc for relations and route analysis - please do. Thanks
After hearing this keynote about diversity and gender at SOTM 13 and previously this video. I thought it would be good to look for ways to broaden our community through the events I help to organise in Scotland. In general, I think we are reasonably successful at attracting a wide range of people and it's worth noting that OpenStreetMap may achieve the best success when it works with existing groups and partners during mapping parties. Some thoughts of participation at mapping parties here. But we can definitely do better.
So I'm following Wikipedia's lead and hoping to work with Glasgow Women's Library to run a community mapping party event some time later this year as one way to make sure the widest and richest set of features are uploaded to OSM and offer a better alternative to inherent bias in other maps.
While organising this event, I was asked this question: 'Tell us about your experience of working with women and diverse communities and how you will ensure this project respects difference and safeguards equalities?'
Good question. Which got me thinking - does OpenStreetMap have a policy safeguarding equality or respecting difference. Well, short answer is no.
The mission statement includes 'open to all' and 'democratic process' but doesn't specifically refer to supporting or promoting diversity or equality. Nor do the draft core values
In contrast Wikimedia values includes this section:
'Our board of trustees, staff members, and volunteers are involved without discrimination based on their religion, political beliefs, sexual preferences, nationalities, etc... Not only do we accept diversity, but we actually look forward to it'
Wikimedia even had a conference about diversity.
So how do we share the mapping love with the broadest range of people?
I'm using this post to suggest we could make it more clear that our mission to 'provide free geographic data' should be backed up with more information around diversity:
A vision (to paraphrase Wikimedia): 'Imagine a map where every human being can freely create and access geographic data about the entire diversity of the world's culture, heritage and places. That's OpenStreetMap'
Short policy statement on equality and diversity to spell out that the Foundation does promote and up hold these values.
Board election manifestos should be mandated to include a section on how the candidate will address equality, diversity, bias etc.
Top ten OSM task list does include translating languages - which is great. I think the top ten task list should always include something around promoting a wider audience/diversity etc
What do you think?
More info on: Diversity mailing list
Meetup Blog - Tu 19 Nov - Saramago bar, CCA, Glasgow
Lots to discuss (more than usual?) at the this quarterly meetup in Glasgow. Following on from an Open Knowledge Foundation event in the same venue the night before, there was a large cross over of people wanting to join us to discuss the spatial side of open knowledge.
Big part of the night was spent discussing plans for a State of the Map Scotland event for next year - possibly early in the year - Spring/Summer. This time we are keen to bring SOTM Scotland back to Glasgow (where it started 3 years ago). Partly because Glasgow is convenient for yours truly, but perhaps more interestingly it's also the winner of a Future Cities Demonstrator prize. The prize is to help Glasgow innovate around smart city ideas and open data to make a better city. So we are hoping to tap into the activity around smart cities & open data in collaboration with wikimedia, libraries and open knowledge foundation to create a really great event.
The future cities demonstrator will be delivered by Future City Glasgow team as part of Glasgow Council. They have a beta website which has just gone live here. Lucy, the outreach co-ordinator was there to give us the lowdown on what's happening. You can follow news on twitter here.
Bob from CityStrolls a keen activist for the Commons was talking about the importance of making social connections and getting people involved. This is always something worth thinking about and also something we can get better at. Interesting to discuss the Common Good - which in Scotland is a legally protected collection of assets owned by the public. And the thought that new commons of wikipedia and openstreetmap etc are in a way, building on this legacy (but hopefully won't be as easily overlooked).
Graeme, wikimedia volunteer - was interested to find out more about openstreetmap and how the two projects are helping each other. Great to see wikipedia using openstreetmap maps as default and perhaps future integration in wikidata. Also it was good to hear how wikimedia was reaching out to get more women involved, working with the Women's Library.
Duncan, was talking about his experiments with opening up the architectural process both by using openstreetmap to create 3D models and generally how to make the whole design process more open. Check out his blog
Julian and Eileen were discussing moving house and trying to find a new place which needed good public transport links and dark skies for astronomy - sounds like a good GIS challenge - there is a dark skies dataset available...
So another great Glasgow meetup topped by Saramago bar providing us free bar snacks (we always manage to get free food at our events...)
Next meet-up in Edinburgh Tu 10 Dec 2013 from 7:30pm at the Guildford Arms more info here
You like cake? You like a challenge?
Grab a slice of the 100 square mile mapcake of the Black Isle - North Scotland. Please spend a few minutes editing here - much appreciate and will help support sustainable transport. Thank you!
Want to know more about making maps of your community, citizen science, open data, and how to get started? All welcome to join the Glasgow OpenStreetMap social meetup at the Saramago bar in the Centre for Contemporary Arts, Sauciehall Street. Tuesday 20 Aug 7pm.
We meet here on the third Tuesday, every 3 months.
You'll find us in the bar, look out for a hi-viz jacket and maybe some laptops. Excellent vegetarian food available and good selection of beer and soft drinks.
There will be people to help with any OpenStreetMap questions about editing and using data. Will also be talking about:
- Up coming State of the Map conference
- Changes to the OSM website design
- How to get data into QGIS and why the plugin is broken
- What a spatial database looks like
- Adventures in GRASS
- What happened to OSM Scotland website
- Scotland OSM stats
- 3D buildings and when we can start 3D printing
- The next mapping party
- State of the Map Scotland
- Should OSM website have export .shp option?
- Smart cities...
See you there!
Cycle route stats
In an effort to calculate some cycle route stats in Scotland for this weekend's cycling protest called Pedal on Parliament, I created a cycle stats website.
It uses an R script to download the latest data from fabulous Geofabrik, it then cookie cuts .osm Scotland into MSP constituencies. The boundary data is from OS Open Data (simplified and processed into polylines in QGIS). The .osm data is converted into data.frames and analysed using osmar library for cycle route relations and ways.
There are more than a few assumptions and hacks in the process and some unexpected results.
Basically, I think I'm asking a bit too much of the osmar library and I need to try a database approach. But it works - kind of.
From the early days of helping with OSM, I've been interested in being able to summarise in some basic way the geographical features in my area. Whether it's the number of bus stops or if there are more car parks than parks? Following the recent Dunbar mapping party, i started to investigate the options for this type of analysis.
Dunbar mapping party:
Previously, I used QGIS for this type of task and to create maps. This is a great tool for visualising but like Excel for spreadsheets it is difficult to reproduce and reapply analysis to different areas. To improve flexibility and quickly reproduce work on different areas requires a database and scripting. A geodatabase can be linked to QGIS but I faced problems with keeping the database up to date. Scripting can also be done in QGIS via a python command line and GRASS plugin. Certainly python has a lots of powerful features here is a video discussing some of them and I'd like to try this more in the future.
But the most suitable tool for me, is using the statistical language R. I have a basic knowledge of the language and it's friendly syntax appeals to my general unfamiliarity with coding.
The R language has an useful library for accessing OSM data with a great guide to get start. Mix this with a nice GUI called R Studio with support of 'knitr' library which can generate webpages from R code - this makes a simple but powerful tool for summarising OSM data and presenting it on the web. Here is a demo webpage created in R from Dunbar OSM data. This has already paid off in some ways because QGIS osm plugin doesn't support 64bit ids - preventing more recent features from being correctly displayed in QGIS (there is a fix). Whereas R isn't affected by this issue. Here is another great example of what can be done in R with osm data
Building data from Dunbar, Scotland via OSMAR library in R:
So in summary - I think R is useful for OSM analysis and hopefully we can see more developments and tools based on R being produced. Try it yourself: download R, then R Studio, open R 'markdown' window and paste my code in. Change the co-ordinates at the start of the script to where you live (go to OSM mainpage map and find your location - hit 'permalink' in bottom right hand corner and copy co-ordinates from www address bar into the script). You may have to install packages/libraries using Tools menu.
Are there more car parks or parks in your area? Is this by mistake or design? (okay, it can't answer the latter question)
Dunbar is looking green with plenty more Park than Car park.
Parks: 3.7112 hectares or approx 23 football pitches
Car parks: 2.0239 hectares or approx 13 football pitches
I started a Dunbar map cake prior to on the ground mapping party next week:
Please dive in - just 5mins would be great. Trace some buildings.
More about Dunbar mapping party here
Dear OpenStreetMap mapper: We are the small non-profit that runs the #6741 website in the world (#1,824 in Germany). We don't have any staff but serve 1 billion map tiles every eleven days, and have costs like any other top site: dragons, power, rent, programs, trolls and legal help. OpenStreetMap is something special. It is like a sensory garden or a learning circle. It is like a temple for cartographers. It is a place we can all go to map and navigate our way through life. To protect our independence, we'll never run ads. We take no prisoners. We run on donations averaging about £30. If everyone reading this gave the price of a cheap tourist map, our fundraiser would be done within an hour. If OpenStreetMap is useful to you, take one minute to keep it online another year. Please help us forget fundraising and get back to mapping. Thank you.
Please help - I'm addicted to OpenStreetMap and couldn't map without it
Lots of OpenStreetMap events in Scotland over the next week or so:
On the third Tuesday every three months (November, February, May and August we have social meetings at 7:30pm at the CCA: Centre for Contemporary Arts Everyone is welcome. This is an opportunity to meet with others and hear about what's going on and learn new mapping techniques or just general chat. We also welcome anyone from outside the Glasgow area to come and meet with us.
The date for the next meetings are: November 20th 2012
We will be in the bar on the top floor (Terrace Bar) - look for a hi-viz jacket. Alternatively, look for us downstairs in the CCA centre.
There is free parking on the surrounding streets after 6.30pm. The CCA is a 10min slow walk from Queens Street or Central Station. The nearest subway stop is Charing Cross.
Follow events on twitter at @OSMScotland
The Edinburgh hack weekend on Nov 2012
Will take place on 24th and 25th in Edinburgh hacklab. Hacklab is currently at Out of the Blue (Drill Hall), just off Leith Walk. Start time 11:00am both days (finish time tbc...) Studio S5, Out Of The Blue, 36 Dalmeny Street, Edinburgh, EH6 8RG
Thanks to Edinburgh Hacklab member for offering to host us at their venue. This venue is limited to 8 people max. First come, first served - indicate your attendance on the wiki and which day(s). Please add to the list of hack challenges and share your ideas. If more than 8 people sign up, you can always drop by to chat but you may end up squeezed out into the corridor.
The State of the Map Scotland 2012 Unconference (SOTMS) will be held in Edinburgh on Friday 19th and Saturday 20th October 2012 10am-4pm on both days. It is free event, everyone is welcome. It will be held at Inspace which is the building next to the Informatics Forum of University of Edinburgh. The primary focus of our conference is to get Openstreetmap.org volunteers to meet and discuss what they have done over the past year and for those new to Openstreetmap to meet us. We also would like to use this opportunity to meet with academic, government or private business groups that are using OSM or freely available data to show us what they are working on
Conference booking: Here
Conference schedule: Here
All welcome to join the Shotts Mapping Party, North Lanarkshire, Scotland.
One-day Shotts Mapping Party
Last Tuesday I met with other OSMers in Edinburgh, we usually meet every 3 months in the Guildford Arms just off Princess Street. There's no wifi and poor phone reception but I like the old place and there's a good selection of beer (and soft drinks too). 10 people turned up with a mix of old and new faces. It was great to see OSM attracting more and more interest. There was a small pre-meeting ad-hoc mapping party along Leith Walk. (I've made a few edits but still lots to do).
We chatted about all sorts from app development, UAVs to climate change and cloud computing. We had a few members of Edinburgh Hacklab join us - we share a love of making things. Other collaboration is going on with Wikipedia members with an event being planned around mapping, photography and wiki writing about monuments. Also there was talk of Edinburgh Council releasing some Open Data which could be really interesting.
Started planning another mapping party which will be in Shotts in July - more details to follow. And rumblings of another State of the Map Scotland...
We talked a bit about visualisations and the new GPS dump and what other OSM data could be made available. We wondered if tile request data could be looked at - seeing which tiles are the most popular etc. There is some related data and info here .
Well, that's all I recall for now...
Next month, there's an OSM meeting in Stirling.
Natural features such as woodlands, trees and gardens as well as built infrastructure like paths, gates, fences and parks can all be mapped on OpenStreetMap. All this information is key to understanding our green infrastructure and presents a new platform for communities and citizens to map and share their local knowledge. This idea lead to an application to Central Scotland Green Network's fund via Clyde Valley Green Network for resources to organise a 'mapping party' at New Lanark and a map making workshop at the Electron Club, CCA, Glasgow. Providing a learning opportunity to local mappers and groups to use OpenStreetMap to map natural and built features around New Lanark and create their own customise maps. The location was chosen because of the great mix of natural woodland and cultural features in this World Heritage Site. A snapshot of the level of detail can be seen in the map below (created at the mapping workshop):
These mapping events were successfully held in March. Attracting mapping enthusiasts, local people concerned for their environment and various environmental and community representatives. All these volunteers helped create a really detailed map of New Lanark using various mapping techniques including using smartphones, cameras and paper-based methods. The data was further enhanced by tracing from aerial imagery and maps from non-copyrighted sources. See the results here Corra Linn, New Lanark (photo taken by Chris Fleming while mapping):
What's next for OpenStreetMap and greenspace? Central Scotland Green Network are looking at ways to integrate OpenStreetMap information to assess pedestrian accessibility to urban greenspace. This is a ground-breaking project, the first to use volunteered geographic information to help increase understanding of greenspace. And we're only just getting started.
Find out about more events and news at OSMScotland twitter or look on the Scotland wiki. The next meet-up is in Glasgow on Tu 15 May 7pm, CCA Terrace bar - all welcome, come along for an informal chat about mapping. Mapping party introduction at New Lanark:
I have been looking at other 'open' projects today. There is a good list of citizen science projects at www.scistarter.com
Unfortunately, it's surprising how many 'open' projects don't release their data as open data.
I did find a great project called openBmap, which is currently looking for support to keep its open project going (yes, really 'open' under odbl).
www.openbmap.org collects info on phone signal coverage - allowing you to see the best operators etc in your area. They also collect wifi data.
Best of all, they use OpenStreetMap as a baselayer on maps!
Hope someone can support this truly open project. Perhaps as a sister project?
After an amazing 2011 of OSM activity in Scotland including State of the Map Scotland huge amounts of mapping, networking, talks and cake eating - what's happening this year?
Here's the plans so far
Hope you can make it along