This is a repost of an entry on my blog.

Last post ended with downloading OpenStreetMap data. This post will leave the data aside and switch to downloading and building a style. There's lots of styles available, but we're going to use OpenStreetMap Carto, the current default on OpenStreetMap.org. Also, because we need software not packaged in Debian, that needs to be installed.

For the script, we're going to assume that the carto binary is in the PATH. Unfortunately, this requires installation, which requires npm, which itself needs to be installed.

Given nodejs and npm is a huge headache of versions, the easiest route I've found is to install nvm, then install nodejs 6 with nvm install 6. CartoCSS is then installed with npm install -g carto.

The shell script starts off with some variables from last time.

#!/usr/bin/env bash

set -euf -o pipefail

OpenStreetMap Carto is hosted on Github, which offers the ability to download a project as a zip file. This is the logical way to get it, but isn't usable from a script because the internal structure of the zip file isn't easily predicted. Instead, we'll clone it with git, only getting the specific revision needed.

OSMCARTO_VERSION="v4.6.0"
OSMCARTO_LOCATION='https://github.com/gravitystorm/openstreetmap-carto.git'
rm -rf -- 'openstreetmap-carto'
git -c advice.detachedHead=false clone --quiet --depth 1 \
  --branch "${OSMCARTO_VERSION}" -- "${OSMCARTO_LOCATION}" 'openstreetmap-carto'

Setting advice.detachedHead=false for this command avoids a warning about a detached HEAD, which is expected.

OpenStreetMap Carto sets the database name to be "gis". There are various ways to override this for development, but in this case we want to override it for the generated XML file. Fortunately, the database name only appears once, as dbname: "gis" in project.mml. One way to override it would be to remove the line and rely on the libpq environment variables like PGDATABASE. Another is replacing "gis" with a different name. It's not clear which is better, but I decided to go with replacing the name, using a patch which git applies.

export PGDATABASE='osmcarto_prerender'

git -C 'openstreetmap-carto' apply << EOF
diff --git a/project.mml b/project.mml
index b8c3217..a41e550 100644
--- a/project.mml
+++ b/project.mml
@@ -30,7 +30,7 @@ _parts:
     srs: "+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"
   osm2pgsql: &osm2pgsql
     type: "postgis"
-    dbname: "gis"
+    dbname: "${PGDATABASE}"
     key_field: ""
     geometry_field: "way"
     extent: "-20037508,-20037508,20037508,20037508"
EOF

With project.mml patched, it's easy to generate the Mapnik XML, because CartoCSS was installed earlier.

carto -a 3.0.12 'openstreetmap-carto/project.mml' > 'openstreetmap-carto/project.xml'

Lastly, OpenStreetMap Carto needs some data files like coastlines. It comes with a script to download them, so we run it.

openstreetmap-carto/scripts/get-shapefiles.py

Taking all of this and re-arranging it as, we end up with the following script.

#!/usr/bin/env bash

set -euf -o pipefail

OSMCARTO_VERSION="v4.6.0"
OSMCARTO_LOCATION='https://github.com/gravitystorm/openstreetmap-carto.git'

rm -rf -- 'openstreetmap-carto'
git -c advice.detachedHead=false clone --quiet --depth 1 \
  --branch "${OSMCARTO_VERSION}" -- "${OSMCARTO_LOCATION}" 'openstreetmap-carto'
carto -a 3.0.12 'openstreetmap-carto/project.mml' > 'openstreetmap-carto/project.xml'

openstreetmap-carto/scripts/get-shapefiles.py

This is a repost of an entry on my blog.

To do something with OpenStreetMap data, we have to download it first. This can be the entire data from planet.openstreetmap.org or a smaller extract from a provider like Geofabrik. If you're doing this manually, it's easy. Just a single command will call curl or wget, or you can download it from the browser. If you want to script it, it's a bit harder. You have to worry about error conditions, what can go wrong, and make sure everything can happen unattended. So, to make sure we can do this, we write a simple bash script.

The goal of the script is to download the OSM data to a known file name, and return 0 if successful, or 1 if an error occurred. Also, to keep track of what was downloaded, we'll make two files with information on what was downloaded, and what state it's in: state.txt and configuration.txt. These will be compatible with osmosis, the standard tool for updating OpenStreetMap data.

Before doing anything else, we specify that this is a bash script, and that if anything goes wrong, the script is supposed to exit.

#!/usr/bin/env bash

set -euf -o pipefail

Next, we put the information about what's being downloaded, and where, into variables. It's traditional to use the Geofabrik Liechtenstein extract for testing, but the same scripts will work with the planet.

PLANET_FILE='data.osm.pbf'

PLANET_URL='http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf'
PLANET_MD5_URL="${PLANET_URL}.md5"

We'll be using curl to download the data, and every time we call it, we want to add the options -s and -L. Respectively, these make curl silent and cause it to follow redirects. Two files are needed: the data, and it's md5 sum. The md5 file looks something like 27f7... liechtenstein-latest.osm.pbf. The problem with this is we're saving the file as $PLANET_FILE, not liechtenstein-latest.osm.pbf. A bit of manipulation with cut fixes this.

CURL='curl -s -L'
MD5="$($CURL "${PLANET_MD5_URL}" | cut -f1 -d' ')"
echo "${MD5}  ${PLANET_FILE}" > "${PLANET_FILE}.md5"

The reason for downloading the md5 first is it reduces the time between the two downloads are initiated, making it less likely the server will have a new version uploading in that time.

The next step is easy, downloading the planet, and checking the download wasn't corrupted. It helps to have a good connection here.

$CURL -o "${PLANET_FILE}" "${PLANET_URL}" || { echo "Planet file failed to download"; exit 1; }

md5sum --quiet --status --strict -c "${PLANET_FILE}.md5" || { echo "md5 check failed"; exit 1; }

Libosmium is a popular library for manipulating OpenStreetMap data, and the osmium command can show metadata from the header of the file. The command osmium fileinfo data.osm.pbf tells us

Header:
  Bounding boxes:
    (9.47108,47.0477,9.63622,47.2713)
  With history: no
  Options:
    generator=osmium/1.5.1
    osmosis_replication_base_url=http://download.geofabrik.de/europe/liechtenstein-updates
    osmosis_replication_sequence_number=1764
    osmosis_replication_timestamp=2018-01-15T21:43:03Z
    pbf_dense_nodes=true
    timestamp=2018-01-15T21:43:03Z

The osmosis properties tell us where to go for the updates to the data we downloaded. Despite not needing the updates for this task, it's useful to store this in the state.txt and configuration.txt files mentioned above.

Rather than try to parse osmium's output, it has an option to just extract one field. We use this to get the base URL, and save that to configuration.txt

REPLICATION_BASE_URL="$(osmium fileinfo -g 'header.option.osmosis_replication_base_url' "${PLANET_FILE}")"
echo "baseUrl=${REPLICATION_BASE_URL}" > 'configuration.txt'

Replication sequence numbers needed to represented as a three-tiered directory structure, for example 123/456/789. By taking the number, padding it to 9 characters with 0s, and doing some sed magic, we get this format. From there, it's easy to download the state.txt file representing the state of the data that was downloaded.

REPLICATION_SEQUENCE_NUMBER="$( printf "%09d" "$(osmium fileinfo -g 'header.option.osmosis_replication_sequence_number' "${PLANET_FILE}")" | sed ':a;s@\B[0-9]\{3\}\>@/&@;ta' )"

$CURL -o 'state.txt' "${REPLICATION_BASE_URL}/${REPLICATION_SEQUENCE_NUMBER}.state.txt"

After all this has been run, we've got the planet, it's md5 file, and the state and configuration that correspond to the download.

Combining the code fragments, adding some comments, and cleaning up the files results in this shell script

#!/usr/bin/env bash

set -euf -o pipefail

PLANET_FILE='data.osm.pbf'

PLANET_URL='http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf'
PLANET_MD5_URL="${PLANET_URL}.md5"
CURL='curl -s -L'

# Clean up any remaining files
rm -f -- "${PLANET_FILE}" "${PLANET_FILE}.md5" 'state.txt' 'configuration.txt'

# Because the planet file name is set above, the provided md5 file needs altering
MD5="$($CURL "${PLANET_MD5_URL}" | cut -f1 -d' ')"
echo "${MD5}  ${PLANET_FILE}" > "${PLANET_FILE}.md5"

# Download the planet
$CURL -o "${PLANET_FILE}" "${PLANET_URL}" || { echo "Planet file failed to download"; exit 1; }

md5sum --quiet --status --strict -c "${PLANET_FILE}.md5" || { echo "md5 check failed"; exit 1; }

REPLICATION_BASE_URL="$(osmium fileinfo -g 'header.option.osmosis_replication_base_url' "${PLANET_FILE}")"
echo "baseUrl=${REPLICATION_BASE_URL}" > 'configuration.txt'

# sed to turn into / formatted, see https://unix.stackexchange.com/a/113798/149591
REPLICATION_SEQUENCE_NUMBER="$( printf "%09d" "$(osmium fileinfo -g 'header.option.osmosis_replication_sequence_number' "${PLANET_FILE}")" | sed ':a;s@\B[0-9]\{3\}\>@/&@;ta' )"

$CURL -o 'state.txt' "${REPLICATION_BASE_URL}/${REPLICATION_SEQUENCE_NUMBER}.state.txt"

I'm Paul Norman, OSM user pnorman. I've been mapping since 2010, and involved in other facets of OpenStreetMap since 2011. For the last three years, I’ve been on the OSMF board, and am running for re-election. During my time I’ve seen the board grow in productivity, the finances become more stable, and us make good strides in transparency.

Outside the board, I’m also involved with the OSMF on the Data Working Group, License Working Group, and Membership Working Group. As a software developer, I’m a maintainer of OpenStreetMap Carto and osm2pgsql, as well as being involved in many parts of rendering toolchain.

In my work life I’m an independent software developer, working on map rendering, cartography, and PostGIS for clients. My main contract right now is with Wikimedia Foundation, as the developer on their maps team. In the past I’ve worked for CartoDB, Mapquest, and other companies.

Looking back at what I put in my 2014 manifesto, I’m moderately pleased with the progress we’ve made in both transparency and productive board meetings. Neither are perfect, but they’re a vast improvement over three years. Overall, I’m satisfied with my time on the board. I accomplished some of what I wanted to, and think my manifesto desires were realistic.

My concerns are now

Conflicts of interest

6/7 board members work with OSM somehow in their jobs. This includes four with employers who sell services based on OSM data and can easily run into conflicts of interest. We are not managing this, which might have worked in the past, but is not a good practice. There’s stuff we need to set up like having an email discussion out of sight of the people with conflicts. Right now it’s considered acceptable for a board member to take part in discussions where they have a conflict of interest. Clear rules would also protect board members from pressure from their employer.

On a working group whenever there’s occasionally been an intersection between my work and the WG. In these cases I’ve removed myself from the discussion. This is what we should all be doing on the board.

Unfortunately, as someone who is paid to work with OSM data, I run into conflicts of interest myself, but in practice, I have less than most with the nature of who I work for.

Support, but not control

The job of the OSMF board is to support the mappers building the map, but not control them. I worry we are losing sight of that, and people increasingly want to exert control and consider the mappers secondary. We need to protect the ability for people to independently do activities, even if it’s not something the board agrees with.

Volunteer capacity

A lack of volunteers was an issue when I ran three years ago. It’s a bit better, but still one of the biggest issues facing the OSMF. Working groups need more people. A growing number of members have been attending board meetings, but I’d like to see multiple ones at every meeting. We need good people on the board, but we also need an active membership who are interested in what we do, watch us, what we do, track that we deliver, and offer appreciation in return.

Dear all,

Today, v4.3.0 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released.

Changes include

• Moving ford and emergency phone to a new tagging scheme
• Moving natural=tree to higher zoom level (z18+)
• Changing embassy color to brown
• Rendering name for waterway=dock
• The same line wrap of amenities for all zoom levels
• Fixing combined railway/highway ordering regression
• Fixing line wrapping bug in Docker
• Some documentation and code cleaning
• Improve ferry line text legibility
• Hide small theme parks and zoos
• Use solid lines for admin borders at low zooms

Thanks to all the contributors for this release, including stevenLAD, a new contributor.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v4.2.0...v4.3.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues

Dear all,

Today, v3.20 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released.

Changes include

• Render aeroway terminal buildings like other buildings
• Removed rendering of landuse=farm
• Added rendering for arts centre, fitness centre, plant nursery, mixed lift aerialways
• Rendering for fens changed
• Typography for point road-related features, addresses, and water features changed
• Removed rendering of waterway=canal as an area
• Take text properties of roads under construction from the type of road they will be

Thanks to all the contributors for this release including Richard Fairhurst and jnachtigall, new contributors.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v3.2.0...v3.1.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues.

Dear all,

Today, v3.1.0 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released.

Changes include

• Added coffee shop rendering
• Added health clinic rendering
• Adjusted place label typography
• Road shield rendering improvements
• Internal code cleanups

Thanks to all the contributors for this release.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v3.0.1...v3.1.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues.

As one of the more visible people in OpenStreetMap rendering and styling, I get a credit for a lot of the work done in projects like osm2pgsql, OpenStreetMap Carto, and other associated projects, sometimes undeservedly. A lot of people are less visible are doing work that we should be thankful for when looking back over the year, and deserve more credit. Here are three:

Sarah Hoffmann (lonvia)

Sarah is not just the administrator of the OSMF Nominatim server, she is the other maintainer of osm2pgsql, and has been responsible for many of the recent changes, and cleaning up and modernizing the code.

Matthijs Melissen (math1985)

As I’ve tried to step back from some of the maintainer tasks with OpenStreetMap Carto, Matthijs has stepped forward and kept up with the reviews, merges, and coordination.

Bas Couwenberg (sebastic)

Bas does most of the Debian packaging work for OpenStreetMap related software on Debian and Ubuntu. Most OpenStreetMap-related servers out there rely on his packaging work in some way. In 2011 setting up OpenStreetMap servers was a lot more complicated as you had to install many of the components from source

I can think of others, both within projects I contribute to and outside on other projects, but I wanted to keep this list short. Also, like many of my blog posts, this is ending up late and was supposed to be vaguely Christmas themed.

Dear all,

Today, v3.0.0 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released.

Major changes include

• Mapnik 3 is now required
• CartoCSS 0.16.x is now required
• Official Tilemill support is dropped
• Shapefiles are downloaded with a new python script

Changes include

• Noto Naskh is now used for Arabic
• Visual impact of campsites and quarries reduced below z13
• Wilderness huts rendered
• Subway entrances rendered

Thanks to all the contributors for this release including jojo4u, a new contributor.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v2.45.1...v3.0.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues

This is a repost from my blog because there's been a fair amount of interest from OSM people on what I wrote.

---

If you want to serve vector tiles, there are a few server options that have developed, each with different strengths and weaknesses.

node-mapnik based

Language: nodejs
Layer definitions: Mapnik layer definitions in XML, typically preprocessed from YAML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

Kartotherian, tessera, and other servers based on tilelive all rely on Node bindings to Mapnik to produce vector tiles. They all work with Mapnik layer definitions. This is a reasonably well understood language and consists primarily of a SQL statement for each layer. This is reasonably flexable and it's possible to do proper code review, git conflict resolution, and other processes you need with an open style.

Some servers can turn the Mapbox Vector Tiles into GeoJSON, but not all do. There are other minor differences, but they all have the same major advantages and disadvantages.

The biggest problem with these options is that you have to either use the exact same versions of everything as the Mapbox developers while hoping their changes work with your code, or lock down your versions to a set of known good versions and periodically update when you need new features, retesting all your code. Neither of these is practical for an open-source style which wants to involve others.

If you don’t do this, you’ll find parts of your server failing with different combinations of Mapnik and node-mapnik.

Tilezen tileserver

Language: Python
Layer definitions: SQL in jinja2 templates, YAML
Vector tile formats: Mapbox Vector Tiles, TopoJSON, and GeoJSON
Data source support: PostGIS

Tilezen tileserver was written by Mapzen to replace their TileStache-based vector tile generation. Having been written by developers who wrote previous vector tile servers, it combines ideas and functionality other options don't have.

The datasource definitions are written in SQL + YAML, a common choice, but unlike other options, the SQL is in its own files which are preprocessed by the jinja2 templating engine. This adds some complexity, but a great deal of power. Selecting different features by zoom level normally requires repetative SQL and lengthy UNION ALL queries, but the preprocessing allows queries to be written more naturally.

Tileserver's unique feature is the post-processing capabilities it offers. This allows vector tiles to be operated on after the database, altering geometries, changing attributes, and combining geometries. Post-processing to reduce size is a necessary feature if targeting mobile devices on slower connections. Mapbox had been working on this in the open, but now that they no longer use node-mapnik it's not clear how they do so. MapQuest had developed Avecado to specifically target this, but it became abandoned when they stopped doing their own map serving.

You don't need any AWS services for a basic Tilezen tileserver deployment, but there might be some dependencies in the more advanced features needed to set up a full production environment.

Tegola

Language: Go
Layer definitions: SQL in TOML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

Tegola is a new server written in Go. It operates with multiple providers which supply layers to maps, allowing them to be assembled different ways. It looks like it has most of the features needed for vector tiles for a basemap, but might be missing a few needed for changing data as zoom changes.

SQL in TOML is similar to SQL in YAML for layer definitions, and like this it is reasonably flexable and makes it possible to do proper code review, git conflict resolution, and other processes you need with an open style.

I haven't had a chance to deploy it yet, so I'm not sure what difficulties there are.

t-rex

Language: Rust
Layer definitions: SQL in TOML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

t-rex is a new server written in Rust. It's unique feature it that it can auto-configure layers from PostGIS tables. It does have all the required features for selecting appropriate data in a basemap.

It's layer definitions are different than Tegola's, but they are both SQL in TOML, and share the same strengths.

Like Tegola, I haven't had a chance to deploy it.

TileStache

Language: Python
Layer definitions: SQL in JSON Vector tile formats: Mapbox Vector Tiles, TopoJSON, GeoJSON, and Arc GeoServices JSON Data source support: PostGIS

TileStache is a general-purpose tile server which Mapzen used to use a fork of to serve their Tilezen schema. They've switched to Tilezen tileserver, but the functionality they added has been merged back into TileStache. Unfortunately, the documentation hasn't caught up yet, so there's not too much information about all of its functionality.

Deploying TileStache tends to be reasonable - particularly compared to node-mapnik - but the language of SQL in JSON is one that's a problem for open projects with multiple authors and prevents proper code review and git conflict resolution.

Tilemaker

Language: C++
Layer definitions: Lua
Vector tile formats: Mapbox Vector Tiles
Data source support: OSM PBF and shapefiles

Tilemaker is built around the idea of vector tiles without a serving stack. It does this by doing an in-memory conversion directly from OSM PBF data to pre-generated vector tiles, which can then be served using Apache, a S3 bucket, or any means of serving files from disk. This vastly simplifies deployment and reduces sources of downtime.

For serving a city or most countries this can be the ideal method, but the same strengths that make it good for this are a problem for processing the planet. It takes large amounts of RAM, can’t consume minutely changes, and has to create vector tiles for the entire PBF at once.

Tilemaker is also the only server to support directly using shapefiles for low zoom data and OSM for high zoom. Other options require loading into PostGIS and using SQL that selects the appropriate data based on zoom.

VectorTileCreator

Language: Python
Layer definitions: osmfilter options
Vector tile formats: o5m
Data source support: OSM PBF and other raw OSM data

VectorTileCreator is part of KDE Marble and takes the unique approach of creating tiles of raw OSM data. It uses osmfilter's language for filtering OSM data, but lacks the means to use other data sources, something most maps will need. The support of o5m vector tiles is also limited. Like tilemaker it runs from the command line and produces a set of vector tiles.

Which should I use?

What you should use depends on your needs. First figure out what support you need for the full planet, updates, data sources, and output formats. If you need diff update support, then you need something that can create a single vector tile and Tilemaker won't work. If you need TopoJSON support, node-mapnik won't work.

For a table showing features supported, see the original blog post

Dear all,

Today, v2.44.1 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released. Also, v2.44.0 was released last month without an email, so this email includes changes in both.

v2.44.0 has been rolled out to the openstreetmap.org servers, but v2.44.1 has not yet.

Major changes are

• Rendering of restricted access roads and paths significantly changed
• Changed to use Noto fonts for all languages

Other changes in both versions include

• A code of conduct adopted, based on the Go code of conduct
• Adjustments to city wall rendering
• Revised low zoom place rendering
• Render both house name and number if address has both

Thanks to all the contributors for this release, in particular Lukas Sommer, Hsiao-Ting Yu and vholten for work in debugging complex font issues with the Noto CJK fonts.

For a full list of commits from both releases, see https://github.com/gravitystorm/openstreetmap-carto/compare/v2.43.0...v2.44.1

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues

Dear all,

Today, v2.43.0 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released. It has not yet been rolled out to the openstreetmap.org servers.

Changes include

• Adjust alotments pattern
• Whitespace cleanups of code
• Adjust colours of dog parks and construction sites
• Increase font size of addresses
• Fix combination of long names and oneway arrows

Thanks to all the contributors for this release, including Ircama and measad, both new contributors.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v2.42.0...v2.43.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues.

I've been experimenting with generating my own vector tiles and client-side rendering with Tangram in order to figure out how to best write a style in its language.

Tangram is a GL-based renderer written by Mapzen and normally used with their Tilezen vector tiles, but I'm interested in being able to make my own vector tiles with different cartographic choices.

Having a diverse selection of vector tile schemas is important, as is avoiding a situation where only large players in the market can get involved like right now.

For a toolchain I used osm2pgsql with ClearTables and Mapnik via Kosmtik to write vector tiles. With the demo I'm serving pre-rendered vector tiles from disk, but Kosmtik is useful in development with it's xray functionality. I input the style into Tangram Play, a web-based editor that automatically reloads the map when you change the style.

The cartography and vector tile definitions are loosely based on OSM Clear, a demo style I wrote. I didn't want to learn the language while designing new cartography at the same time. Being a learning exercise I don't consider the style complete or free of bugs.

The demo page is on my server at http://tangram-clear-demo.faramir.paulnorman.ca/ with the style and vector tile code at https://github.com/ClearTables/tangram-clear-demo.

I'm not sure what direction I'm going to take next as I don't have any particular style goals right now, or collaborators.

Cross-post from https://lists.openstreetmap.org/pipermail/dev/2016-August/029448.html

Today, v2.41.0 of the openstreetmap-carto stylesheet (the default stylesheet on openstreetmap.org) has been released.

Changes include

• More consistent fonts for POI labels
• Less saturated stadiums
• Rendering obelisks and dog parks
• An updated list of font packages
• Cleaning up the font list
• Rewriting the road colours script for easier changes
• Various bug fixes

Thanks to all the contributors, including jdhoek, a new contributor.

For a full list of commits, see https://github.com/gravitystorm/openstreetmap-carto/compare/v2.40.0...v2.41.0

As always, we welcome any bug reports at https://github.com/gravitystorm/openstreetmap-carto/issues.

This is a repost from my blog

Periodically people make the claim of over 2 million active users for OpenStreetMap, but what this mean? This is the total number of accounts, including those who never edited, those who left long ago, spammers, and actual active contributors.

The closest metric to a standard is active users over the last 30 days. Although we can't get that number, we can look at the changeset dump and analyze it with ChangesetMD and some SQL.

The SQL is fairly simple.

SELECT COUNT(DISTINCT user_id) AS active_users, date::date FROM osm_changeset JOIN generate_series('2007-01-01'::timestamp, '2015-12-31'::timestamp, '1 day') AS d(date) ON (created_at <= d.date AND created_at > d.date - '30 days'::interval) GROUP BY date ORDER BY date ASC;

OpenStreetMap was around before 2007, but the way data was stored was different so changeset dumps aren't reliable that far back.

Taking the resulting file and a bit of gnuplot magic gives us a graph.

There's been a steady upwards trend with strong seasonal variations. This makes sense, since mapping is an outdoors activity.

Why does this matter? Besides accuracy, it's important to be using a meaningful number when looking at growth.

Another reason is comparing with any other membership numbers. We want the OpenStreetMap Foundation to be representative of OSM contributors, but we need to measure that against a realistic number. The OSMF has 700 members, or 2-3% of OSM contributors. This could use improvement, but is in the normal range for foundations. If we incorrectly measured against the total number we'd get 0.03%, an absurdly wrong number.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I've added annotations in italics where I felt they would be useful.

Yantisa Akhadi

• OSM User Iyan
• Manifesto

Questions

Where do you currently participate in the OSMF?

I want to contribute in the Engineering Working Group, unfortunately the meeting time is equal with 00.30am my time. I hope in the future there will be meeting time to cater peoples from different timezones.

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

her than edits, which is still not much, I was more interested in writing and introduce people about OSM. I have given talks in multiple local, national, international levels to introduce OSM as well as its application in disaster management. There have been multiple blogpost written in our OSM-ID website to tell the story about the event. I am also actively participated in OSM-ID Facebook group, since this much more popular channel compared to OSM-ID mailing list.

Do you use OSM at work for business purposes?

Yes, I currently work as Humanitarian OpenStreetMap Team Project Manager for Indonesia. Here in Indonesia, we are focusing on how we can use OSM in data preparedness for disaster. In the past 4 years, our team have trained more than 2,400 peoples on data collection using OSM, QGIS and InaSAFE in various phases of disaster as well as economic development.

Conflicts of Interest

I currently work as Project Manager for Humanitarian OpenStreetMap Team (HOT) Indonesia. In my experiences, any interest of OSM or OSMF would also bring benefits to HOT. So far I haven't identify how there will be conflicts of interest. Sometimes people identify me from OSM Indonesia rather than HOT, since people tends to more familiar with it. HOT also support OSM community growth in Indonesia by providing resources and deliver OSM outreach to various institutions.

Who should the OSMF serve?

OSMF needs to serve the goal of the foundation which in the end it can only be achieved by serving its members and the OSM users at large. This would also relates with the diversity of its member and users throughout the world, does they already accommodated and represented through its policy and action? Here I would like to represent Asia communities, one of the minorities in term of board/member representation.

Role of the board

Has not responded.

Communication with the community

I am mostly active on Facebook Groups, since in Indonesia, that is where the biggest interaction happens. The OSM-ID talks is pretty quiet. I also like to read the user blogs. I also often meet OSM colleagues from Asia in SOTM Asia or FOSS4G-Asia where we regularly share stories and activities in our country. Occasionally I read articles on OSM from Twitter or shared in Facebook by fellow OSM.

Community involvement in OSMF

I think this relates with transparency as well, other than release the meeting notes, OSMF needs to be more active in social media. It can be by providing regular updates about OSMF activities and working group meeting schedule through Twitter or Facebook. More importantly, we need to listen what members want and what needs to be improved.

Diversity

I am an Indonesian. I once heard that I'm the only Indonesian joining OSMF as a member. I hope that is wrong. I hope for more active involvement from OSMF members in the region. It would be great if there is more people from my region to join OSMF. I still feel that despite our efforts to continuously proliferate OSM in the region, we still have lots of thing to do. The OSM community in Asia is growing, in Indonesia alone, there are more than 3,000 people joins OSM ID Facebook groups, not to mention OSM Groups formed by universities students. I belief that diversity would keep the OSM community rich, vibrant and alive, yet it is also important to direct these energy to move forward and map the world.

Paying People

Has not responded.

Board term limits

I believe members should be the one to decide such limit and duration. As an OSMF member myself, I choose the board based on how they performed or based on their manifesto and answering QA.

Transparency

I support transparency in OSMF board's work since it would provide more open environment where member can also contribute and discuss. I do realize that there are some private matter that cannot be discussed due to legal constraint.

License violation enforcement

I think the existing mechanism works quite well. We cannot always throw every license violation case to OSMF, but surely we can provide mechanism on how to follow up or escalate the issues related to license and possibly involving local communities in the areas.

Imports and remote mapping stance

I support Import as far as it follows the Import Guidelines. I have couple of opportunities where government institutions or organization want to put their spatial data in OSM. Especially as they saw this as an opportunity to join cause in open data movement and their data licence is compatible with ODbL. With remote 'armchair' mapping, there should be guidelines as well to reduce common mistakes and assumption.

Commercial and Organized Editing Policy stance

Has not responded.

Changelog

Added new answers.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I've added annotations in italics where I felt they would be useful.

Douglas Ssebaggala

• Location: Uganda
• OSM User Douglo
• Manifesto reproduced below

First edit was 4 years ago, and have not looked back ever since, getting involved in other activities beyond mapping, with these upcoming OSMF Board elections, i have great anticipation to show how global the OSM project is: one of the ways that the global reach can be reflected is to have an intercontinental representation on the OSMF Board. Some of the gaps that (i think) need to be covered is how the OSMF relates to the community of users, being part of the awesome board members, and well wishers will greatly improve on how best this relationship can be communicated, and strengthened.

Questions

Where do you currently participate in the OSMF?

I have participated in the Local Chapters Working Group, and would be willing to have more frequent involvement in this WG.

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

• History of edits
• Potlatch Manual - made back in the old days
• OSM Events and Activities in Uganda
• Activating the Ugandan mailing list
• Contributed on other OSM mailing lists (OSM-Talk).
• Most of all, would like to see OSM grow both as a data/map database and as well as a community.

Do you use OSM at work for business purposes?

Yes. To date, mapping coordinator for Uganda, with a mix of voluntary work under a local non-profit Organisation Fruits of Thought

Conflicts of Interest

No, there would be no conflict of interest, most of the work i do is for the advancement and betterment of OSM, which would most likely be inline with the OSMF

Who should the OSMF serve?

Has not responded.

Role of the board

Has not responded.

Communication with the community

Has not responded.

Community involvement in OSMF

Has not responded.

Diversity

From Uganda to East Africa and the whole of Africa, there are traces of OSM presence, with some unknown communities doing great efforts and work with OSM. My presence would prove and bring this in light through the team support of the OSMF, and providing some more time to the efforts of the LCWG.

Paying People

Has not responded.

Board term limits

Has not responded.

Transparency

Has not responded.

License violation enforcement

Has not responded.

Imports and remote mapping stance

Has not responded.

Commercial and Organized Editing Policy stance

Has not responded.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I've added annotations in italics where I felt they would be useful.

Martijn van Exel

• OSM User mvexel
• Manifesto reproduced below

Bringing almost a decade of grassroots OSM experience and 4 years of US Chapter board experience to the table. Read my OSM diary to learn more about me. Please get in touch with me if you have questions, or to let me know what you want from the OSMF board. I work for Telenav on OSM

Questions

Where do you currently participate in the OSMF?

Board member of OSM US 2011-2015. Started User:Mvexel/Welcome_Working_Group (now dormant). Participated in LWG and CWG, but not particularly actively.

Note: OSM US work is not part of OSMF

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

Some highlights: * Mapping since 2007 (see my profile). * (Co-)Organizer of many events small and large: SOTM '09, SOTM US '12, '13, '14, '15. Local events in Amsterdam and Salt Lake City since 2008. * Started local groups in Amsterdam and Salt Lake City. * Blogging about OSM, first on my own blog and now on my OSM diary. * Author of MapRoulette (together with emacsen) and many other smaller tools. See my Github page. * Active participant in OSM mailing lists and IRC channels.activites * Did a whole lot of talks and workshops on OSM since 2010. * Find me on Twitter, LinkedIn. * Spending part of my time at Telenav creating and releasing useful data and tools for mappers. * Taught OSM to Geography students.

Do you use OSM at work for business purposes?

Yes. My work at Telenav revolves around OSM. We use OSM data to power many of our navigation products. My job is to ensure that OSM is the best map it can be for that purpose. If you read my diary, you will get a sense of what that means. A lot of my time goes into working on things that are a mutual benefit to Telenav and OSM mappers. Recent examples are the Missing Roads and Traffic Flow Direction JOSM plugins. More good stuff in the pipeline!

Conflicts of Interest

I work for Telenav on OSM which may raise a COI from time to time. I will follow the COI policy. If there is none, I will bring experience of defining one for OSM US to the table to implement one as soon as possible.

Who should the OSMF serve?

The OSMF board serves the members of the foundation. If folks feel that there is not proper representation of the community, then we should have more people from the community sign up as members. Again, I see a role there for local chapters.

Role of the board

Has not responded.

Communication with the community

I only subscribe to talk, talk-us and legal-talk. I read the forum very occasionally. I read IRC and participate there. I go to any SOTM conference I can feasibly get to. I spend time on Twitter, Facebook, LinkedIn and most importantly the OSM diaries to learn about new developments in OSM. I follow what's going on on Github. I email / talk with a lot of OSM friends personally on a regular basis. In my professional network I talk about OSM a lot, learning what other companies and organizations are doing with OSM. Like the office hours idea, I started something similar in the US a while ago with 'Mappy Hours' on Google Hangouts.

Community involvement in OSMF

Local chapters are key. When someone becomes a member of a chapter, it should be easy (automatic even?) to also become a member of the foundation. Our experience in the US shows that folks often don't even know these things are separate.

Diversity

My work with the US chapter board has involved a lot of thinking about and working on creating a more inclusive and diverse community. See how we did for example with SOTM US this past spring. This was a team effort but I do bring this experience with me. Also having had a leadership role communities in both Europe and the US gives me some perspective on diversity.

Paying People

Entirely in favor. What is more, some key responsibilities all but require professionals or would greatly benefit from having them: legal, finance, marketing and PR.

Board term limits

Has not responded.

Transparency

Fundamentally the board's proceedings should be open, but at its discretion confidentially needs to always be an option.

License violation enforcement

I think we should retain legal council to address license issues and especially move forward the ODbL guidelines. We cannot leave this to IANALs or, worse, external stakeholders.

Note: Martijn has recently joined the LWG

Imports and remote mapping stance

The solution to bad data entering OSM, be it through ill-advised imports or manual edits, is not in more policing, but in better onboarding of new mappers. Initiatives like LearnOSM / TeachOSM / Mapgive are great but it would be even better if we would provide these resources ourselves. Over time, the days of every mapper having full access to editing OSM from the day they sign up may need to come to an end. I see a role for a reboot of the Welcome WG.

Commercial and Organized Editing Policy stance

Don't think so, rather we should engage with these organizations to make sure they have the tools to teach their people how to map. I would like to see some data on the lack of intrinsic motivation of paid mappers, my anecdotal evidence points in a different direction altogether.

Note: Martijn leads the Telenav editing team

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I've added annotations in italics where I felt they would be useful.

Gonzalo Perez

• Location: Argentina
• OSM User Zalitoar
• Manifesto quoted below

I wish to serve in a broader sense to the project. The projects and activities that we have done in South America in the last year boosted the growth of the communities in this part of the world, a strategy which I consider necessary and possible in other regions.

Questions

Gonzalo has not responded to any questions, and his manifesto is too short to fill in answers from.

This post was an accidental duplicate, and is deleted.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I've added annotations in italics where I felt they would be useful.

Wille Marcel

• Location: Brazil
• OSM User wille
• Manifesto

Questions

Where do you currently participate in the OSMF?

I have contributed to the Communication Working Group.

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

Beyond edits, my main contributions are: * Talks about OSM in conferences in Brazil. See on [https://www.openstreetmap.org/user/wille/diary my diary]. * Organize local meetings. See on [https://www.openstreetmap.org/user/wille/diary my diary]. * I am one of the managers of [http://twitter.com/openstreetmapbr @OpenStreetMapBR] twitter account and I have done some efforts to put OSM on the media in Brazil. * I am developing a software to detect harmful changesets, it's divided in two parts: [https://github.com/willemarcel/osmcha osmcha] (python library) and [https://github.com/willemarcel/osmcha-django osmcha-django] (web interface). In the next weeks I'll commit some features and officially release it. * Contributed to translation of some softwares, like iD editor. * I've done some more contributions to OSM related software, see on my [https://github.com/willemarcel/ github]

Do you use OSM at work for business purposes?

I am a GIS and web developer. Sometimes I use data or tiles of OSM on my work, but it is not the main part of my work.

Conflicts of Interest

I don't see risk of conflicts of interest with my job.

Who should the OSMF serve?

I think OSMF must not control OSM, but it must do all the possible to make OSM bigger. I think my main priorities will be to support local communities in places like Latin America, Africa and Asia, where we don't have a community as big as in Europe or United States.

Role of the board

Has not responded.

Communication with the community

I try to read the user diaries in English, Spanish and Portuguese periodically. I participate of OSMF-Talk mailing list, as well as the Brazilian and Latin American lists and telegram groups.

Community involvement in OSMF

I think we can do simple things to give more visibility to the Foundation, like put a campaign on the homepage to increase the number of members, give more support to local communities and events. I believe we will achieve more involvement between the community and the Foundation only when the Foundation wish to be more present on the life of the project.

Diversity

One of the reasons I am running for OSMF is to put a vision from Latin America on the board. I participate both in the Brazilian and Latin American communities. I believe OSMF should give more support to local communities in regions like Asia, Africa and Latin America. I also think we should seek resources to have more participants on the FOSS Outreach Program and start a program more focused on teach mapping skills to women and other underrepresented groups.

Paying People

I am in favor of paid work on the Foundation.

Board term limits

I think it's essential to have diversity on the board, but I don't see as crucial to have a term limit, since the members choose the board by voting.

Transparency

I don't have nothing against open vote reporting and I believe transparency is an essential value.

License violation enforcement

I have notified some organizations that used OSM without give attribution. In most part of the cases, it was unintentional and solved easily.

Imports and remote mapping stance

I think imports must be discussed in each local community and the data to be imported needs to have great quality. Remote mapping can be useful in some occasions, but what we need is people all over the world to map what can not be seen in satellite imagery.

Commercial and Organized Editing Policy stance

It's natural that there are companies mapping and, until now, we don't have many incidents, so I also think we needn't worry about this now.