OpenStreetMap

migurski's diary

Recent diary entries

It’s Easier To Contribute to OSM’s Website Now

Posted by migurski on 22 March 2021 in English (English).

If you’ve ever wanted to help build the OpenStreetMap website but found our codebase intimidating, you’re in luck. With the recent addition of Jamie Alessio’s Docker support in pull request #2409, it’s now easier to contribute well-tested changes to the OSM site. I’ve never been comfortable with Rails, Chef, or other Ruby code like the kind we use to build OSM.org. Many helpful changes to the site can be made with small front-end tweaks in just HTML or CSS. The new support for Docker provides a quick way to install a working version of the OSM site on your local computer and preview changes before you contribute.

Here’s a recent change I made to implement Andy Allan’s suggestion for improved inbox/outbox navigation, using Bootstrap navigation tabs in place of plain links. This small design tweak is now live on OSM.org My Messages:

new Inbox image

Contrast this with the old look, with simple HTML links in the page header that don’t quite look like usable navigation:

old Inbox image

This change took just a few minutes, but I would not have been able to test and contribute without a way to easily run OSM.org locally.

Follow Along

Follow the directions in DOCKER.md to set up a running local version of OSM.org on your computer. It takes about 5-10min to get everything working on my Macbook Pro laptop. When you’re done you can navigate to http://0.0.0.0:3000 to see your local version running (give it about 30 seconds on first load; Rails seems to require some boot time to be ready).

To test features that require a logged-in user, navigate to http://0.0.0.0:3000/user/new and create your account:

New account page

The next screen mentions a confirmation email, which you will not receive because you don’t have email configured. Fortunately, there’s a workaround:

Confirmation email

The link that would be in the email is also printed to your docker-compose logs -f terminal output. Copy the part starting with “/user” after “openstreetmap.example.com”, add it after “http://0.0.0.0:3000” in your browser, and you’ll have a working user account:

Terminal log

That’s it! When you submit your contribution as a pull request, you can include screen shots showing the results of your work so it’s easier for community members and site maintainers to understand your proposal. A full code test suite will be run automatically as you update Github with your changes, so you’ll know ahead of time if your changes are safe.

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Summary Maps of OSMF 2021 Survey Results (Part 2)

Posted by migurski on 26 February 2021 in English (English).

This post is a followup to yesterday’s maps of OSMF 2021 Survey results. We looked at demographics of survey respondents and their feedback to the board on 2020 actions like establishing new committees and fundraising.

I’ll start with a recap of survey respondents vs. the general editing population of OSM. Jennings Anderson also posted yesterday with a view of worldwide mapping behavior and a simple methodology for determining who’s a local mapper for each survey region. This first map shows the number of survey respondents per 1k local OSM mappers, ranging from fewer than 10 in Indonesia and the Middle East to 35+ in Belarus:

xxxxx

The geographic survey responses are aggregated to preserve anonymity, in some cases grouping dozens of countries under a single region. The maps on this page show simple survey results using the Board’s country-to-region mapping to make the responses easier to interpret. To re-create the map above, and all other maps in this post, source tabular data, region GeoJSON, and a QGIS file are available in a Github gist.

All maps here should be considered “No Rights Reserved,” CC0, or public domain.

2021 Board Priorities

Survey respondents were asked to prioritize the OSMF’s activities in the next year by ranking seven possible tasks in order:

  1. Stability of the core infrastructure (hardware, software, human capital)
  2. Outreach to Local Chapters and Communities
  3. Takeover protection
  4. Fund-raising
  5. Recruitment for Working Groups
  6. Attribution guidelines
  7. Brexit

The rank order makes this difficult to summarize on a map. Fortunately, the OSMF already elects its board members using a ranked-choice voting system called Scottish Single-Transferable Vote, so I’ve applied that method to the survey responses. I’ve scored priorities for each separate region with one, two, and three possible winning choices to determine overall community sentiment about each of the seven possible choices.

First priority for OSMF by region

There is clear, strong alignment on core infrastructure stability as the top global priority. Almost all regions chose this as their first priority, and the ones that did not chose it as their second. Stability is a clear winner.

Second priority for OSMF by region

Takeover Protection and Outreach emerge as the next two priorities, approximately neck-and-neck. Protection from hostile takeover of the OSMF is important to European respondents, while outreach to local chapters and communities has a strong showing elsewhere.

Third priority for OSMF by region

Fund-raising rounds out the top half of the priority list for the board. Working group recruitment and attribution guidelines appear in just one region apiece, and Brexit did not appear important to respondents at all. I interpret this to mean that Brexit-related changes for the UK-based Foundation are an administrative or procedural task that the Board is expected to handle through delegation.

Technology Policy

Survey respondents were asked for their opinion on two hot-button technical issues, AI/ML techniques for expanding the map (sometimes known as “robot mapping”) and a long asked-for vector tiles service to complement the existing OSM.org raster tiles.

Regarding AI/ML, the survey asked for a single response to this question:

Some OSM community members favor use of artificial intelligence and machine learning tools to assist mapping.  Others oppose use of these tools.  The Board of Directors of the OpenStreetMap Foundation has taken no position on the matter since such a decision would likely violate the core principle that the Board does not tell anyone how to map or what to map.  Should the Board…

  • “…adopt a policy banning artificial intelligence/machine learning tools?”
  • “…continue to take no position (i.e., leave the issue to local communities), since the Foundation tells nobody what or how to map?”
  • “…adopt a policy formally approving artificial intelligence/machine learning tools (blanket approval)?”
  • “…adopt a policy formally approving artificial intelligence/machine learning tools but only if local communities agree?”
  • No opinion

Community responses settled on two leading approaches: continuing to take no position, or formal approval of artificial intelligence/machine learning tools conditionally if local communities agree.

OSMF AI/ML Policy

Regarding vector tiles, the survey asked for a single response to this question:

Some OSM community members think the Board should fund change of the map displayed on openstreetmap.org from raster tiles to vector tiles.  Bearing in mind the potential cost of such an undertaking, and the possible impact on the OSM ecosystem, in your opinion the Board should…

  • “…pay developers to create vector tiles for openstreetmap.org, but in a manner calculated not to compete with commercial developers.”
  • “…do nothing in this regard.”
  • “…encourage volunteers to create vector tiles for openstreetmap.org, but nothing more.”
  • “…pay developers to create vector tiles calculated to demonstrate the full breadth of OSM data.”
  • No opinion

Community responses settled on three leading approaches: encouraging volunteers, paying developers to demonstrate full breadth, or not offering an opinion at all.

OSMF Vector Tile Policy

Conclusion

This post and its predecessor summarized OSMF 2021 Survey responses providing feedback to on past actions and future priorities. If you’d like to dig deeper into the underlying mapping population to put the regional survey results into context, please visit Jennings Anderson’s earlier post with code and data

Source data and files for all the maps above are available in a Github gist, including a GeoJSON file with survey response regions you can use to map official results.

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Summary Maps of OSMF 2021 Survey Results

Posted by migurski on 26 February 2021 in English (English).

OpenStreetMap’s 2021 survey wrapped up just over ten days ago, and the Board has posted summarized results on the Wiki. The geographic survey responses are aggregated to preserve anonymity, in some cases grouping dozens of countries under a single region. The maps below show simple survey results using the Board’s country-to-region mapping to make the responses easier to interpret.

All maps below should be considered “No Rights Reserved,” CC0, or public domain.

Mappers

Let’s start with the mappers. The majority of survey respondents worldwide agreed that the term “mapper” described their involvement with OSM:

“Optional Demographic: Are you a mapper?”

To re-create the map above, and all other maps in this post, source tabular data, region GeoJSON, and a QGIS file are available in a Github gist.

Jennings Anderson posted earlier today with a view of worldwide mapping behavior to put the above results into context. He provides a simple methodology for determining who’s a local mapper for each survey region:

Assuming the mapper “lives” in this country is perhaps inferring too much, but we can definitively say that this mapper makes the majority of their recurring edits within this country, so they must have more knowledge of this country than any other.

The maps below show the prevalence of local and non-local mappers with more than one day of editing activity in each regional population according to Jennings’s data, with a high of 100-200 mappers per 1m total population in Western Europe and lows of ~0-25 mappers per 1m total population across South America, Africa, and Asia.

Local Mappers with >1 days editing experience

Non-Local Mappers with >1 days editing experience

What Mappers Want

The survey asked for sentiment responses to five questions about board activities. Each question asked for feedback on recent board actions on a scale of 1 to 5, where 1 is strongly disagree, 2 is disagree, 3 is neutral, 4 is agree, and 5 is strongly agree. Sentiment on each of the five questions is neutral-to-positive to varying amounts, with no average negative sentiment recorded in any geographic region.

Action 1: Code Of Conduct

“In 2020 the Board created a Diversity and Inclusion Special Committee, and asked the Local Chapters and Communities Working Group to recommend actions in response to complaints about Systemic Offensive Behavior in the OSM Community.”

Feedback on Code Of Conduct

Action 2: Fundraising

“In 2020 the Board began raising funds via large donations to cover projected increased costs of maintaining the platform (hardware, software, personnel) in the face of rapidly growing demand for OSM data.”

Feedback on Fundraising

Action 3: Software Investments

“In 2020 the Board began funding small-scale, one-time, limited-scope, short-term projects to improve existing software products that are considered core to OSM: Nominatim, osm2pgsql, and Potlatch. These products had previously been maintained wholly by volunteers.”

Feedback on Software Investments

Action 4: iD Editor

“In 2020 the Board voted to take over the cost of a single software developer for maintaining the default iD editor from a private firm that had been funding the position, thereby shifting control of iD maintenance from a third party to the OSM community.”

Feedback on iD Editor

Action 5: Microgrants

“In 2020 the Board began funding microgrants for one-off projects.”

Feedback on Microgrants

Survey Respondents

Similar to the Mappers example above, survey respondents were asked whether they identified with a non-exclusive list of identities. After “mappers,” the largest numbers of respondents identified as “data users” and members of “local communities without chapter status.”

Optional Demographic: data user

Optional Demographic:  Are you a member of a local community without chapter status

Joiners

Many people indicated involvement with formal OSM activities, such as membership in working groups, local chapters, and the OSM Foundation itself.

Optional Demographic:  Are you a member of a working group

Optional Demographic:  Are you a member of a local chapter

Optional Demographic:  Are you a member of the OpenStreetMap Foundation

Organizers

Communication and event organization featured more heavily in the global South and West.

Optional Demographic: Are you a communicator

Optional Demographic: Are you a event organizer

Workers

Some people agreed that they work for OSM-involved organizations, such as corporate sponsors, commercial companies using OSM data, or non-profit organizations using OSM data.

Optional Demographic:  Are you a member of a corporate sponsor of OSM

Optional Demographic:  Are you a member of a commercial company using OSM data

Optional Demographic:  Are you a member of a non-profit organization using OSM data

Nerds

Finally, a small number of respondents are software developers, maintainers, and sysadmins.

Optional Demographic: Are you a developer / maintainer

Optional Demographic: Are you a hardware / systems operator

Next Steps

This post summarizes OSMF 2021 Survey responses providing board feedback on past actions and basic activities. In a followup post, I’ll share maps of ranked choices of future priority order of tasks the Board of Directors has set for itself. If you’d like to dig deeper into the underlying mapping population to put the regional survey results into context, please visit Jennings Anderson’s earlier post with code and data

Source data and files for all the maps above are available in a Github gist, including a GeoJSON file with survey response regions you can use to map official results.

Update

I’ve published a follow-up post with maps of ranked choices of future priority order of tasks.

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

I’m Running for OSMF Board

Posted by migurski on 30 November 2020 in English (English).

I’ve made OpenStreetMap a major part of my life and work since 2005. It’s not just an audacious, community-built, complete, freely-licensed, street-scale map of the world. OSM is also a big tent that collects the skills and support of a huge range of individual and organizational contributors.

In 2021, OSM’s community has two opportunities to grow stronger together: we should make the OSM organization support a wider diversity of participants and we must succeed at starting to manage our technical operations professionally.

I’m a good candidate to help with both of these existing 2020 OSMF board goals. As a product and engineering leader in several organizations, I can help the Foundation succeed at finding and keeping great engineering talent. With my history on the community and business sides of OSM, I am experienced in making open data attractive to new community members and soliciting support from large organizations.

I’d like to make myself available for conversations with anyone who has questions about my candidacy, manifesto, priorities, or really anything else. I’m blocking these four times over the next two weeks prior to the close of voting and AGM on Dec 12; get in touch here or via my personal email if you’d like to chat by text, voice, or video!

Read my complete manifesto on the Wiki for more about why I think I’d make a good OSMF board member.

Location: Longfellow, Oakland, Alameda County, California, 94609, United States

Daylight’s Fourth Release and New Website

Posted by migurski on 26 September 2020 in English (English).

Facebook is releasing an update to Daylight, our complete, downloadable preview of OpenStreetMap data.

📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.4.osm.pbf (59.5GB).

Version 0.4 marks the first of our expected monthly Daylight releases through the end of 2020. With this release, we are including new coastline data, updated Microsoft ML Building Footprints data, and for the first time we’re including data about specific features we’ve excluded from a Daylight release.

There’s also a new site with a machine-readable feed of future Daylight releases: DaylightMap.org.

Read the complete announcement here.

Daylight Map Distribution

Coastline example

Fixed Sevier Lake

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Just Released: Third Update to Daylight Map Distribution

Posted by migurski on 4 September 2020 in English (English).

Facebook is releasing a third update to Daylight, our complete, downloadable preview of OpenStreetMap data.

📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.3.osm.pbf (57GB).

Earlier this year, Facebook released Daylight as part of our work with OpenStreetMap. We use maps to let our users find friends, businesses, groups and much more. OpenStreetMap (OSM), the open source wiki map, has a substantial global footprint of map data built and maintained by a dedicated community of global mappers making OSM a natural choice for Facebook.

Every day, our mapping teams work to confirm OSM’s millions of community contributions for consistency and quality by excluding intentional and unintentional edits that are incompatible with our use cases. Daylight is the result of that process.

Daylight Map Distribution

With this third release, we’ve continued our collaboration with the Microsoft Building Footprints project on an extension to Daylight. Users of Daylight can continue to apply missing building footprints in the United States, Canada, Uganda, and Tanzania to render more complete-looking maps:

Visual comparison between Daylight v0.3 and Daylight with Buildings

Larry Freil at Microsoft has posted a detailed exploration of this data, including suggestions for how to apply buildings data to Daylight for rendering OpenStreetMap with Mapnik. Microsoft provides the building footprint data under the Open Data Commons Open Database License (ODbL).

Throughout the remainder of this year, we will continue to publish releases of Daylight and conflated building footprints files from Microsoft approximately each month. Additionally, we will release details of the OSM tags we consider for inclusion in Daylight after converting them to a format suitable for use with existing community data tools.

  • Daylight planet files are composed of 100% OSM data, released under the terms of the Open Database License
  • We’ve checked public OSM changes contained in the distribution, and allowed only those which have been evaluated through validation algorithms and manual processes against commonly identified use-cases to provide the best mapping experience to our display maps
  • We publish the data in PBF and OSC formats, common file formats universally supported by OSM tools and already used for planet-scale file distribution

How To Reach The Team

If you have any questions about this data distribution, we have created a #daylightdistro_feedback Slack channel in OSM US. Members of the team will be there periodically to answer questions. You can also email the team at osm@fb.com.

Learn more about the technology behind our process from our engineering team:

Download Daylight Map Distribution files in OSM-compatible formats:

If you’ve benefited from OpenStreetMap data in your work, consider joining the OSM Foundation as a voting member to help steer the future of this global open data project! Time is running out to join 90+ days before the expected OSMF December Annual General Meeting as an eligible voting member.

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Announcing an Update to Daylight Map Distribution

Posted by migurski on 16 July 2020 in English (English).

Facebook is releasing an update to Daylight, our complete, downloadable preview of OpenStreetMap data.

📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.2.osm.pbf (56GB).

Earlier this year, Facebook released Daylight as part of our work with OpenStreetMap. We use maps to let our users find friends, businesses, groups and much more. OpenStreetMap (OSM), the open source wiki map, has a substantial global footprint of map data built and maintained by a dedicated community of global mappers making OSM a natural choice for Facebook.

Every day, our mapping teams work to confirm OSM’s millions of community contributions for consistency and quality by excluding intentional and unintentional edits that are incompatible with our use cases. Daylight is the result of that process.

Daylight Map Distribution

With this second release, we’ve also collaborated with the Microsoft Building Footprints project on an extension to Daylight. This extension includes buildings detected from aerial imagery in the United States, Canada, Uganda, and Tanzania. These building datasets were conflated with this Daylight Map Distribution to only add missing footprints, and provided as a changeset to Daylight in OSC format. Users of Daylight can now easily expand coverage to include additional AI-generated building footprints not found in OSM or mapped via the RapiD editor. Microsoft provides the building footprint data under the Open Data Commons Open Database License (ODbL).

📥 Download Microsoft ML Buildings right now in OpenStreetMap OSC format: 📦 ms-ml-buildings-v0.2.osc.bz2 (9.9GB).

After our initial announcement of Daylight, the community asked about future plans. With this second release, we plan to establish a regular cadence of releases tracking OSM changes into the future, to ensure that upstream changes to OSM are available with minimal delay.

  • Daylight planet files are composed of 100% OSM data, released under the terms of the Open Database License
  • We’ve checked public OSM changes contained in the distribution, and allowed only those which have been evaluated through validation algorithms and manual processes against commonly identified use-cases to provide the best mapping experience to our display maps
  • We publish the data in PBF and OSC formats, common file formats universally supported by OSM tools and already used for planet-scale file distribution

How To Reach The Team

If you have any questions about this data distribution, we have created a #daylightdistro_feedback Slack channel in OSM US. Members of the team will be there periodically to answer questions. You can also email the team at osm@fb.com.

Learn more about the technology behind our process from our engineering team:

Download Daylight Map Distribution files in OSM-compatible formats:

If you’ve benefited from OpenStreetMap data in your work, consider joining the OSM Foundation as a voting member to help steer the future of this global open data project!

Location: Downtown, Oakland, Alameda County, California, 94612-2202, United States

Announcing Daylight Map Distribution

Posted by migurski on 10 March 2020 in English (English).

Facebook is releasing a complete, downloadable preview of OpenStreetMap data we plan to start using in a number of our public maps.

📥 Download Daylight Map Distribution right now in OpenStreetMap PBF format: 📦 planet-v0.1.osm.pbf (42GB).

At Facebook, we use maps to let our users find friends, businesses, groups and much more. OpenStreetMap (OSM), the open source wiki map, has a substantial global footprint of map data built and maintained by a dedicated community of global mappers making OSM a natural choice for Facebook.

Every day, OSM receives millions of contributions from the community. Some of these contributions may have intentional and unintentional edits that are incompatible with our use cases. Our mapping teams work to scrub these contributions for consistency and quality. In the course of this work, we also build additional tools and technologies on top of OSM.

OSM is a complex data product. Many tools, services, and companies have been created to make it full-featured. We’ve always developed our OSM-related tools with the hope that our approach to keeping maps current and accurate for our own use cases may also benefit others in the OSM community. To that end, we’re pleased to announce the release of the Daylight Map Distribution, one of our internal OSM datasets scrubbed to meet the quality standards of our wide-ranging products.

Daylight Map Distribution

What’s Included in the Daylight Map Distribution:

  • This planet file is composed of 100% OSM data, released under the terms of the Open Database License
  • We’ve checked public OSM changes contained in the distribution, and allowed only those which have been validated to contain no malicious vandalism to prevent it from being shown to our users in our display maps
  • This is currently a one-time release, and we’re looking for feedback from the community to decide on a useful cadence for future releases
  • We publish the data in PBF format, a common binary format universally supported by OSM tools and already used for planet-scale file distribution

How we use OSM

We use maps made with OSM across Facebook to show people, events, and places:

map examples

Intentional, and possibly malicious edits range from headline-grabbing hate speech to outdated names. Some examples include:

bad edit examples

Other bad edits include geometry errors and small instances of grafitti. Here are some samples of minor and unintentional bad edits:

bad edit examples

bad edit examples

OSM forms a critical part of how our users interact with the world around them and our hope is that this release will make it easier for others to benefit from our work ensuring that it’s appropriate for display and free of vandalism. Through our use of OSM, we’ve encountered a variety of issues and inconsistencies and we’ve included fixes in our release of the Daylight Distribution. We also contribute these fixes back to OSM for the benefit of the larger community.

Working in Open Source

Our approach toward creating the Daylight Map Distribution was inspired by the success of the Linux operating system: starting with a pair of experts-only floppy disks in early 1991, user demand along with a liberal software license led to an explosion of ”distros,” curated collections of software that could be readily installed by casual users. The first Linux distribution was created less than a year later at the Manchester Computing Centre in February 1992. Today there are hundreds of distros including major products like Red Hat and Ubuntu. Distributors optimize for different uses making it easy and safe to use Linux on servers, laptops, phones, tablets, hardware hacking platforms, virtual machines, distributed systems, and embedded devices.

Consistent with the spirit of OSM, it is our hope that the Daylight Map Distribution (and subsequent iterations) will inspire individuals and companies to release their own datasets under open data licenses as well.

With the Daylight Map Distribution, we also hope to showcase all that is possible with a stable, efficient community-drive mapping effort. Open source is by its nature inclusive and welcoming to all. No contribution is too small or too large and we’re proud to stand together with every OSM contributor as we work toward shared goals of improving OSM and mapping the world.

How To Reach The Team

If you have any questions about this data distribution, we have created a #daylightdistro_feedback Slack channel in OSM US. Members of the team will be there periodically to answer questions. You can also email the team at osm@fb.com.

Learn more about the technology behind our process from our engineering team:

This release is just a sneak peek preview. We plan to start using this version of the data in our public maps soon, but you can start using it today. Download Daylight Map Distribution right now in OpenStreetMap PBF format:

If you’re interested in engineering and other roles working on OpenStreetMap at Facebook, get in touch!

Location: Belle Haven, Menlo Park, San Mateo County, California, 94025-1246, United States

Initial Results When Testing OSM Chef With CI Tools

Posted by migurski on 19 January 2020 in English (English).

I’ve been trying some light contributions to OSM’s Chef repository. In the OpenAddresses project we learned early that reliable and responsive continuous testing and integration make it easier for contributors to approach our project, and I’m hoping to build similar tests for OSM Chef. We already do a basic syntax lint, but these new tests would run each complete cookbook on a clean disposable host and notify Github of the results:

kitchen test --parallel --destroy=always all && notify-passed || notify-failed

Contributors would see an additional green check-mark in their pull requests, and OSM admins would be able to accept contributions confident that they’ve been fully tested.

Why Mess With Chef?

I’ve been in a long conversation with Andy Allan about small ways to help with OSM’s operational infrastructure. He nudged me in the direction of OSM’s Chef configuration, which shouldn’t be a surprise: Chef is how OSM manages the configuration of all the servers run by the OpenStreetMap Foundation’s Operations Working Group. Contributions to Chef are specifically cited in Andy’s Getting Involved post and mentioned in policies for both the Operations Working Group (OWG) and the Sysadmins group.

Andy recommended that I pay special attention to the Wiki cookbook: it’s the system that has the most outside interest from non-sysadmins over the last three years. For people who would like to change the configuration of wiki.openstreetmap.org a working cookbook would make it easier to test locally with test-kitchen and offer contributions that are known to work prior to deployment. Today, “we only find out if the changes actually work when we run them on the live servers.”

This PR is my initial pass at fixing some bugs in Chef, and there’s feedback there from Andy, Tom Hughes, and Grant Slater about OSM’s Chef expectations. My hope is to see this ultimately become functional, trusted, and automated enough that OWG repo admins are comfortable defaulting to “Yes” and accepting any change with community support that passes tests without technical debate. Chef is not a widely-understood technology and I think a lot of the casual DevOps world has moved on to container-based approaches, so it’s critical for OSM’s Chef to work in an automated and well-understood way to welcome new contributors.

Getting To Headless

Contributions to open projects are encouraged by providing a smooth entry path and headless continuous integration is an excellent way to make this happen. Github provides strong support for automated status updates that we’ve used for the OpenAddresses project throughout the past four years. We use it to automated feedback to users and generate screenshots of their contributions so they know they’re doing the right thing and we can safely rubber-stamp their input:

Screenshot showing Github user experience in OpenAddresses

For OSM Chef, we could run all cookbooks under test-kitchen. I’ve tested the PR above in a few environments that might potentially be used as part of an automated flow: on my local Mac where a developer might check their work prior to publishing, under the current long-term supported Ubuntu 18.04 that might be part of an AWS EC2 setup, and under the previous Ubuntu 16.04 that might be part of a Github Actions setup.

Results So Far

Host OS Test Driver Result
Mac OS Vagrant 🔶 cookbooks/mediawiki/resources/site.rb line 528: Column 'cuc_user' cannot be null
Ubuntu 16.04 Vagrant 🚫 Timed out while waiting for the machine to boot
Ubuntu 18.04 Vagrant 🚫 Timed out while waiting for the machine to boot
Ubuntu 16.04 Docker 🔶 cookbooks/mysql/recipes/default.rb line 23: No such file or directory - /sbin/status
Ubuntu 18.04 Docker 🔶 cookbooks/mysql/recipes/default.rb line 23: No such file or directory - /sbin/status
Mac OS Docker 🔶 cookbooks/mysql/recipes/default.rb line 23: No such file or directory - /sbin/status

Vagrant under Mac OS gets furthest, then ultimately gets stuck on problems in the cookbook itself. This is potentially fixable in OSM/Chef by updating the Mediawiki and Wiki cookbooks.

There is no meaningful difference between Ubuntu 16.04 and 18.04. The older version needs Ruby 2.4+ to be installed explicitly, but both run test-kitchen with minimal fuss. Although OSM is deployed to Ubuntu 18.04, the host OS for these tests varies.

Here’s the approximate script I’m using to run the tests from the Ubuntu 18.04 host OS:

git clone https://github.com/migurski/chef.git osm-chef && cd osm-chef
sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install build-essential ruby ruby-dev docker.io vagrant virtualbox 
sudo gem install test-kitchen kitchen-docker kitchen-vagrant
kitchen test --destroy=always wiki-ubuntu-1804

Vagrant on Ubuntu fails immediately. Googling for error strings does not turn up any obvious mistakes. The guest OS running under Virtualbox is unresponsive to input from Vagrant. I’m not experienced with either Vagrant or Virtualbox on Linux, so I’m unsure how to dig up problems here.

Docker on Mac OS and Ubuntu gets further and looks more promising, but bumps into a very different problem. Most of the cookbook runs happily until the MySQL cookbook tries to start the database and immediately fails to find /sbin/status, a script that no longer exists under systemd. The guest OS under Docker is identical to that under Vagrant, so I am confused and stumped by this error.

Next Steps

  • OSM operations group prefers to Vagrant to Docker for test configuration, but it fails inscrutably. Figure out why Vagrant guest OS’s under Ubuntu are unreachable.
  • Docker guest OS gets quite a bit further but runs into the /sbin/status issues. Figure out why this happens and advocate for switching from Vagrant to Docker in OSM test-kitchen configuration.
  • Return to OSM/Chef repository and get the Wiki cookbook working all the way through to the end.

I’m Running for OSMF Board

Posted by migurski on 5 December 2019 in English (English).

I’ve made OpenStreetMap a major part of my life and work since 2005. I love the sparkling audacity of a complete, freely-licensed, street-scale map of the world built from the ground up by a community motivated to share work for collective benefit. Living in the United States I know firsthand the benefits of high-quality free public data like we get from the US Geological Survey and Census, and OSM promises to extend this benefit to the whole planet.

OSM for me has always been a story of global impact through work: at Stamen Design I led our participation in open mapping with a mix of early San Francisco mapping parties, paid clients, and experimental projects. Some, like Field Papers (presented as “Walking Papers” at SOTM 2009 in Amsterdam) helped OSM become a force in humanitarian mapping and have since matured into useful infrastructure. I’ve helped OSM US since its first conference in Atlanta in 2010 and served a term on the US Foundation board in 2012. In 2016, I coined the term “craft mapping” in a widely-read and commented-upon blog post to tell a story about the contrast between OSM’s hand-made history and its potential utility future. OSM’s greatest strengths are its worldwide reach, open contribution model, and free license. I have recently grown so excited by the large-scale use and expansion of OSM that I joined the Facebook mapping team last year to contribute to our OSM efforts.

Read my complete manifesto on the Wiki for more about why I think I’d make a good OSMF board member.

Participating in the OSM US ED hiring process

Posted by migurski on 6 January 2018 in English (English).

OpenStreetMap US is hiring an Executive Director, and they are looking to build a hiring committee:

We are thus inviting community members (YOU) to self-nominate if you are interested in participating in the hiring process. … If you are interested in nominating yourself, expressions of interest will be accepted by email (to hiring@openstreetmap.us) until January 12th, 2018.

I’m interested, so I sent the email below.


Hello,

I would like to participate in the Executive Director hiring process as part of the 7-person committee! I’m pretty sure I know a few of the people who might be reading this email, but I thought I’d make it a more formal one to underscore my interest in helping.

I enthusiastically support the idea of finding a leader for OSM US at this time. The US Foundation has consistently been on the cutting edge of international OSM work, from hosting the most consistently well-produced regional conferences to pioneering advances in OSM’s use for disaster response and international development. OpenStreetMap needs a way to communicate with the many organizations whose mission or business depends on it, and hiring a full-time ED will help this happen.

OSM as a whole is experiencing rapid change, and faces both risks and opportunities I wrote about in this widely-shared blog post last year: http://mike.teczno.com/notes/openstreetmap-at-a-crossroads.html

As a long-time participant in open source and non-profit technology efforts, I bring experience and perspective that will help ensure a good process and result. I’m on the boards of mapping non-profits GreenInfo Network and Digital Democracy, I served as CTO of government technology non-profit Code For America for three years, and I’m currently Executive Director for a new non-profit political mapping project, PlanScore.org.

Finally, as a long-term participant in OpenStreetMap I am deeply familiar with the project background. I’ve mapped since 2006 and attended almost every SOTMUS conference since 2010 in Atlanta. I’ve contributed to OSM in a variety of ways focused on the approachability and usability of the project. I’m motivated to see OSM grow into a mature, healthy, successful organization and I’m excited to see the US Foundation take the important step of finding its first full-time ED to make this happen.

-mike.

Location: Longfellow, Oakland, Alameda County, California, 94609, United States