OpenStreetMap

Diary Entries in English

Recent diary entries

Preparing accurate history and caching changesets

Posted by geohacker on 12 April 2017 in English (English)

It's important to see what exactly happened to features in a changeset. This means identifying the state of each feature, the history, including geometry and tags that changed. The OSM changeset page doesn't give you a clear idea of what happened in a changeset - you see a list of features that changed, and the bounding box of the changeset.

image

The changeset XML from OpenStreetMap only has current version of the features that changed in the changeset.

Overpass offers augmented diffs between two timestamps that contains current and previous versions of each feature that changed in that period. We put together an infrastructure that queries Overpass minutely, prepares changeset representation as a JSON, and stashes them on S3. The augmented diffs are also cached on S3. This means that the load to Overpass instance can reduce drastically while many of us are looking at the same changeset.

image

This is directly used in changeset-map - a utility to visualise OSM changesets.

JSON

The cached changeset JSONs are available here: https://s3.amazonaws.com/mapbox/real-changesets/production/changeset-id.json. The JSON looks like this for a changeset by user Rezhin Ali.

This is inspired by the work Development Seed did with Planet Stream. We use osm-adiff-parser to convert the augmented diff to changeset JSON.

// 20170411184718
// https://s3.amazonaws.com/mapbox/real-changesets/production/47656996.json

{
  "elements": [
    {
      "id": "4787752634",
      "lat": "36.1823442",
      "lon": "44.0158941",
      "version": "2",
      "timestamp": "2017-04-11T13:12:35Z",
      "changeset": "47656996",
      "uid": "5323129",
      "user": "Rezhin Ali",
      "old": {
        "id": "4787752634",
        "lat": "36.1823442",
        "lon": "44.0158941",
        "version": "1",
        "timestamp": "2017-04-11T08:02:21Z",
        "changeset": "47649032",
        "uid": "5323129",
        "user": "Rezhin Ali",
        "action": "modify",
        "type": "node",
        "tags": {
          "name": "ێەبد مەنان",
          "name:ar": "ێەبد مەنان",
          "shop": "car"
        }
      },
      "action": "modify",
      "type": "node",
      "tags": {
        "name": "Abd Manan",
        "name:ar": "Abd Manan",
        "shop": "car"
      }
    }
  ],
  "metadata": {
    "id": "47656996",
    "created_at": "2017-04-11T13:12:34Z",
    "open": "true",
    "user": "Rezhin Ali",
    "uid": "5323129",
    "min_lat": "36.1823442",
    "min_lon": "44.0158941",
    "max_lat": "36.1823442",
    "max_lon": "44.0158941",
    "comments_count": "0",
    "tag": [
      {
        "k": "created_by",
        "v": "MAPS.ME ios 7.2.3"
      },
      {
        "k": "comment",
        "v": "Updated a car shop"
      },
      {
        "k": "bundle_id",
        "v": "com.mapswithme.full"
      }
    ]
  }
}

Empty changesets

It's possible that certain changesets are empty. They could have been opened, but failed to upload changes due to unreliable network, and eventually gets closed in 60 minutes. Empty changesets are not cached.

Long changesets

Changesets can also remain open for a long time. For example this one from user Manuchehr was opened 36 mins. Experienced users like to survey outdoors, and upload data in bulk. Some editors also don't close changesets automatically. Idle changesets get closed eventually after 60 mins.

When features of changeset comes through in a later minutely diff, we update the cache on S3. This will ensure, changeset remain complete.

Database transactions and augmented diffs

A changeset being closed doesn't mean that all features that changed have been committed to the OSM database, and appear in the minutely diff right after. Some features may take longer to commit to the database, we handle these by updating the augmented diff from S3, and then recreating the changeset JSON. You can read more about this case here.

Missing changesets

Changesets that are after March 1, 2017 are cached. We are considering doing a slow backfill, but this is entirely dependent on Overpass. If you see something missing, or unclear, please open a ticket and let us know!

Location: Indiranagar 1st Stage, Indiranagar, Bengaluru, Bangalore Urban, Karnataka, 560001, India

10 years of OpenStreetMap in Tyumen Oblast (Russia)

Posted by mavl on 12 April 2017 in English (English)

Tyumen Oblast is small region in Western Siberia. It has total area about 160 000 square kilometres and population about 1 340 000 (308 persons (or 0.02%) are OSM contributors).

Years and years ago first human settlement was marked on OpenStreetMap in Tyumen Oblast. It was Tyumen (was added by contributor LA2). This event happened in 2007.

Many local contributors improved OpenStreetMap after it.

New local contributors

Year is an year when an account was created.

Probably, these contributors live in Tyumen Oblast because they usually edit in this area. Of course, some contributors of other regions improve OpenStreetMap in this area also.

Since 2016 there are local maps.me contributors.

New local maps.me contributors

Maps.me contributor is a contributor who created first changeset/note using maps.me program. One contributor created his account in 2011 but he created his first changeset in 2016.

Many local contributors (64%) usually edit Tyumen's map. Tyumen is a regional capital with population about 720 000. Tyumen city's population amounts to about 54% of Tyumen Oblast's population.

Local contributors

Tyumen Oblast has 2 cities, 3 towns and about 1200 villages. Now OpenStreetMap contains all these human settlements as nodes (or in more details). It's a good result.

Congratulations.

Location: Комарово, Калининский административный округ, Tyumen, городской округ Тюмень, Tyumen Oblast, Ural Federal District, 625000, Russian Federation

Cayo District

Posted by Charlie_Plett on 12 April 2017 in English (English)

2017-03-20 I went selling in Cayo for Country Foods and added dozens of shops to Cayo, Benque Viejo and Bella Vista(Toledo).

Location: Morotown, San Ignacio, Cayo District, Cayo, Belize

Some gentle support for future HOT board members

Posted by dekstop on 11 April 2017 in English (English)

Another round of HOT board elections is about to close, and for the first time I'm participating as a voting member. As I write this I don't yet know the results, we will review them at the member AGM tomorrow. An exciting moment! The community discussions around this also made me aware that these election cycles are always an opportunity for a new generation of HOT members to become our representatives. From personal experience I know that this can be a daunting transition, so I invite all candidates to lean on your community for support: we believe in you, and we can offer you advice and support, if desired. (Chances are you're already very knowledgeable and experienced.)

Such a moment might feel particularly daunting if you're not used to being in such a prominent position within a large public organisation. This is likely true for most humans! Possibly with some exceptions -- as a white male I practically get status thrown at me, and I mainly just needed to learn how to accept it with grace. But people's experiences differ. Maybe you were taught modesty as an important virtue, and to not be too assertive in your interactions. Through many conversations over the years I have learned that such small differences can affect our respective self-image, regardless of our actual competencies; and they may inform how we approach the prospect of becoming a board member.

I'm writing this post in anticipation that we may see some new faces on the board, if not this round then later. I'm writing to share the things I've been taught to take for granted; and I think you should take them for granted too. (This is not a universal set of recommendations. Many people won't be able to relate to this, or only in parts. That's fine. You will know if this speaks to you.)

First of all, I believe in your achievements, and I will call you an expert without thinking twice about it.

If you're not the brazen kind then I suggest you practice how you can introduce herself in professional settings: hallo I'm X, I'm a board member of HOT. Hallo, I'm on the board of a large international volunteer org. Etc. Learn to anticipate what kind of greeting may resonate best according to the setting, and never be too shy to state your full title. You're not bragging, you're providing important context: you're now a representative.

Whenever in doubt, know where to seek advice. Form relationships with your fellow board members. You're always welcome to email or even call your peers and close contacts, anytime. Because you're now in an exceptional situation of responsibility, we believe you deserve exceptional support.

Most importantly, have confidence in your expertise, and listen to your instincts. You're here because of your achievements and connections, but also your specific sensibilities. Speak up when everyone agrees, but something feels funny to you. Never be afraid to ask a simple question; and never be afraid to ask a hard one either.

Let your servant nature work for you, not against you, if you think that you have one (I know that I do.) Remind yourself that it is your duty to act on your instincts. Your community has entrusted you with this responsibility because they consider you a worthy representative, and you can act with the full weight of their support.

Do you need to consider yourself a leader to do this job well? It's up to you. I'm personally a fan of servant leadership: nobody's boss, and everybody's assistant. However, sometimes you will need to be firm in order to get the best outcome for your community.

(We could also chat about many practicalities: keeping notes, balancing commitments, relationships with peers, boundaries, burnout, managing your ego, managing conflicts of interest, etc; maybe something to discuss in the comments?)

Mapeamento Colaborativo de Jacaraípe - Meu lugar através de fotos

Posted by Jordana Newton on 11 April 2017 in English (English)

A foto foi tirada 17 horas da tarde do dia 13/03/2017, por do Sol de um dia ensolarado. Foi em um local de Jacaraípe desconhecido, no qual a rua possui muitas árvores e sem asfalto além de ser tranquila e serena.... Lagoa de Jacaraípe, Serra, Espirito Santo, Brasil.

Location: Lagoa de Jacaraípe, Região de Jacaraípe e Manguinhos, Serra, Microrregião Vitória, Greater Vitória, Mesorregião Central Espírito-Santense, Espírito Santo, Southeast Region, Brazil

Hi

Posted by Thyeutuan on 10 April 2017 in English (English)

Thy

Location: Sherman Hill, Des Moines, Polk County, Iowa, 50319, United States of America

Possibly importing USGS forest data

Posted by ff5722 on 10 April 2017 in English (English)

USGS has published tree cover data based on 2010 Landsat captures. I wonder if this data would be suitable for importing. Especially outside of Europe, forest cover is largely incomplete now, and 2010 is fairly recent for this kind of data.

https://landcover.usgs.gov/glc/TreeCoverDescriptionAndDownloads.php

In the licence requirement it says: University of Maryland, Department of Geographical Sciences and USGS; use is free to all if acknowledgement is made. So it is not obvious if using this data is allowed.

The data is provided asgreyscale geoTIFFs, i have uploaded one tile as a preview here:

https://api.mapbox.com/styles/v1/hindbaer/cj19m696v009v2ro4rwlq18k3.html?title=true&access_token=pk.eyJ1IjoiaGluZGJhZXIiLCJhIjoiY2lqOGt4bG1wMDAweXR0a25vcGx4ZmUzNCJ9.R55CveYJBChT-Olntza9Mg#7.21/23.611/121.796

Is the licence ok to use for adding data to OSM? If not, we could seek explicit permission from USGS, this data was directly derived from Landsat anyway, so they may be able to allow less strict attribution requirements.

Should the licence be suitable, then of course there will be many issues. For starters, the data is about 'tree cover' which could be 'landuse=forest', 'natural=wood', 'landuse=orchard', etc. Then there is the difference between OSM, where an area can only be forest or not forest, and this 1-100 scaled data. At which threshold is it a wooded area?

Welcoming new mappers around your mapping patch

Posted by manings on 10 April 2017 in English (English)

OSMCha has this great feature of watching edits around your neighborhood using the bounding box (bbox) filter. Today, I reviewed several edits using the combination of bbox and the new mapper flag.

screen shot 2017-04-10 at 16 07 54 New mappers in the Philippines in the last couple of days

Note: Being a new mapper does not mean making bad edits per se, but by being new, we can commit errors because of unfamiliarity with the best practices of editing. Admit it old mappers, we made mistakes before. ;)

As I review each changeset, I often fix common errors based on my familiarity with the area. After fixing, I usually add a comment to the user's changeset explaining my edits and encouraging them to continue contributing to the project.

screen shot 2017-04-10 at 16 16 46 My changeset comments

While re-reading my comments, I realized I'm doing something wrong! My comments were peppered with OSM technical jargon new mappers may not even know of. I'm imagining this new user's reaction 🤔 to my comment.

  • What is a changeset?
  • What do you mean by tags? Amenity, shop, building; what's the difference?
  • How can I go back to my edits and correct them?

Next time, I'll try to simplify my comments and avoid hard to understand terminologies. It is challenging to simplify without losing the substance of the comment, but, it is equally important to engage new users as first step in engaging in a conversation.

How do you welcome new users in your neighborhood?

OSMF Selling Data to Google?

Posted by Omnific on 9 April 2017 in English (English)

It's odd that this hasn't generated any discussion: http://www.weeklyosm.eu/leaked-news-big-osmf-deal

If OSMF is truly planning to implement such a deal wherein Google gets to copy useful data in exchange for funds, I would certainly stop contributing. The price tag is almost certainly a pittance (a few million per year, probably) compared to the value of all that local data, and given that Google spends billions on map data per year. This is counter to the entire reason I contribute to OSM. If this goes through, I might as well contribute directly to Google's commercial maps. And the stipend idea is hilarious and almost an insult, as it will likely be pennies per hour.

Thoughts?

waterway=dam + beaver_made=yes

Posted by michalfabik on 9 April 2017 in English (English)

During a walk in a riparian forest, I happened upon a beaver dam:
beaver dam

and some construction works in progress:

beaver-gnawed tree

This got me thinking if it makes sense to map physical features created by animals. We've got the man_made tag, which some consider sexist and suggest that it be changed to human_made. Now what if I consider "human_made" too anthropocentric? Let's use the more general animal_made=* + animal=human (or animal=beaver in this case).

OK, jokes aside. There are animal-made physical features which are reasonably permanent and prominent enough to serve for orientation. Beaver dams, stork nests, anthills and termite mounds come to mind. I'm sure there are more but I'm no biologist, maybe someone else will come with other examples.

Oh and BTW, how do you tag a riparian forest? It's a forest with soaked ground and smelly pools of water here and there but nowhere near a swamp like in the film Southern Comfort (wetland=swamp).

Location: Bulhary, okres Břeclav, Jihomoravský kraj, Southeast, Czechia

Parahat 7/4 and 7/5

Posted by apm-wa on 9 April 2017 in English (English)

April 8 I drove around Parahat 7/4 and Parahat 7/5 collecting GPS traces to add the new residential streets and alleys that are now paved. Parts of these etraps are occupied already but many buildings remain under construction, and not all streets are open yet.

Location: Parahat 7/5, Ashgabat, Ahal Region, Turkmenistan

lat/lng: (21.97925,96.11786)

Posted by THEIN SOE COMPUTER on 8 April 2017 in English (English)

lat/lng: (21.97925,96.11786) Latitude 21.97925 Longitude 96.11786

Location: 58th Street, Chanayethazan, Mandalay, 0095, Myanmar

Correcting Speed Limits in Toledo, Ohio

Posted by BWToledo on 7 April 2017 in English (English)

I'm a taxi driver in Toledo, Ohio and I'm updating speed limits as I notice errors.

Manitou Springs updates

Posted by grayforgestudio on 7 April 2017 in English (English)

I've been working on updating businesses along Manitou Ave and the trails around Manitou. The Intemann trail, in particular, is incorrect. I've hiked most of it and will try to update from my tracks and the latest official posted maps (web and physical signs - they're the same).

The state of San Francisco buildings

Posted by Chetan_Gowda on 7 April 2017 in English (English)

In 2016, San Francisco city Government published LiDAR-derived building footprints that include both geometry and height information. One of the main goals of the San Francisco OpenStreetMap community was to import this height data to the existing buildings in OpenStreetMap. The project was initiated by SF community in November 2016, recently Brandon and Daniel lead a great Mapathon at Mapbox SF office.

sf_illustration

Together with the community, Mapbox data team remotely joined and imported nearly 140,000 building heights to OpenStreetMap. Now we can get a real world like experience using popular 3D renderers like OSM Buildings and Mapbox GL JS or create realistic city using Blender or style buildings using Unity. Cheers to the all of them who made this project successful!

screen shot 2017-04-06 at 10 47 27 am

We will be validating these buildings through street level imageries. Both Mapillary and OpenStreetCam have great coverage in San Francisco City.

Tasking manager:

http://tasks.openstreetmap.us/project/71

OSM/Mapillary comparison tool:

Right now we are using comparison tool to validate these buildings heights. Here are the detailed instructions to validate these buildings.

Please get in touch with the community in the Gitter channel for validation and for other updates.

Reviewing recent changes in your area with OSMCha

Posted by manoharuss on 7 April 2017 in English (English)

It is always interesting to review the latest map edits in your local area. It can be fun to send a welcome message to a new contributor or track changes to a neighbourhood that a fellow mapper has surveyed. It's an opportunity to both learn from an experienced contributor as well as teach someone new a helpful tip to make mapping more engaging. In a more rare case, these tools can help investigate some missing data or suspicous mapping activity.

The OSM Changeset Analyzer (OSMCHA) tool by Wille Marcel has a powerful interface to filter changesets in your area and flag any suspiscous changesets for further action.

Here's how you can start using OSMCha in your area:

Step 1 : Understanding and using Filters in OSMCha

OSMCha

image

Currently available filters in OSMCha

Introduction to filters

  • Date fields

    • These date fields are based on changeset closed time as on OSM. We can use date range to narrow down changesets (yesterday, last week)
  • Creations, Modifications and Deletions

    • This is a simple count of type of edits in a changeset
  • A changeset is suspicious when the changeset is flagged by one of the reasons seen below. These are compare functions that flag a certain type of edits. I will go into more details about how compare functions work in OSMCha and how we can use them to flag specific edits. This has to be an another diary post.

  • White-list - When you login into OSMCha using your OSM credentials, OSMCha creates a very basic profile that consists a list of changesets from OSM users you wish to not see in your search. This is a personal custom list for each OSMCha user/reviewer. You can add a user to the white-list when you are in that user's changeset.

  • BBOX - This filter allows to easily give an area of interest we would like to validate. In this case, you can zoom into your particular neighborhood and OSMCha retrieves only changesets whose bbox falls on the area you have given.

These filters give a lot of freedom and flexibility to narrow specific type of changesets we would like to retrieve from a specific area. In OSM one can already use the History tab to see the changesets that overlap with the area on the map but OSMCha adds a lot of filters to this idea to assist the reviewer.

Step 2: Validating your neighborhood

bbox2

Using bbox filter to select area of interest

When we click the search button, OSMCha presents us with a list of changesets that it thinks fit into our search criteria. An embed version of changeset-map shows geometric and feature tags of edits on the map.

Along with the contextual information where the edits are on the map, geometric edits to the feature, you can also click on the features in the changeset map touched by that changeset to see previous version feature tags and current version feature tags.

Did you try using OSMCha? What worked, and what did you not like? Let us know in the comments below.

Feedback

If you come across a bug you would like to give feedback on, use OSMCha-django repository to file an issue. An issue with overlap of global changesets when using bbox filter in OSMCha has been already raised in my previous diary post. We are working on to best resolve this issue.

If you would like improve detection of a particular type of feature edit, check out our OSM-compare repository and open an issue.

New MapRoulette challenges: Add missing Wikidata ids to cities and capitals

Posted by PlaneMad on 7 April 2017 in English (English)

Just created a couple of mapping challenges for the Wikidata enthusiasts:

There are over 8,000 important places on the map with neither a Wikipedia or Wikidata tag. These challenges are a great way for mappers interested in Wikidata as well as Wikidata contributors to get started with OSM. Do spread the word in your local communities and share your feedback on how to improve the challenge.

Clickable roads demo

Posted by mmd on 7 April 2017 in English (English)

I put up a small proof of concept to demonstrate @jotpe's proposal on Github:

Link to Clickable Roads Demo

Just click on some street with a name and see all parts of the street highlighted, regardless of how fragmented the underlying OSM data is. Small gaps are also ok.

image

Caveat: a bit of patience may be needed, in particular during London night time. As dev instance doesn't run on fast and shiny SSDs, other heavy load jobs have quite some impact. The night window starts at about 01:00 (UTC+0) and finishes at about 08:00 (UTC+0).

Clever use of OSM in marketing brochure

Posted by Alan Bragg on 6 April 2017 in English (English)

I recently received a brochure from my on-line bank showing the location of their no-fee ATMs near my home. The attribution in the bottom left corner reads "LOCR/OpenStreetMap"

I found more information at Locr selects OSM

I was impressed.

Location: The Great Road, Bedford, Middlesex County, Massachusetts, 01730, United States of America

My Hisory

Posted by Emi Rizki Ayunanda on 6 April 2017 in English (English)

My name is Emi Rizki Ayunanda

Older Entries | Newer Entries