OpenStreetMap

PierZen's Diary Comments

Diary Comments added by PierZen

Post When Comment
Testing planet import about 2 months ago

Hi mmd, for those that are not familiar with these programming tools, could you describe what type of import is done exactly with Planet-OSM - moving into Overpass db or anything else ?

Opening up HOT Governance (help wanted) 9 months ago

Hi Heather

For people that are not familiar with the HOT organisation, could you clarify Who is the we that want to shift the governance model at HOT? - The membership - The Board - The Governance working group

Cleaning up large, untagged ways 9 months ago

@richlv to test if less careful changes to forests and other areas, a first step would be to search OSM history using overpass with the [date: YYYY-MM-DDT00:00:00Z’] parameter to assure the way was not previously a relation member.

Mapping leads to the end of winter 10 months ago

Upper north on the Richelieu river near Montréal, the canadian and white geese should halt in march-april coming from areas like Chesapeake Bay and similar areas south of USA. In october-november they halt again coming from up north. They also make these nice V-shaped formation!

Bonne cartographie.

Lot of frustration due to bad H.O.T. tasks 11 months ago

Hi @pedrito1414,

What we should try to do is find ways to better document the process and provide infos for better monitoring.

The OSM Togo contributor changesets history around 4 years ago shows that he used the same changeset in various countries including South Africa https://www.openstreetmap.org/changeset/56700180 , USA https://www.openstreetmap.org/changeset/56701932

To better document, I think that it is better to add other tags to the changeset, letting the contributor describes is contribution in the changeset command.

The call to osm editors from the Tasking manager and similar projects should simply add distinct tags to better reference the process. Adding host=url link Project + tags for task and action (mapping or validation) would facilitate monitoring projects from the Changesets metadata.

Lot of frustration due to bad H.O.T. tasks 11 months ago

@G1asshouse, these hashtags are inexact since they refer to a Togo project. A contributor might have move from one TM project to the other and forgot to change his Changeset comments.

We also need to assure a better identification of Coordinated projects. It seems that iD now adds systematically in the changeset metadata the tag host with the url link to the project. This should be more systematic and also appear from edits in JOSM and other editors accessed from the Tasking manager. See https://www.openstreetmap.org/changeset/115157629 that contains

key value
created_by iD 2.20.2
hashtags #hotosm-project-11385;#mogadishu-swm
host https://tasks.hotosm.org/projects/11385/map/

In case of TM projects, we can interrogate the API to validate if changesets metadata will refer to the exact TM project. This is not the case actually since the Project creator manually insert the TM project id in the Comments. I spotted mentions with no TM id or wrong id. See recent examples in the list below.

TM projects with inexact Project reference

hot_TM Changeset comment
8570 #hotosm-project-OpenCities-Abidjan
9313 Gindabour to Kumfabiala, Ghana #hotosm-project-8836 #Leclara #sotmafrica#liei-larabanga #OSMGhana
9317 Bole, Ghana #hotosm-project-8837 #Leclara #liei-larabanga #OSMGhana
9896 #hotosm-project-10218 #OSMGeoWeek #MapLibya #LNSDI #MapSabha #LNDTI #missingmaps #Vegetation #LandUseLandCover #Buildings #Infrastructure #StreetsRoads #Water #Soil #Air
9897 #hotosm-project-10218 #OSMGeoWeek #MapLibya #LNSDI #MapTripoli #LNDTI #missingmaps #Vegetation #LandUseLandCover #Buildings #Infrastructure #StreetsRoads #Water #Soil #Air
10797 #hotosm-project-10753 #hotosm-project-10752 #LittleEarth #Tajikistan #CartONG #buildings #roads #rivers #MissingMaps #Kulolon
10798 #hotosm-project-10753 #hotosm-project-10752 #LittleEarth #Tajikistan #CartONG #buildings #roads #rivers #MissingMaps #Sangdara #Saydon
10890 #hotosm-project-10660 #LittleEarth #Tajikistan #CartONG #buildings #roads #rivers #missingmaps #Langar #Khayrobod
11880 #hotosm-project-11879 #OSM Madagascar #Université de l’Itasy #CoopDec RNA-RI
Lot of frustration due to bad H.O.T. tasks 11 months ago

@pedrito1414 The changeset related to way 1013314708 mentionned at the beginning of this diary. refers to TM-11385 that completed in the last few days.

Highway https://www.openstreetmap.org/way/124549760 deserves a mention since we see 10 highways reported as underground. This highway did already exist in the database before this project. A contributor traced buildings over it, then revised the highway tags, adding layer=-2.

See this Overpass query that loads both the highway and the overlapped buildings.

http://overpass-turbo.eu/?Q=(way(id%3A124549760)%3B%0Away(around%3A0)%5B%22building%22~%22.%22%5D%3B%0Anode(w)%3B)%3B%0Aout%20meta%3B%3E%3B%20out%20meta%3B%0A&C=2.03235;45.32651;20

Lot of frustration due to bad H.O.T. tasks 11 months ago

I should add about monitoring with Osmose, that there is delay reporting from Osmose. As we can see on the Age column of the summary report, items reported are from an observation 2 days ago. If a contributor corrected the problem since then and we try to edit, we will see that the problem is already fixed.

Lot of frustration due to bad H.O.T. tasks 11 months ago

hotosm project https://tasks.hotosm.org/projects/11385 is active in the area. Yes this is a dense area with probably only alleys not clearly visible between buildings. Such area should surely not be mapped by newbies, needs better imagery then usual and contacts with local contributors to validate, identify alleys and attribute the appropriate tags.

Below I provide some infos how to monitor the area using Osmose HTML-List. And we will see that the layer tag seems to have been used to correct reported overlaps between buildings and roads.

From the Osmose HTML-List page, it is possible to obtain summary statistics for the a bbox if we specify item=xxxx. With the hotosm project 11385 bbox, the url link is http://osmose.openstreetmap.fr/en/issues/open?item=xxxx&source=&class=&username=&bbox=45.279918,2.000613,45.330809,2.037097

It reports 564 Highway intersecting buildings 231 Building intersections 93 Large building intersections 10 Long Highway underground and no tunnel and yes a lot of very imprecise buildings tracings could be added to the list.

To list objects reported for specific items, we can specify the item in the link. For underground highways (item=4110): http://osmose.openstreetmap.fr/en/issues/open?item=4110&bbox=45.279918,2.000613,45.330809,2.037097

The (j) link let’s load a specific object into josm. See for example, highway=residential that should probably be tagged as highway=path without the tag layer=-2 !

##########HASHTAGS###### Syndrome? Catch them young! about 1 year ago

Hi all,

The absence of significant comments is part of a more general problem where we escalate in adding constantly new TM projects but do not closely manage them. We have inititors/promoters that create TM jobs and promote crowmapping but do not closely follow participants to TM projects. And most participants are newby with not sufficient training. It would be interesting to see if in-person mapathons let add more signficant hashtags and more data quality.

This TM structuration with Project managers / Validators is more in the description of projects but prove often to be quite inefficient. How can we explain that too often 100% validated projects contain so much bad quality buildings tracing ?

We need more accountabilty from Project managers, more monitoring tools to spot rapidly Incomplete metadata and bad quality data.

Solar power mapping in China: a comparison against ML data about 1 year ago

I downloaded the predicted file and looked at this area where we see a lot of structures with no spaces between each row of solar plants. https://www.openstreetmap.org/#map=18/40.010487/112.891686

From my own observation, Images available via Bing, ESRI and Maxar are not clear and detailed enough to assess if the structures are solar plants, greenhouse or nets over the plants.

As always, AI predictions are quite related to image quality. I wonder if the team had better images to interpret the structures observed.

OpenStreetMap Contributor LifeSpans - Revisiting and expanding on 2018 research paper about 1 year ago

Hi Jennings, in your diary gray boxes, I cannot see tables and graphs and no links to data.

About prior discussion, Note that there a lot more then the study you refer to. Other then academic publications, you should look at the OSM community where there have been many discussions and analysis about contributors lifespan and contributors profile.

Like Simon, I illustrated the Maps.me bump effect in 2016 and presented lifecycle plots. Look at my diary OSM Contributors Outlook - The Pulse of OpenStreetMap Contributors. The first graph shows survival curve of yearly cohorts for the first 12 months. For each yearly cohort, we first observe a dramatic decrease after the first month (often after the first day of contribution) and the constant decrease in the following months.

You will also find analysis of contributors vs contribution share for various OSM contributors profile. For example, graph2 shows a constant increase of «Discover» contributors (1-2 days of participation) with a minimal share of contributions (ie. objects edited).

My publications on slideshare about major OSM humanitarian responses also include various analysis of contributors profile where we see the problematic with mapathons of massive number of contributors that participate only once. Their weight in number of objects edited is a lot smaller. More difficult is to analyse the quality of their contribution and how efficient are teams in such mapathons to integrate newcomers and assure quality.

OpenStreetMap Isn't Unicode about 1 year ago

I used to have often such encoding problems with acccented characters in python 2. Gladly, a more systematic usage of UTF-8 by the various tools have fixed the problem.

For french language, encoding ( é for é) like the example given by mmd for https://osm.org/node/880591396 is one that we often see. This particular example concerns an import for Haiti 2010 for health centers. These were bad imports and no success since then to have an inventory of health centersé In such a context. I generally avoid to correct the syntax. I would prefer that field survey validate first if the center exist.

Searching characters is the easy part. Below is an overpass query with a regex expression to search for multipe accented characters. This is here a partial list of accented characters in french and we can see the result searching around Port-au-Prince, Haiti.

http://overpass-turbo.eu/s/1csJ

Running JOSM on X86 without Oracle's JAVA over 1 year ago

Hi John,

Could you document how easy to install ? If we already have jdk installed, is the download / installation simply replace it ?

Growing features and making OSMCha operationally excellent almost 2 years ago

Type of edit could be also identified such as imports, organized editing to facilitate monitor these activities.

Growing features and making OSMCha operationally excellent almost 2 years ago

Hi Wille,

It would be great if some backstore work was done to complete Changeset metadata from the Planet file, adding infos about the type of edit. I dont think that such info is yet available.

objects edited - buildings - roads - waterways - landuse - Administrative polygons etc.

Actions new, modifiy, delete

This would add a lot more flexibility about analyzing edits / spot problems.

Analysis of Bounding Box Sizes Over the Last Eight Years over 2 years ago

From the Planet Changesets dump, I did publish brief statistics for 2017 in a Twitter Moment Called Weight of Continents.

I observed for 2017 10 million changesets 6 million where BBOX smaller then 100x100m 11,000 changesets that cover more then a continent

Some kind of Heat Map - I could also represent areas mapped over the continents by tracing all BBOXes except the smaller and larger ones.

Tag Transformations in OpenStreetMap almost 3 years ago

I repeat the block for the unformatted table example

key value thematic Feature
amenity hospital Health Hospital
building hospital Health Hospital
amenity clinic Health Clinic
Tag Transformations in OpenStreetMap almost 3 years ago

To export OpenStreetMap data to the UNOCHA Humanitarian Portal, I group OSM data by thematics such as health, education, etc. For such an Extract-Transform-Load system, we need to assure that we cover well the various thematics (we identifiy all the key-value combinations) and validate key-values to assure quality.

ogr2ogr let’s me load OSM Planet for a country into PostGIS. Sqlite-Spatialite can be used for smaller projects. It recognizes PostGIS functions. Otherwise PostgreSQL-PostGIS.

This way it is easy to test-validate the data. For DR Congo planet file, I filter data using series of condition statements. But this could be done using a translation table :

key value thematic amenity hospital Health building hospital Health amenity clinic Health

In the example above, we could also create a generic feature variable to group various key/values that represent the same feature (for example, hospitals).

Quite important to support projects like for the Ebola outbreak north of DR Congo, we need to identify all features, even if misspelled tags. Tag classification assures that we dont miss features. It also help validate the data with a summary of key/values combination for the country.

For example

Thematic Major Key/value Flag
Education amenity=school  
Education amenity=university  
Health amenity=doctors  
Health amenity=hospital  
Nd. highway=Rond point Gouvernorat !!!
Nd. amenity=Voyage !!!
Public Services and Offices office=Bureau Commussion Ebola Butembo !!!
OpenStreetMap Carto release v4.19.0 almost 4 years ago

It would be nice to start looking at major infrastructures like hydro-electric dams and high-voltage power network. Why not start at the same zoom level then major roads?