OpenStreetMap

Users' Diaries

Recent diary entries

../

Posted by Raquel Dezidério Souto on 8 March 2024 in English.

An event to strengthen alliances between mappers from Brazil and Africa


Read in Portuguese

Between March 2nd and 5th, 2024, the event OPEN DATA DAY - Communities mapping communities: Brazil-Africa connection was held in Salvador (Bahia, Brazil), with the participation of Brazilians and Africans, who presented special keynotes and case studies on collaborative mapping, on the themes of sustainable development, gender equity, the right to adequate housing, humanitarian mapping, disaster risk reduction and urban planning.

The YouthMappers at UFBA, producer of the event, also held mapatonas, or mapping marathons, in the format of hybrid workshops, where participants mapped collaboratively on-line, using the editor iD for OpenStreetMap.

YouthMappers UFRJ participated in the event, with the special keynote Collaborative mapping for risk and disaster reduction - Case study: Municipality of Maricá (Rio de January, Brazil), given by its president, Dr. Raquel Dezidério Souto (IVIDES.org).

WATCH the RECORED VIDEO (in Portuguese and English), clicking here.

READ The PDF PRESENTATION FILE (in Portuguese and English), available at this link.

KNOW about this mapping, on this link.

cover-video-palestra

IVIDES_logo

youthmappers-ufrj

Um evento para aproximar mapeadores do Brasil e da África


Read in English

Entre os dias 02 e 05 de março de 2024, foi realizado em Salvador (Bahia, Brasil), o evento DIA DOS DADOS ABERTOS YouthMappers UFBA - Comunidades mapeando comunidades: Conexão Brasil-África, com a participação de brasileiros(as) e africanos(as), que apresentaram palestras e estudos de caso sobre mapeamento colaborativo, nas temáticas do desenvolvimento sustentável, equidade de gênero, direito à moradia digna, mapeamento humanitário, redução de riscos de desastres e planejamento urbano.

O capítulo YouthMappers at UFBA, produtor do evento, realizou ainda, as mapatonas, ou maratonas de mapeamento, no formato de oficinas híbridas, ministradas ao-vivo pela web, onde os(as) participantes mapearam colaborativamente e utilizando o editor iD para OpenStreetMap.

O YouthMappers UFRJ participou do evento, com a palestra Mapeamento colaborativo para redução de riscos e desastres - Estudo de caso: Município de Maricá (Rio de Janeiro, Brasil), ministrada pela sua presidenta, Dra. Raquel Dezidério Souto (IVIDES.org).

ASSISTA à PALESTRA GRAVADA (em português e inglês), clicando aqui.

LEIA o arquivo PDF da APRESENTAÇÃO (em português e inglês) está disponível neste link.

CONHEÇA mais sobre o MAPEAMENTO, clicando aqui.

capa-video-palestra

IVIDES_logo

youthmappers-ufrj

Introduction

I am writing this tutorial because adding wireless connectivity to the cheapest hobbyist RTK module (Ardusimple simpleRTK2B) is very useful but not trivial for those inexperienced with electronics, and Ardusimple closed down their forums for… some reason. Hopefully this is something anyone can follow to make RTK more accessible to the general hobbyist mapper.

This entry is building on from my previous diary entry “Affordable, High Quality 360 Street Level Imagery using GoPro Fusion and Ardusimple”, I have now been using the rig for a couple years and it is holding up very nicely.

A quick summary is that I used the simpleRTK2B with my phone and GoPro Fusion 360 to make a very cheap and ultra portable street level imagery rig, for Mapillary or Kartaview. It can be used handheld or mounted to a car or bicycle.

The last survey I went on, whilst mostly fine, did bring up some issues regarding the reliability of the USB OTG data connection. This has always been a weak point of the rig in other ways because;

  • the USB connection would drain my phone’s battery and make it difficult to charge the phone simultaneously whilst surveying, unless using wireless charger
  • the phone always needed to be in close proximity to the simpleRTK2B
  • too easy to accidentally disconnect
  • The OTG adapter is wobbly in the type-c port

As my Android app of choice “SW Maps” permits a bluetooth connection for the NTRIP corrections, it was the obvious way to go.

Bluetooth module choice

Ardusimple sells the Bluetooth XBEE module for the simpleRTK2B for €34 which is quite expensive. I know you’re reading this Ardusimple! Third party Bluetooth XBEE modules are somewhat cheaper (around $20 AUD), however I had a generic bluetooth UART module laying around so I decided to just solder it in. Similar ones are quite cheap to buy new, for example the arduino HC-06 module goes for around $10 AUD.

Connecting it up

Here is how I soldered mine; Picture of simpleRTK2B wired to bluetooth module

Make sure you know everything is working before making the permanent connections though! I found that just shoving the wires into the holes was fine for testing purposes; no need to solder on headers if you’re feeling lazy.

Here is the wiring diagram; Wiring diagram

It is important to connect IOREF to 3.3 volts (or whatever logic level is appropriate for your module) otherwise the hardware UART pins will be disabled. Then connect TX to RX and RX to TX.

Note; I was unable to get the UART2 working on the ZED-F9P even after configuring the target in u-center. Apparently it’s a common problem. Maybe this is another advantage of this method over the XBEE method.

Everything in the case and velcro strapped to a power bank; Finished project

The most important thing to getting it working is to match the baud rate between the Bluetooth module and the simpleRTK2B. Most bluetooth modules are configured to 9600 by default but for some reason mine was set to 57600 and I wasted a bunch of time troubleshooting this. The simpleRTK2B UART is often set at 38400. Both should be configurable but I found it easier to configure the simpleRTK2B baud rate using ublox U-center, so I set it to 57600.

Connect the simpleRTK2B to your computer and open U-center, and make sure everything is working OK. Then go View > Messages view (F9), expand UBX > CFG (Config) > PRT (Ports) and set the baud rate to whatever works. Click “Send” and then go UBX > CFG (Config) > CFG (Configuration) and hit “Send” again. I recommend pairing your phone to the bluetooth receiver and using the “Serial Bluetooth Terminal” app to make sure you are receiving the NMEA messages properly rather than just garbage (eg. �������������).

Finally, you can use SW Maps to establish the NTRIP connection. Go to Bluetooth GNSS and select your Bluetooth device. Make sure “Instrument Model” is set to u-blox RTK and that you are logging the data to a file. Then you can authenticate into the NTRIP server and CORS station of your choice and get that sweet RTK.

Posted by mayanaut on 7 March 2024 in English.

Added sidewalks south of Center Avenue between Hazelwood and Hiland, north of ORB/PA-65. Connected sidewalk to previous additions from Ben Avon.

I plan to extend coverage north of Center Avenue within Emsworth, and south of ORB/PA-65.

Added some parking lots here and there where most obvious, and cleaned up the area around Camp Horne Road & ORB intersection. This area could likely use some more work.

Realigned buildings here and there based on updated aerial photography. Might make another pass because there are quite a few that seem to be several meters out of alignment. I’ll have to check the original data these were derived from, but at least so far it seems that the PEMA imagery is the most accurate and precise available. I’ve cross-checked several notable points against the Allegheny County GIS viewer to confirm alignment to the best of my ability for now.

Location: Emsworth, Allegheny County, Pennsylvania, United States
Posted by b-unicycling on 7 March 2024 in English.

(All images via WikimediaCommons)

So, I went on a little outing with a friend to look at some historic/ folkloristic sites that I hadn’t been to before. One of them was an old monastic site where there is still a working church with a holy well (place_of_worship=holy_well) and high cross (historic=high_cross) nearby. When we went to the holy well/ high cross site, I was delighted to see that the site also had a rag tree which hadn’t been recorded on the government’s Historic Environment Viewer. rag tree at St Laserian's Well Now, there could be several reasons for that which I won’t go into, but it reminded me that rag trees were something I had wanted to map, but of course, there was no tagging scheme.

I had mapped some previously using just natural=tree with various other tags, but I thought I needed a tagging scheme that was applicable to other countries and cultures as well.

To explain what a rag tree is (very simplified version): In the Celtic tradition (apparently, but some of these customs go back to the Neolithic), they were often found near a sacred well, and if you had an affliction, lets say sore eyes, you would take a piece of cloth, dip it in the sacred/ holy water, clean your eyes with it or at least rub the afflicted area of your body and tie the rag/ piece of cloth to the tree/ bush. As soon as that piece of fabric had rotted away, your ailment would be gone as well. Needless to say that this only “works” with organic fabrics, not synthetics. This is a process called a votive offering whereas you transfer the ailment onto a place holder (piece of cloth, but in antiquity, could also be a wax model of the afflicted body part - a little bit like voodoo) and leave that at a sacred/ holy/ magic place, so that God/ the Gods/ spirits can take care of your ailment by proxy, so to say. There are also (according what I only read on Wikipedia) “ex-voto” places of worship where you leave a symbol for your ailment behind after you have been healed or saved. There are many crutches left at Lourdes, for example. In the Middle Ages, people would apparently carve ships into church walls after being saved from a shipwreck etc. (That there might be graffiti of ships for other reasons is another matter…) Another weird fact relating to that: People in antiquity and the Middle Ages believed that the womb was like a toad wandering in the body of a woman (something to do with hysterics; I won’t go there now), so whenever they had “women’s trouble”, the votive offering would take the shape of a toad.

So, I pretty quickly settled on the key place of worship, because there is a ritualistic element to the custom which is still very much in use in Ireland, at least. There are also, as someone already pointed out in the Irish Telegram group, similar traditions like wassailing in England which result in decorated trees.

In my opinion, there are also traces of that tradition in the Grimm Brothers fairy tale “Cinderella” which I might go into when I do a video about the topic. ;-) DBP 1965 486 Wohlfahrt Aschenputtel.jpg

I had mapped the Irish examples I had seen and photographed and some I found on Wikimedia was place_of_worship=rag_tree, but I decided to change that to place_of_worship=sacred_tree to cover more religions and cultures. There are several trees tagged as “sacred” in some way or other already on OSM, according to taginfo: https://taginfo.openstreetmap.org/search?q=sacred+tree#values

I’m suggesting on the wiki to use sacred_tree=rag_tree for those in Ireland and the clootie/ cloutie/ cloughtie trees in Scotland to avoid the several spellings in Scotland. Some might also be known as “fairy tree”, but be careful about tagging those; reasons are explained on the Wiki.

I only retagged the Irish examples today, so the infobox still says “This tag does not appear in the OSM database.”, but there are actually a few.

Location: Oldleighlin, Old Leighlin ED, The Municipal District of Muinebeag, County Carlow, Leinster, Ireland

¡Hola a todos! Quiero compartir con ustedes algunos insights y avances del proyecto en el que he estado colaborando, específicamente en la validación de datos. TASK

La calidad de los datos en proyectos de gestión de riesgos es fundamental para garantizar la fiabilidad y precisión de la información utilizada en la toma de decisiones. ¡Pero también es importante recordar que los errores son oportunidades de aprendizaje!

Estos son algunos de los mensajes que he enviado en el task a las personas con base a los errores de mapeo que he encontrado:

En primer lugar un mensaje de agradecimiento: ¡Gracias a todos por su contribución a este proyecto! ¡Sigamos trabajando juntos en el mapeo colaborativo!

Ahora si:

📝 He agregado construcciones faltantes: Tip: Puedes poner imágenes diferentes para detectar todas las construcciones, sin embargo, recuerda siempre dibujar sobre la capa solicitada.

📝 He modificado la etiqueta de una vía: Recuerda que las unclassified las empleamos para las vías que conectan centros poblados.

📝 He modificado la etiqueta de una vía: Recuerda que empleamos la etiqueta service para vías que conectan a un lugar en específico como casas, industrias, negocios o barrios con acceso privado.

📝 He modificado la etiqueta de una vía: Recuerda que la etiqueta residencial la empleamos para vías en las ciudades o centros poblados.

📝 He encontrado que algunas construcciones compartían nodos: Te recomiendo emplear en JOSM el plug-in Building y presionar la letra Ctrl en tu teclado para evitar que las construcciones se peguen.

📝 He encontrado que algunas construcciones eran muy pequeñas: Esto suele pasar cuando los techos tienen color diferente. Te recomiendo emplear otras imágenes como guía.

Velar por el aprendizaje de los nuevos mapeadores es velar también por la calidad del mapa. ¡Ay que seguir apoyando a los nuevos maperos para seguir mejorando juntos!

TASK 16125

Location: Orienco, Pacayacu, Parroquia Pacayacu, Lago Agrio, Sucumbíos, Ecuador

Today marks a milestone in the development of OpenStreetMap NextGen. After months of rigorous development, I conducted the 1st OpenStreetMap NextGen performance benchmark, a crucial step towards realizing the vision of a more robust, efficient, and user-friendly OpenStreetMap.

The focus of today’s benchmark was on evaluating static and unauthenticated requests. Since this core functionality is unlikely to change significantly during future development, it’s the perfect time to test it.

Future benchmarks will focus on timing authenticated requests as well as API 0.6.

What was measured

The benchmark analyzed request processing speed, excluding network and client latency. Both osm-ruby and osm-ng support the X-Runtime response header, which tracks how long it takes to process a request and generate a response.

X-Runtime header in browser inspect tools

Here’s a general breakdown of a typical static request processing:

  • Grabbing configuration settings
  • Checking for authorization (cookies, oauth, etc.)
  • Configuring translations
  • Rendering the HTML template

The setup

The benchmarking setup consisted of local machine testing, as well as official production and development websites.

I initially planned to run the benchmark solely on my local machine, following the official Docker instructions. However, I quickly discovered that the production deployment instructions were outdated and required some Ruby knowledge to fix, which I lacked. In particular, the instructions for replacing the Rails server with Phusion Passenger had been redirected to a generic support page.

osm-ng was launched in production mode with all Python and Cython optimizations enabled. Since we were only dealing with static requests, both local Postgres databases remained empty.

The benchmarking script

I created a basic HTTP benchmarking script that first warmed things up with a few requests before launching into the actual test. It then measured runtime times for a series of HTTP requests, and I repeated the benchmark multiple times for consistency.

A note before the results

It’s important to remember that OpenStreetMap NextGen processes static requests in a similar way to osm-ruby, and it does not (currently) introduce any new caching logic for templates, especially since that would significantly impact the benchmark results (and some people would consider it cheating). osm-ng remains completely backwards compatible with the existing OpenStreetMap platform. Additionally, it’s important to emphasize that the X-Runtime header used for benchmarking is agnostic to network latency, meaning it only measures the processing time on the server itself.

And the winner is…

Here’s a detailed breakdown of the results:

Environment Minimum Runtime (s) Median Runtime (s)
Ruby (local) 0.04264 0.04521
Ruby (official) 0.01892 0.02921
Ruby (test) 0.00913 0.01725
Python 0.00314 0.00325

As you can see, osm-ng consistently outperformed osm-ruby in all test scenarios. The fastest Ruby deployment had the minimum runtime of 0.00913 seconds, while osm-ng achieved the blazing-fast time of 0.00314 seconds, a remarkable 290% performance improvement.

Support the NextGen revolution

I’m truly convinced that OpenStreetMap NextGen will be a game-changer for OpenStreetMap, not just in terms of performance, but also in privacy, security, usability, and overall openness.

If you believe in this project as much as I do, please consider donating so I can keep working on it full-time! 🙏 Every contribution helps push us closer to that first stable release.

And a huge thank you to those who have already supported me!

Today, we benchmarked not just a system, but the future. And the future is bright.

Posted by rtnf on 6 March 2024 in English.

So, I stumbled upon this rain forecast map.

Cool. But I’m still not satisfied. I want to check the forecast situation around my house, specifically.

I have an idea. What if I overlay this image on OpenStreetMap?

Aligning

The first problem that I encountered is on how to properly align it with the OpenStreetMap map. Thankfully, this forecast map includes a coordinate grid.

So, I can show QGIS’s coordinate grid and align it properly.

I see -5.35 on the Y grid and 106.8 on the X grid, so I set a resolution of 0.05 on the Y-axis and 0.2 on the X-axis.

Next, I have to move and scale the image manually by using freehand raster georeferencer.

Stretching

Once it’s roughly aligned, the next thing I do is check the bottom-left coordinate of the image and the top-right coordinate of the image. Leaflet can “stretch” the image based on these two reference coordinate.

But anyway, now I need some code..

Thank you, Poe..

Done

And.. Finished..


But my adventure doesn’t end here…

The next day, after finishing this app, I attended a field event with my local community. To test the accuracy of this forecast system, I frequently used this app to predict whether the event location was going to rain or not. After conducting some direct observations, I came to a conclusion: there are other forecast models that are generally more accurate than this one. One of them is probably cmax(z).

Well, i’m not really sure whether this is a “forecast model” to predict the future, or simply a “data model” to describe the current sky’s situation. I’m not a professional meteorologist, but I consulted with my meteorologist friend on X (formerly Twitter) about this. He said “cmax” means column maximum, the maximum rain intensity around an atmosphere’s vertical column, measured in dbZ unit. So, roughly, it describes the current situation: which place is raining right now, and how intense it is.

Alright. So, what’s next? Just do the exact procedure as before, but use a different image overlay? No, no, no. This one is different. Instead of simply overlaying the whole image, I have to manually “select” each color code and delete every other color that exists on the image.

Thankfully, we’re able to do pixel-by-pixel manipulation using the Canvas API.

And it’s done.

You can check the source code here.


Epilogue

Apparently, because I made too many requests to the weather station’s server, at some point, my IP address was blocked.

At first, I was planning to release this app publicly so that anyone could see the current weather conditions around a specific coordinate in real-time. But knowing that the weather station’s server is very sensitive to surges in traffic, I had to postpone this plan. The app was deployed to my own phone instead, and I used it for my personal use only.

Well, maybe I should talk to the weather station’s authorities before releasing this app publicly.

Posted by spwoodcock on 5 March 2024 in English. Last updated on 6 March 2024.

See pt1 of this series here.

See pt2 of this series here.

A Long Overdue Release

The goal with the Field Mapping Tasking Manager (FMTM) was to adopt an agile development style, making a new release around once per month.

3 months have passed since the last blog post and there was no FMTM release in between!

What gives??

Well, the team has been working extremely hard simplifying usage of FMTM and making it much more usable.

It’s been hard to find a good point to stabilise a release due to so many great and rapidly developed updates!

From this release onwards we plan to follow through with a new version increment every month. Look out for version 2024.4.0 next month.

Public Beta Now Live

  • The main goal we have been working towards is releasing a public beta this month.
  • The public beta for FMTM is now live on https://fmtm.hotosm.org
  • The idea to to have the public test out it’s functionalities & definitely break a thing or two!
  • With your valuable feedback we want to make FMTM the best it can possibly be 🙏

splash

See full details of the release on Github

The main contributors to thank for this release are: @varun2948, @nrjadkry, @Sujanadh, @NSUWAL, @Prajwalism, @manjitapandey and a entirely new contributor @cordovez.

Using The Beta

Feel free to browse around and report any issues via Github or the HOTOSM Slack.

To create a new project, be sure to select the ‘FMTM Public Beta’ organisation while doing so, with the ‘Use Default ODK Credentials’ box checked.

default-org

This will grant you permission normally reserved for organisation admins.

FMTM In Brief

For the uninitiated, I thought I would add a quick aside for how FMTM works:

  • Project is created in an area with three things:
    • Data extract: the features you want to map, say building polygons.
    • ODK XLSForm: the survey for mappers on the ground to fill out for each feature.
    • Task areas divided by feature count and linear features (e.g. rivers, roads).
  • Users assign a task area for themselves, and generate a QR code that is opened in ODK Collect.
  • User navigates to the feature and fills out the XLSForm survey, then submits.
  • The submissions are collected by ODK Central, which feeds the data back into FMTM, for cleaning, conflation with existing data, and pushing back to OSM.

As you can see FMTM is built on top of OpenDataKit actually collect the survey data about map features.

Conflation will be integrated in a future release through HOT’s conflator module by @robsavoye.

Release Highlights

There have been many, many, changes in FMTM since the last release, a lot of them being refinements to the backend too.

Some of the less visible changes: fixing iOS usage, streamlining processes to reduce resource consumption, reducing code complexity for more sustainable development, implementing best practice security for login and session management, fully encrypted ODK credentials, using modern geospatial formats such as flatgeobuf for more efficient data usage and many more.

Full Integration of HOT’s raw-data-api

We have @kshitijrajsharma to thank mostly for this.

raw-data-api allows for near real-time data extraction from OSM, using an innovative data storage structure for very efficient usage.

When you click on ‘Generate Data Extract’ during project creation, this is the tool that works underneath.

raw-data-api

Streamlined Project Creation

Project creation has been significantly streamlined, especially for users that may not know what an ODK XForm is.

project-creation

User Roles

User roles have been implemented, but for the purpose of the beta are mostly disabled to allow full access to test.

Creation of organisations can be requested through a simple web form.

  • Organisation admin: can create projects.
  • Project admin: can modify projects and grant users permissions.
  • Field admin: has special rights to add mappers to a project and unblock tasks.
  • Validator: can validate and submit the final data to OSM.
  • Mapper: the default role for most.

In public project, everyone has the role of mapper.

In private projects this must be explicitly granted.

Project UI

The project details page has a much improved user interface from before, with a listing of task activities and links to various information.

project-details

When the user is ready, they can click ‘Start Mapping’ to lock the task area and scan the QR code in ODK Collect:

qr-code

Submissions UI

A new shiny UI is being developed by the team at NAXA to view the final data submissions.

submissions-ui-1

submissions-ui-2

Project Editing

It is now possible to edit and delete existing projects.

project-edit

Monrovia Test

One of the first projects to use the FMTM Beta was the Liberia Slum Mapping Pilot Project.

Based in Monrovia, HOT and Slum Dwellers International (SDI), plus local partners FOLUPS and YMCA, undertook a mapping campaign for temporary urban settlements, that was very valuable in providing feedback into FMTM use cases.

Two Crucial Insights

  • Not all field mappers are necessarily OSM’ers, so we need to adapt.
  • Many projects in developing countries will not have the features mapped in OSM already, so we first need to get imagery (drone ideally), and integrate HOT’s Tasking Manager to map the features prior to on-the-ground tagging.

Note this is a perfect use case for AI-assisted mapping

Drone Imagery Coverage

monrovia

Mapping Area

morovia-task-area

Generating Polygons From Drone Imagery

building-polygons

Get Involved

Contributing to FMTM with code, documentation, or ideas would be very welcome!

Software developers, technical documentation writers, software testers, or anyone with a general interest - feel free to get in contact.

Even just adding a star on Github helps to show your support 🙌

Future Posts

This was an update on some of the latest features of FMTM and future plans.

There is much that I could not cover, so please check Github for the latest updates!

I plan to make future posts about developments, technical deep dives, and usage of FMTM’s features.

See you next time.

Location: Patte d'Oie, Ouagadougou, Kadiogo, Centre, Burkina Faso
Posted by mayanaut on 5 March 2024 in English.

Started working on extending sidewalk mapping coverage into Emsworth Borough. I’m working much the same way as I did for Ben Avon, mapping sidewalks using a combination of aerial imagery, street-level photographs (Bing, PEMA, Esri, as well as my own), and in-person survey.

I was somewhat surprised to see that a footpath I have used numerous times going back many years was unmapped, so I added it (marked on the associated map).

I am trying how best to approach the area of Memorial Field (adjacent to Ohio River Boulevard / PA-65). The area currently marked shows as a football field, but the actual usage of the field is also a baseball diamond seasonally. In addition the area is shaped and sized differently than currently shown. Also, there are footpaths around it that are somewhat challenging to see on aerial imagery, so I suspect I will have to do some in-person GPS tracking to get better data. I am debating whether or not to map the pedestrian tunnel that passes beneath ORB/PA-65, as in the past I have been admonished that it is illegal to use. But it’s not physically blocked, nor are there posted signs indicating the access status. I just don’t want someone to wander through it and get into trouble, so maybe I’ll map it and tag it private or something to discourage public use until I can confirm.

I will probably bounce around between sidewalks and other features as I come across them.

Location: Emsworth, Allegheny County, Pennsylvania, United States
Posted by Zkir on 5 March 2024 in Russian (Русский).

OSM в руках синьки и норкоманов – это несомненный успех.

Если по существу вопроса – если раньше мы жили в гипо-информационном обществе, то теперь живем в гипер-информационном. Что-то спрятать не получится, придется привыкать.

С другой стороны, когда я на свой майнкрафт-сервер приделал онлайн карту, тоже далеко не все игроки обрадовались.

OSM in the hands of alcoholics and drug addicts is an undoubted success!

Previously we lived in a hypo-information society, now we live in a hyper-information society. It is not possible to hide something, you’ll have to get used to it.

On the other hand, when I attached an online map to my Minecraft server, not all players were happy.