OpenStreetMap

NorthCrab's Diary

Recent diary entries

Welcome to the sixth OpenStreetMap NextGen development diary.

This week, I continued the preparation of the project for the first development release scheduled for the end of this month 🔨.

🔖 You can read my other development diaries here:
https://www.openstreetmap.org/user/NorthCrab/diary/

📖 This project is open-source and publicly available:
https://github.com/Zaczero/openstreetmap-ng

🛈 This independent initiative is not affiliated with the OpenStreetMap Foundation.

In Case You Missed It…

OpenStreetMap-NG is planned to have its first development release at the end of this month, May. After this milestone, the project will be open for new contributors! My current work focuses on delivering on that promise, finishing the core functionalities, and stabilizing the code.

Originally posted in diary #5.5.

Finishing Up the Elements Sidebar

The elements sidebar has been mostly finished. The “Part of” and “Members/Nodes” sections are now more consistent in design and have received pagination support. The orange-colored map visualization is also now working.

Preliminary API 0.7 /map Implementation

The data layer is now working and uses the new API 0.7 for map requests. It also utilizes the new rendering algorithm, which will provide much better performance than the current implementation, handling more elements in view. The new map view padding optimization also reduces the number of API requests when panning around, providing a smoother user experience.

Public Feature Ideas

I have also publicized some collected feature ideas on the GitHub Issues tracker. These currently include only medium and long-term ideas. Short-term ideas will be added closer to this month’s deadline. Some features embedded within the code will also be added to this list soon - currently it’s not exhaustive as it covers only my personal notes.

General Code Cleanup

This week, the code has undergone cleanup. The database usage for private structures has also been reduced. By switching from UUID to Snowflake ID-inspired type, the type now takes just 64 bits of space instead of 128 bits.

Project Sponsors 🏅

Time for the weekly appreciation of the current project patrons. Thank you, everyone, for supporting the project, including those who starred it on GitHub! We are making it real 😎.

Currently, the project is sponsored by 14 people!
Five private and four public donors on Liberapay, and five public on GitHub Sponsors.

If you can, please consider supporting the OpenStreetMap-NG development 🦀:

Donate using Liberapay

Welcome to my fifth (and a half) OpenStreetMap NextGen development diary.
Tomorrow, I’m returning home and I’ll be able to resume work at full speed 🔥.

This is a short edition of the development diary.

🐙 This project is open-source and publicly available:
https://github.com/Zaczero/openstreetmap-ng

Intro

For the past 13 days I have been on a journey of finding a new place to rent. Without my home office, I wasn’t able to become productive. The place I’m staying at doesn’t have a good office spot and being on my laptop doesn’t help. However, I am now very motivated to get back to work and push even harder!

May Will Be Big

At the end of May, OpenStreetMap-NG will include necessary functionality to run on a testing server, as well as to invite new contributors into the project. Starting with 6th of May, I won’t have any time-consuming plans for this month so I’ll do my best to wrap everything up. What’s exactly left has been described in Diary #5 Short-Term Development Plan. I have already started to prepare the All-in-One Contributor Guide which will also be finished up (it currently lacks backend/frontend-specific guides). This is going to be the first major milestone of the project!

Project Sponsors 🏅

I was happily surprised to see new faces even during my lower activity period. I will do everything to deliver the promised results. As always, thank you for supporting the project, monetarily, and with staring the project on GitHub!

Currently, the project is sponsored by 13 people!
Five private and four public donors on Liberapay, and four public on GitHub Sponsors.

Disclaimer

This project is not affiliated with the OpenStreetMap Foundation. It’s an independent and community-sponsored initiative.

I am currently on a visit to Ireland 🇮🇪 and a lack of proper office space makes it difficult to stay productive. I will try to prepare something cool to show off this week. Sorry for keeping you waiting!

🍟

Location: Murphystown, Leopardstown Rise, Glencullen Electoral Division, Sandyford, Dún Laoghaire-Rathdown, County Dublin, Leinster, D18 CV48, Ireland

Welcome to my fifth OpenStreetMap NextGen development diary.
This week has been mostly focused on GPS Traces 🛰.

🔖 You can read my other development diaries here:
https://www.openstreetmap.org/user/NorthCrab/diary/

🐙 This project is open-source and publicly available:
https://github.com/Zaczero/openstreetmap-ng

GPS Traces Demo

It’s best to experience the refreshed traces in a video form, so I prepared a short demo (no audio):
⏯️ https://files.monicz.dev/osm/traces-demo.mp4

For comparison, here’s how the same trace looks on the current website:
https://www.openstreetmap.org/user/bielebog/traces/11326871

You will quickly notice the super-fast upload speed and the new trace animations. If there’s something wrong with the file you attached, you will receive instant feedback on the upload page. One new feature is possibility to edit trace name. Previously, this feature has been hidden behind API 0.6.

One more planned feature is rendering the map behind the trace animation. Now that the system works on individual coordinates, it will be fairly easy to implement. Traces without a human-understandable point of reference are not as useful as they could be.

Unified Traces URLs

Let’s start with discussing the current URL routes.

  • If you want to access a way, you visit /way/<ID>
  • If you want to access a note, you visit /note/<ID>
  • If you want to access a trace, you visit /user/<USER>/traces/<ID>

Which is not consistent. OpenStreetMap-NG unifies this experience by introducing a new URL route: /trace/<ID>. All existing URLs remain backwards compatible and are automatically redirected.

Short-Term Development Plan

There’s just a few things left before reaching the core feature parity with the existing website. Those are the things I want to finish before inviting new contributors ツ.

  • Elements sidebar (50% work in progress)
  • User Diaries
  • User Profiles
  • Applications (OAuth) settings

Contributor Benefits

Last week I hinted towards the announcement of a contributor benefit. Today, I will talk shortly on 1 of 2 currently planned ideas, that will help the project grow and stay strong.

Firstly, who are the “contributors”? Those are the people who help the OpenStreetMap-NG project. For example: by testing the website, donating, contributing code, helping with localization, graphics and interface design, etc.. The scope is broadly defined, as people can contribute in many different ways!

Contributors joining before the project is officially accepted as the main OpenStreetMap website, will be able to become a member of the NextGen Founders invite-only community and receive a small badge on their user profiles. This feature is a part of the original announcement (under the name “Community Profiles”).

This is a time-limited benefit, that provides a unique thank-you to all people that help (and will help) making this project a reality. The 2nd benefit will be announced in some time in the future.

Project Sponsors 🏅

Here’s my weekly appreciation to the current project patrons. Thank you for believing and helping me do what I love :-)

Currently, the project is sponsored by 11 people!
Five private and three public donors on Liberapay, and three public on GitHub Sponsors.

If you can, please consider supporting the NG development:

Donate using Liberapay

Disclaimer

This project is not affiliated with the OpenStreetMap Foundation.

Welcome to my fourth OpenStreetMap NextGen development diary.
Sorry for being a day late! I wanted to finish up one of the new features which caused this small delay. ✨

🔖 You can read my other development diaries here:
https://www.openstreetmap.org/user/NorthCrab/diary/

🐙 My work is available publicly on this GitHub repository:
https://github.com/Zaczero/openstreetmap-ng

GitHub stars counter

Let’s summarize the last week’s work:

Client-side Trace Images

While migrating the traces functionality, I came up with an amazing and seemingly obvious idea. Why not make trace images SVGs and render them client-sided? This feature has few significant advantages: even faster trace uploading, no additional disk usage, unlimited customization, infinite resolution, faster page loading. And so here it is:

Comparison screenshot of the new client-side generated trace images (SVGs)

The application can freely adjust the quality of generated images. The code can also be reused for implementing proper trace-on-map rendering, which is one of the new upcoming features.

SVG supports animations too!

Animated trace using SVG

Refreshed Traces UI

Screenshot showcasing refreshed public traces UI

» Open in full screen

Last week I have also worked on refreshing the traces UI, focusing on making it more open and friendly. If you have been following my previous diaries, you may recognize some of the new style language.

Deployment Scripts

I also wrote and successfully tested server-deployment scripts for the application. They are currently a part of the openaedmap-backend project but will soon be copied over to the openstreetmap-ng. Both projects share many similarities in how they are run.

Final Words

Previous development diary #3 was packed with lots of new stuff. I took this week a little slower to catch a breath. Meantime, I contributed to other projects (openaedmap, starlette) and also helped OSM-PL with server migration process. I was also away for a short time for some BBQ🌞!

Project Sponsors 🏅

Thank you to this week’s project patrons! I truly appreciate your every contribution!

Currently, the project is sponsored by 11 people!
Five private and three public donors on Liberapay, and three public on GitHub Sponsors.

If you can, please consider supporting the NG development. Any amount helps me focus on making high-quality free software for everyone to enjoy! (shh… next week I will announce a new supporter benefit) 😋

Donate using Liberapay

Disclaimer

This project is not affiliated with the OpenStreetMap Foundation.

OpenStreetMap NextGen Development Diary #3 - Packed with Goodies

Posted by NorthCrab on 31 March 2024 in English. Last updated on 1 April 2024.

Welcome to my third OpenStreetMap NextGen development diary.
This week has been super busy and I can’t wait to show you the progress 🧑‍🍳!

You can subscribe to my diary updates on RSS here:
https://www.openstreetmap.org/user/NorthCrab/diary/rss

Video Presentation 🎉

This week I was focused on integrating features together, as we get closer and closer to the first development release. I am super happy with how things are progressing and I decided to record a short video presentation (3 minutes).

⬇ Click below to play ⬇

Video thumbnail

or click here: https://peertube.monicz.dev/w/kGnomi7LTveXZNaaQtEwH6

Refreshed Changeset UI

Changeset sidebar comparison screenshot, showcasing simplified and refreshed UI

OpenStreetMap-NG now features a refreshed changeset sidebar. Featuring, profile pictures and de-emphasized discussion subscription button. I’ll collect feedback on this design decision (and others) after the first development release.

Working Rapid & iD editors

OpenStreetMap-NG delivers on its promise and features Rapid editor as an alternative to iD.

Navbar screenshot, highlighting Rapid editor option

A lot of effort was put in into securing against potentially malicious JavaScript code making requests as a user to the website (not API). Uncompressed Rapid editor weighs about 5MB, with iD being around 4MB. A library of such size is impossible for a human to review properly, making it an easy target for abuse. With OpenStreetMap-NG, hosted editors can’t physically use user cookies, mitigating all the risk.

New Icons System

Elements view, showcasing deleted nodes with defibrillator and bench icons

There are plenty more feature icons and many original ones are now in higher resolution. Many of the icons have been borrowed from OpenLevelUp editor as they closely resemble the existing icon style and blend together nicely. There are currently 286 features with icons.

There is also this neat new feature, icons and feature names are now displayed correctly for deleted elements. This should make browsing through changesets easier.

Elements Pagination 2.0

Screenshot of pagination control under the elements list

Navigating through element pages is now instant and does not require a full sidebar reload. Your browser only loads necessary resources to display the current page of elements. It’s fast, it’s efficient, it’s simple — and most importantly, it’s not annoying to use!

(check out the video presentation for a short demo 😉)

Support for Multiple Value Tags

OpenStreetMap-NG introduces native support for key prefixes/suffixes and multi-value tags. The system can be easily extended with support for new tagging schemas with the use of formatting plugins. Keys are separated by : and values are separated by ;.

And More…

All of my work is publicly accessible on this GitHub repository. While my development diaries focus on selected highlights, there are many more details to read through. You can always browse the commit history with day-by-day updates.

Project Sponsors 🦀

I am super grateful for all this week’s project patrons. Your growing support makes me work at 110% capacity! Everything is possible if you just believe it. Thank you 🫰.

Currently, the project is sponsored by 10 people!
Five private and two public donors on Liberapay, and three public on GitHub Sponsors.

This week’s donations go directly towards the development of OpenStreetMap-NG.

Donate using Liberapay

Disclaimer

This project is not affiliated with the OpenStreetMap Foundation.

Welcome to my second OpenStreetMap NextGen development diary. I am ready to highlight this week’s progress — and there was a lot 👷!

You can subscribe to my diary updates on RSS here:
https://www.openstreetmap.org/user/NorthCrab/diary/rss

Finished Settings Page

Last week I showcased initial progress on the new settings page for OpenStreetMap. This week, I’ve finished it. Here’s what’s new. Keep in mind that if you believe some features should be done differently, there will be a period of public testing where I’ll collect this kind of feedback 😉.

New settings page screenshot

Click here to view it in full screen.

Aside from the improved layout, I’ve introduced two notable changes. First, the application language selection has been redesigned; it’s now a simple dropdown selector. I believe the current system (where you input locale names by hand) is difficult for less technical people to understand, and I want to make the website more accessible to everyone!

New privacy controls screenshot

Secondly, OpenStreetMap finally gains account privacy controls. Want to opt out of activity tracking but still send crash reports? No problem! Today’s opt-out implementation relies solely on the Do-Not-Track browser setting. Even when opted out, you’ll still connect to analytics servers (for some reason). The NextGen codebase supports Do-Not-Track headers, the new Global Privacy Control, and does not connect to analytics servers when opted out. Account configuration takes precedence over browser configuration.

Seamless 180th Meridian Handling

This week I’ve invested time in smoothing out handling of the 180th meridian in both the frontend and backend. When browsing the map around the 180th meridian, there are no more visual glitches when crossing it. Additionally, the map and notes layers will no longer throw exceptions around it. The new position handling script is also significantly smaller compared to the existing Ruby codebase. The backend API now seamlessly handles 180th meridian requests.

The current API integration when requesting an area from 160° to 200°:

  • Send 1st request from 160° to 180°
  • Send 2nd request from -180° to -160°
  • Merge results

The new integration:

  • Send 1 request from 160° to 200° (or from -200° to -160°)

Splitting, wrapping, and merging are done under the hood by the API, significantly reducing code complexity in many consumer applications. Less complexity means fewer bugs and easier integration with OpenStreetMap.

System Apps and Test Users

As I actively work towards creating the best developer experience, this week I focused on two new features: system apps and test users.

System apps are OAuth2 applications managed by the application itself, not by users. Currently, there are two system apps: iD and Rapid — the supported web editors.

INFO:     | 2024-03-23 23:43:45 | Registering system app 'iD'
INFO:     | 2024-03-23 23:43:45 | Registering system app 'Rapid'

In the current Ruby codebase, you need to follow all these steps just to get your own iD instance running. OpenStreetMap-NextGen does this automatically on application launch for you — you’re welcome 🙂.

Debug login panel screenshot

Test users are just that: users for testing. I like when applications provide developers with multiple users ready to be used, rather than requiring them to go through the registration process. Contributors can now log in to any of these users with a single button press!

Progress on Notes and Changesets API 0.6

Changeset read API response screenshot

I’m actively working on the migration of existing API features. This week, I successfully tested CRUD (Create, Read, Update, Delete) operations on notes and changesets. There were a few issues that needed correction, but nothing notable.

And More…

All of my work is publicly accessible on this GitHub repository. While my development diaries focus on selected highlights, there are many more details to read through. You can browse the commits history.

Project Sponsors 🦀

I want to take a moment to personally thank this week’s project patrons. It was super motivating to see new supporters! I really appreciate all of you! 🫰

Currently, my work is sponsored by 5 patrons on Liberapay, including three private donors. I have one public donor on GitHub Sponsors and have also received a private donation via PayPal.

~1847430, gileri, 快乐的老鼠宝宝/LaoshuBaby

If you’d like to join my development sponsors, you can find me on Liberapay or GitHub Sponsors. Currently, all contributions go directly towards the development of OpenStreetMap NextGen 🚀.

Donate using Liberapay

Disclaimer

Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices.

Today I decided to introduce a new format for sharing OpenStreetMap-NextGen development progress with the community. I’ll post weekly/bi-weekly updates highlighting changes and the current project status. Since this is the first update, I’ll cover some recent highlights.

You can subscribe to my diary updates on RSS: link.

New Settings Page (⭐ Highlight)

New settings page screenshot

I’ve begun migrating the settings/preferences section. My goal is to streamline this experience, as I’ve found the current system a bit complex. Surprisingly, many users don’t know it’s possible to change the default editor — I want to make this more obvious.

A new menu on the left of the screenshot (hidden, not yet finished) will provide clear navigation between general, 2FA, OAuth, and other settings.

This page is still work in progress. I intend to add a help text explaining how to contribute to translations and that the translations are made by the community.

Image optimization logging output, console screenshot

This screenshot highlights a new image optimization algorithm, which uses a binary search-like algorithm to find the perfect image optimization configuration in limited amount of steps.

Last Week’s Progress

I am heavily focused on migrating the HTML templates and pages. I believe it is a critical step towards opening up the NextGen codebase to new contributors. Without those (mostly) functional pages, it is difficult to add new features or improvements.

The following templates have been worked on:

  • /welcome - finished
  • /fixthemap - finished
  • email base template - finished
  • email signup confirm - finished
  • /settings - work in progress

I have additionally addressed issues that lead to some issues with API endpoints, as well as worked on frontend and backend optimizations. You can see the full breakdown in the repository commits log. I keep my work completely transparent.

OpenStreetMap Website Vulnerability Report

I finally published my OpenStreetMap website vulnerability report. I conducted this security audit while studying the website source code, which was a mandatory step to preserve backwards compatibility.

Some of the highlight findings is a security flaw allowing an attacker to blindly reply to any private message as anybody. Another surprising finding is that the Ruby website stores user authentication tokens in plain text. If an attacker had gained access to the server where these tokens were stored (with just read access), they could have potentially compromised a large number of accounts.

All of the vulnerabilities have already been fixed or are being fixed in the NextGen implementation.

OpenStreetMap NextGen Benchmark 1 of 4: Static and unauthenticated requests

I have recently published the first benchmark of the OpenStreetMap-NG. It focuses on measuring static and unauthenticated requests as this code is fairly stable unlikely to be changed. Future benchmarks will include more realistic scenarios.

I compared the results with the current Ruby website implementation. I faced issues with reproducing deployment scenario on my local machine due to outdated documentation (and since I am a Ruby-noob, I couldn’t fix it myself).

Despite the imperfect benchmarks, I believe the obtained numbers hint at the potential performance gains of NextGen’s codebase.

🦀 Project Sponsors

In my development diaries, I want to include a dedicated section thanking my current project patrons. It’s through their support that I’m able to work full-time on OpenStreetMap-NextGen. Rather than focusing on the amount donated, I want to highlight the individuals themselves — it’s the gesture that is the primary driving factor.

Currently, my work is sponsored by 2 patrons on Liberapay, including one private donor, and one public donor with the mysterious looking username ~1847430.

Thank you to both of you, you made me smile 😋.

If you’d like to join my development sponsors, you can find me on Liberapay or GitHub Sponsors. Currently, all contributions go directly towards the development of OpenStreetMap NextGen.

Donate using Liberapay

Disclaimer

Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices.

OpenStreetMap Website Vulnerability Report

Posted by NorthCrab on 15 March 2024 in English. Last updated on 18 March 2024.

As part of my commitment to OpenStreetMap-NextGen migration, I undertook a comprehensive security review of the existing OpenStreetMap website. I followed the principles of coordinated vulnerability disclosure, working directly with the maintainers to responsibly report my findings. Today, I’m making the details of this process public and verifying the status of the fixes.

Disclosure Timeline

  • 2023-11-04 - Contacted Ruby security maintainers & disclosed the timeline publicly
  • 2023-11-08 - Maintainers acknowledged the report
  • 2023-12-04 - Reported additional vulnerabilities with a 3-month deadline
  • 2023-12-06 - Maintainers acknowledged the additional report
  • 2024-03-02 - Publicize vulnerability details
  • 2024-03-15 - Publicize vulnerability details

1. Plain-Text Authentication Token Storage

Authorization tokens (oauth, session, user tokens) are stored in plain text. Read access to the database allows full impersonation of any user.

Status as of 2024-03-15: Partially vulnerable (oauth still depends on plain text storage)

NextGen Codebase Status: Fixed (all authorization tokens and credentials are hashed or encrypted for secure storage)

2. Insecure Email Reply Address Tokenization

The tokens used to ensure the authenticity of email replies are too short (24-bit), making them susceptible to brute-force attacks. An attacker could potentially guess the token and impersonate a legitimate user in email conversations. This vulnerability is especially dangerous if the attacker already has access to a conversation’s metadata (allowing for complete security bypass).

Status as of 2024-03-15: Fixed (now with 48-bit security)

NextGen Codebase Status: Fixed (128 or 256-bit security; undecided)

3. Unbounded GPX Extraction Denial of Service

The OpenStreetMap website does not impose limits on the number of files that can be extracted from a GPX archive. Attackers can exploit this by uploading specially crafted archives containing a massive number of files, potentially overwhelming the system and causing it to crash.

Status as of 2024-03-15: Mostly vulnerable (a partial fix limits points within trace files but does not fully address the issue)

NextGen Codebase Status: Fixed

4. Notes Search Query Denial of Service

The /api/0.6/notes/search endpoint is poorly optimized. This allows attackers to craft specific search queries that are extremely resource-intensive, effectively using the search feature to launch a denial of service attack.

Status as of 2024-03-15: Vulnerable

NextGen Codebase Status: Not yet tested

5. String Comparison Timing Attack

The way the code compares strings can, under certain circumstances, leak timing information. When attackers can carefully measure these timing differences, they might be able to extract secret information, such as security keys and tokens.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Fixed

6. User Block and Rate Limit Bypass

A loophole exists where a malicious user can repeatedly delete and recreate their account (using the same credentials) to circumvent security measures like user blocks and rate limits.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Not yet tested

7. Application Preference Leakage

OpenStreetMap allows different applications to store user preferences. A flaw exists where any application can access preferences created by a different application. This could leak information, and might be abused to attack other vulnerable applications that mistakenly trust data stored in user preferences.

Status as of 2024-03-15: It’s intended behavior

NextGen Codebase Status: Fixed (in API 0.7; via optional preference partitioning)

8. Unrestricted Email Checking via Password Recovery

The password recovery feature can be abused to check if any arbitrary email address is associated with an OpenStreetMap account.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Fixed

9. Forced Downgrade from OAuth1.0a to OAuth1.0

Under specific conditions, the authentication protocol can be downgraded from the more secure OAuth1.0a to the less secure OAuth1.0. This downgrade would omit important security checks.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Fixed

10. Insufficient User Token Integrity Checks

Tokens, used for things like password resets or confirming email changes, aren’t properly differentiated. An attacker might be able to use a password reset token to change an email address, or vice-versa.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Fixed

11. Inadequate Randomness Source in Token Generation

The system generating security tokens relies on a weak source of randomness. Predictable tokens are much easier for attackers to guess or manipulate.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Fixed

12. Unicode Normalization Flaw Allowing Email/Display Name Duplication

The lack of Unicode normalization means a user can register with an email address or display name that is identical to an existing one (but uses different Unicode characters). This could be used to trick users or bypass database restrictions.

Status as of 2024-03-15: Fixed

NextGen Codebase Status: Not yet tested


Support the NextGen development

I work on the OpenStreetMap-NextGen full-time. If you find my work valuable, please consider supporting the development on Liberapay or GitHub. Every contribution helps push us closer to that first stable release. Thank you! 🦀

And a huge thank you to those who have already supported me!

Submit a correction

If you find any inaccuracies or have suggestions to improve this diary, please feel free to contact me via https://monicz.dev/#get-in-touch.

Disclaimer

Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices.

Today marks a milestone in the development of OpenStreetMap NextGen. After months of rigorous development, I conducted the 1st OpenStreetMap NextGen performance benchmark, a crucial step towards realizing the vision of a more robust, efficient, and user-friendly OpenStreetMap.

The focus of today’s benchmark was on evaluating static and unauthenticated requests. Since this core functionality is unlikely to change significantly during future development, it’s the perfect time to test it.

Future benchmarks will focus on timing authenticated requests as well as API 0.6.

What was measured

The benchmark analyzed request processing speed, excluding network and client latency. Both osm-ruby and osm-ng support the X-Runtime response header, which tracks how long it takes to process a request and generate a response.

X-Runtime header in browser inspect tools

Here’s a general breakdown of a typical static request processing:

  • Grabbing configuration settings
  • Checking for authorization (cookies, oauth, etc.)
  • Configuring translations
  • Rendering the HTML template

The setup

The benchmarking setup consisted of local machine testing, as well as official production and development websites.

I initially planned to run the benchmark solely on my local machine, following the official Docker instructions. However, I quickly discovered that the production deployment instructions were outdated and required some Ruby knowledge to fix, which I lacked. In particular, the instructions for replacing the Rails server with Phusion Passenger had been redirected to a generic support page.

osm-ng was launched in production mode with all Python and Cython optimizations enabled. Since we were only dealing with static requests, both local Postgres databases remained empty.

The benchmarking script

I created a basic HTTP benchmarking script that first warmed things up with a few requests before launching into the actual test. It then measured runtime times for a series of HTTP requests, and I repeated the benchmark multiple times for consistency.

A note before the results

It’s important to remember that OpenStreetMap NextGen processes static requests in a similar way to osm-ruby, and it does not (currently) introduce any new caching logic for templates, especially since that would significantly impact the benchmark results (and some people would consider it cheating). osm-ng remains completely backwards compatible with the existing OpenStreetMap platform. Additionally, it’s important to emphasize that the X-Runtime header used for benchmarking is agnostic to network latency, meaning it only measures the processing time on the server itself.

And the winner is…

Here’s a detailed breakdown of the results:

Environment Minimum Runtime (s) Median Runtime (s)
Ruby (local) 0.04264 0.04521
Ruby (official) 0.01892 0.02921
Ruby (test) 0.00913 0.01725
Python 0.00314 0.00325

As you can see, osm-ng consistently outperformed osm-ruby in all test scenarios. The fastest Ruby deployment had the minimum runtime of 0.00913 seconds, while osm-ng achieved the blazing-fast time of 0.00314 seconds, a remarkable 290% performance improvement.

Support the NextGen revolution

I’m truly convinced that OpenStreetMap NextGen will be a game-changer for OpenStreetMap, not just in terms of performance, but also in privacy, security, usability, and overall openness.

If you believe in this project as much as I do, please consider donating so I can keep working on it full-time! 🙏 Every contribution helps push us closer to that first stable release.

And a huge thank you to those who have already supported me!

Today, we benchmarked not just a system, but the future. And the future is bright.

I am really excited to finally share some tangible progress on OpenStreetMap NextGen! While I have been working on this project almost day-by-day for the past 3 months, just recently I reached the point of launching the website (for now locally) and begun development of its frontend features.

OpenStreetMap-NG has grown immensely in scope since the original announcement. Now the project not only focuses on core performance, usability, and accessibility but also on improving the user-facing interface and experience. I want this project to truly stand by its name, providing NextGen value to all aspects of the website. And so far, it’s on the right track.

Directions side-by-side comparison.

So far, the number of improvements is counted in hundreds, including but not limited to various bug-fixes, performance optimizations, privacy, and security enhancements. And the list keeps on growing!

Share panel side-by-side comparison.

My current primary focus is to wrap up frontend baseline development and prepare an amazing developer documentation so other people can hop in and help. I still need a little more time to prepare the grounds as I want to provide a good contributor experience and ensure everyone’s time is used effectively.

Data export side-by-side comparison.

The project is currently funded through individual donations. You can help it grow by donating here, or by leaving a star on the GitHub project itself, or both :-)

An orange crab holding a glowing star, space ship flying in the background

Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices. The presented side-by-side comparisons are subject to change as the development is ongoing.

OpenStreetMap Service Availability (2023-12-20 - 2024-01-20)

Posted by NorthCrab on 25 January 2024 in English. Last updated on 27 January 2024.

I have started an independent collection of OSM SLA statistics. Approximately once a month, I will publish my results with the aim of enhancing transparency regarding the reliability of OSM services. I use uptime-kuma to run monitoring. I also verify connectivity with non-OSM services (to prevent false positives). The current configuration includes checking the availability of openstreetmap-website and openstreetmap-cgimap (API). Tile layer availability is not currently included in the checks. The health-check resolution is set to 30 seconds, and the checks are executed from a single server in the Hetzner datacenter in Germany. For the endpoint to be marked unavailable, two consecutive checks must fail. This should be well-representative of an average user experience.

Summary

Total API downtime: 10 minutes and 37 seconds

API 31D SLA: 99.976%

Total website downtime: 30 minutes and 6 seconds

Website 31D SLA: 99.932%

Note that some functionalities of the website require API to also be available.

Details

2024-01-02 11:30:00 - 2024-01-02 11:34:32

  • Total downtime: 4 minutes 32 seconds
  • 🌐 Website unavailable

2024-01-09 12:51:39 - 2024-01-09 12:53:10

  • Total downtime: 1 minute 31 seconds
  • 🌐 Website unavailable

2024-01-09 12:56:58 - 2024-01-09 12:59:28

  • Total downtime: 2 minutes 30 seconds
  • 🌐 Website unavailable

2024-01-09 13:07:18 - 2024-01-09 13:07:48

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-09 13:09:57 - 2024-01-09 13:10:27

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-09 13:16:36 - 2024-01-09 13:19:21

  • Total downtime: 2 minutes 45 seconds
  • 🌐 Website unavailable

2024-01-14 16:10:19 - 2024-01-14 16:11:04

  • Total downtime: 45 seconds
  • 🌐 Website unavailable

2024-01-14 16:15:55 - 2024-01-14 16:16:25

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 16:17:58 - 2024-01-14 16:18:29

  • Total downtime: 31 seconds
  • 🌐 Website unavailable

2024-01-14 16:20:02 - 2024-01-14 16:21:02

  • Total downtime: 1 minute
  • 🌐 Website unavailable

2024-01-14 16:25:02 - 2024-01-14 16:25:32

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 16:31:45 - 2024-01-14 16:32:30

  • Total downtime: 45 seconds
  • 🌐 Website unavailable

2024-01-14 16:44:43 - 2024-01-14 16:45:13

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 16:49:22 - 2024-01-14 16:49:52

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 16:50:54 - 2024-01-14 16:51:24

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 16:54:14 - 2024-01-14 16:54:59

  • Total downtime: 45 seconds
  • 🌐 Website unavailable

2024-01-14 17:00:18 - 2024-01-14 17:01:18

  • Total downtime: 1 minute
  • 🌐 Website unavailable

2024-01-14 17:11:08 - 2024-01-14 17:12:09

  • Total downtime: 1 minute 1 second
  • 🌐 Website unavailable

2024-01-14 17:14:44 - 2024-01-14 17:15:29

  • Total downtime: 45 seconds
  • 🌐 Website unavailable

2024-01-14 17:19:07 - 2024-01-14 17:19:37

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 17:21:42 - 2024-01-14 17:22:12

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 17:41:20 - 2024-01-14 17:43:35

  • Total downtime: 2 minutes 15 seconds
  • 🌐 Website unavailable

2024-01-14 18:33:53 - 2024-01-14 18:34:38

  • Total downtime: 45 seconds
  • 🌐 Website unavailable

2024-01-14 18:38:46 - 2024-01-14 18:39:16

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-14 18:48:46 - 2024-01-14 18:49:17

  • Total downtime: 31 seconds
  • 🌐 Website unavailable

2024-01-14 18:52:57 - 2024-01-14 18:53:57

  • Total downtime: 1 minute
  • 🌐 Website unavailable

2024-01-14 20:32:08 - 2024-01-14 20:33:23

  • Total downtime: 1 minute 15 seconds
  • 🌐 Website unavailable

2024-01-14 20:35:29 - 2024-01-14 20:36:29

  • Total downtime: 1 minute
  • 🌐 Website unavailable

2024-01-14 20:55:32 - 2024-01-14 20:56:02

  • Total downtime: 30 seconds
  • 🌐 Website unavailable

2024-01-15 02:21:49 - 2024-01-15 02:23:19

  • Total downtime: 1 minute 30 seconds
  • 🚩 API unavailable

2024-01-15 02:25:49 - 2024-01-15 02:27:24

  • Total downtime: 1 minute 35 seconds
  • 🚩 API unavailable

2024-01-15 04:50:55 - 2024-01-15 04:57:42

  • Total downtime: 6 minutes 47 seconds
  • 🚩 API unavailable

2024-01-19 22:53:45 - 2024-01-19 22:54:30

  • Total downtime: 45 seconds
  • 🚩 API unavailable

OpenStreetMap Service Availability (2023-11-20 - 2023-12-20)

Posted by NorthCrab on 27 December 2023 in English. Last updated on 27 January 2024.

I have started an independent collection of OSM SLA statistics. Approximately once a month, I will publish my results with the aim of enhancing transparency regarding the reliability of OSM services. I use uptime-kuma to run monitoring. I also verify connectivity with non-OSM services (to prevent false positives). The current configuration includes checking the availability of openstreetmap-website and openstreetmap-cgimap (API). Tile layer availability is not currently included in the checks. The health-check resolution is set to 30 seconds, and the checks are executed from a single server in the Hetzner datacenter in Germany. For the endpoint to be marked unavailable, two consecutive checks must fail. This should be well-representative of an average user experience.

Summary

Total API downtime: 2 hours 34 minutes 13 seconds

API SLA: 99.643%

Total website downtime: 43 minutes 15 seconds

Website SLA: 99.900%

Note that some functionalities of the website require API to also be available.

Details

2023-11-30 10:07:24 - 2023-11-30 10:09:40

  • Total downtime: 2 minutes 16 seconds
  • 🚩 API unavailable

2023-11-30 10:16:16 - 2023-11-30 10:21:31

  • Total downtime: 5 minutes 15 seconds
  • 🚩 API unavailable

2023-11-30 21:34:44 - 2023-11-30 21:36:14

  • Total downtime: 1 minute 30 seconds
  • 🚩 API unavailable

2023-12-01 04:27:18 - 2023-12-01 04:29:00

  • Total downtime: 1 minute 42 seconds
  • 🚩 API unavailable

2023-12-01 17:26:20 - 2023-12-01 17:29:20

  • Total downtime: 3 minutes
  • 🚩 API unavailable

2023-12-01 17:45:06 - 2023-12-01 17:50:39

  • Total downtime: 5 minutes 33 seconds
  • 🚩 API unavailable

2023-12-01 17:50:39 - 2023-12-01 18:04:09

  • Total downtime: 13 minutes 30 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-01 18:04:09 - 2023-12-01 18:06:51

  • Total downtime: 2 minutes 42 seconds
  • 🚩 API unavailable

2023-12-01 18:09:38 - 2023-12-01 18:10:23

  • Total downtime: 45 seconds
  • 🚩 API unavailable

2023-12-01 18:12:25 - 2023-12-01 18:17:09

  • Total downtime: 4 minutes 44 seconds
  • 🚩 API unavailable

2023-12-02 17:26:09 - 2023-12-02 17:32:09

  • Total downtime: 6 minutes
  • 🚩 API unavailable

2023-12-02 17:44:34 - 2023-12-02 17:47:34

  • Total downtime: 3 minutes
  • 🚩 API unavailable

2023-12-03 14:43:50 - 2023-12-03 14:52:25

  • Total downtime: 8 minutes 35 seconds
  • 🚩 API unavailable

2023-12-03 14:52:25 - 2023-12-03 14:56:10

  • Total downtime: 3 minutes 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 14:56:10 - 2023-12-03 15:05:16

  • Total downtime: 9 minutes 6 seconds
  • 🚩 API unavailable

2023-12-03 15:05:16 - 2023-12-03 15:06:46

  • Total downtime: 1 minute 30 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:06:46 - 2023-12-03 15:09:45

  • Total downtime: 2 minutes 59 seconds
  • 🚩 API unavailable

2023-12-03 15:09:45 - 2023-12-03 15:10:30

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:10:30 - 2023-12-03 15:11:58

  • Total downtime: 1 minute 28 seconds
  • 🚩 API unavailable

2023-12-03 15:11:58 - 2023-12-03 15:12:43

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:12:43 - 2023-12-03 15:14:48

  • Total downtime: 2 minutes 5 seconds
  • 🚩 API unavailable

2023-12-03 15:14:48 - 2023-12-03 15:15:33

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:15:33 - 2023-12-03 15:18:13

  • Total downtime: 2 minutes 40 seconds
  • 🚩 API unavailable

2023-12-03 15:18:13 - 2023-12-03 15:19:43

  • Total downtime: 1 minute 30 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:19:43 - 2023-12-03 15:23:06

  • Total downtime: 3 minutes 23 seconds
  • 🚩 API unavailable

2023-12-03 15:23:06 - 2023-12-03 15:25:21

  • Total downtime: 2 minutes 15 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:25:21 - 2023-12-03 15:28:11

  • Total downtime: 2 minutes 50 seconds
  • 🚩 API unavailable

2023-12-03 15:28:11 - 2023-12-03 15:28:56

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:28:56 - 2023-12-03 15:30:22

  • Total downtime: 1 minute 26 seconds
  • 🚩 API unavailable

2023-12-03 15:30:22 - 2023-12-03 15:32:37

  • Total downtime: 2 minutes 15 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:32:37 - 2023-12-03 15:34:04

  • Total downtime: 1 minute 27 seconds
  • 🚩 API unavailable

2023-12-03 15:34:04 - 2023-12-03 15:37:04

  • Total downtime: 3 minutes
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:37:04 - 2023-12-03 15:40:23

  • Total downtime: 3 minutes 19 seconds
  • 🚩 API unavailable

2023-12-03 15:40:23 - 2023-12-03 15:41:08

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:41:08 - 2023-12-03 15:42:57

  • Total downtime: 1 minute 49 seconds
  • 🚩 API unavailable

2023-12-03 15:42:57 - 2023-12-03 15:44:27

  • Total downtime: 1 minute 30 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:44:27 - 2023-12-03 15:47:15

  • Total downtime: 2 minutes 48 seconds
  • 🚩 API unavailable

2023-12-03 15:47:15 - 2023-12-03 15:50:45

  • Total downtime: 3 minutes 30 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 15:50:45 - 2023-12-03 15:56:35

  • Total downtime: 5 minutes 50 seconds
  • 🚩 API unavailable

2023-12-03 15:57:38 - 2023-12-03 16:08:42

  • Total downtime: 11 minutes 4 seconds
  • 🚩 API unavailable

2023-12-03 16:08:42 - 2023-12-03 16:10:57

  • Total downtime: 2 minutes 15 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 16:10:57 - 2023-12-03 16:16:50

  • Total downtime: 5 minutes 53 seconds
  • 🚩 API unavailable

2023-12-03 16:16:50 - 2023-12-03 16:17:35

  • Total downtime: 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 16:17:35 - 2023-12-03 16:22:57

  • Total downtime: 5 minutes 22 seconds
  • 🚩 API unavailable

2023-12-03 16:22:57 - 2023-12-03 16:26:42

  • Total downtime: 3 minutes 45 seconds
  • 🚩 API unavailable
  • 🌐 Website unavailable

2023-12-03 16:26:42 - 2023-12-03 16:27:39

  • Total downtime: 57 seconds
  • 🚩 API unavailable

2023-12-07 04:15:18 - 2023-12-07 04:16:03

  • Total downtime: 45 seconds
  • 🚩 API unavailable

2023-12-13 20:13:39 - 2023-12-13 20:14:24

  • Total downtime: 45 seconds
  • 🚩 API unavailable

🌂 The Past, The Present, The Future

Posted by NorthCrab on 15 August 2023 in English. Last updated on 17 August 2023.

Over the past few days, my perspective on OpenStreetMap (OSM) has undergone a seismic shift. For the longest time, I held OSM in the highest regard, viewing it as a beacon of transparency and people-centricity amidst a sea of profit-driven tech conglomerates.

Chapter 1: The Banner of Surprises

My view started to waver a couple of days ago when a new donation request banner popped up on the main OSM page. Though its sheer size and prominence were unsettling, the real issue lay in the near-invisible close button. When I voiced concerns on its ready-for-production state, opposition awaited me on the other end. Was this a shift in OSM’s priorities?

Chapter 2: The Amazon Alliance

Another revelation added fuel to the fire. Scrutinizing OSM’s 2022 and 2023 budgets, I stumbled upon a yearly expenditure of €24,000 directed to Amazon for S3 storage. This alliance with the tech giant seemed out of character for OSM. With an impressive arsenal of OSM-owned servers, the choice to rely on Amazon’s rented storage raised questions about OSM’s commitment to independence.

Chapter 3: Speaking My Mind

Emotions ran high as I took to the forums to articulate my frustrations. Admittedly, my initial approach was less than tactful, potentially causing offense. I since rephrased my statements, but my core concerns remained:

  1. OSM’s puzzling choice to rely on Amazon’s AWS.
  2. The opaque nature of information regarding S3 use within OSM.
  3. The seemingly wasteful storage of files on the expensive S3.

These revelations painted a rather concerning picture of OSM’s current trajectory.

I intentionally choose not to delve deeply into the now-publicized details concerning S3 dependence here. For a comprehensive understanding, I’d recommend referring to the original thread. Rehashing all the specifics here would be redundant.

Chapter 4: Lost in Dialogue

As conversations unfolded, a pattern began to emerge. Some comments seemed baseless, and when challenged, many simply turned a blind eye [1] [2] [3] [4]. This behavior wasn’t exclusive to the Amazon issue. It seemed to be a growing trend [1], even among those holding significant roles within OSM. A constructive dialogue relies on evidence-based arguments, and I found this lack of engagement disheartening.

Chapter 5: The Silenced Voices

My frustrations were further amplified when certain posts were taken out of context, resulting in the closure of the entire discussion thread. While I acknowledged my initial lack of professionalism, I took steps to rectify it. The decision to close the thread felt like a suppression of open dialogue, an ethos I believed OSM staunchly upheld.

Chapter 6: Reflection and Realization

These series of events forced me to confront a bitter truth: OSM’s priorities seem to have shifted. While my concerns about transparency were met with indifference, minor transgressions in tone drew significant attention. My principles are unwavering: I can only support an OSM that champions transparency, values its community, and remains open to feedback.

Endnote: Parting Ways, but Not in Spirit

My journey with the global OSM, in its current state, must sadly come to a halt. Yet, this doesn’t signify the end of my commitment to the open-source community. I will continue to support local OSM projects, like the OpenAEDMap, and will ensure that my AI-related projects [1] [2] remain accessible to all. While my vision of revolutionizing OSM mapping might be on hold, my passion for open source and my dream of a transparent, user-centric digital world remain undeterred.

Sunset

Photo by Alvesgaspar. See terms.

🚸 Mapowanie Przejść dla Pieszych za pomocą AI: Rewolucja

Posted by NorthCrab on 3 August 2023 in Polish (Polski). Last updated on 11 August 2023.

🗺️🦀 Witajcie, społeczność OpenStreetMap,

Dziś dzielę się z Wami moim najnowszym projektem, osm-yolo-crossings — nowym narzędziem wykorzystującym zaawansowaną technologię AI do samodzielnej detekcji i mapowania przejść dla pieszych (zebry) w OpenStreetMap. Po udanym imporcie budynków w Polsce za pomocą AI, przyszedł czas na poprawę bezpieczeństwa pieszych!

Baner aplikacji

Dzięki mocy detekcji obiektów YOLOv8, to narzędzie automatyzuje mapowanie brakujących przejść dla pieszych na naszych mapach. Z imponującą precyzją wynoszącą ponad 99,7%, jest w stanie zaimportować około 88% wszystkich wykrytych przejść. Pozostałe 12% jest odrzucane z powodu niskiego poziomu pewności. Dzięki inteligentnemu filtrowaniu, system ten jest niesamowicie wydajny. Na przykład, jest w stanie zmapować całą Polskę w ciągu zaledwie dwóch miesięcy, używając pojedynczego serwera bez karty graficznej. To AI pracuje mądrze, a nie ciężko!

Przykładowy zestaw zmian.

Przykład detekcji obiektów YOLOv8

Jedną z kluczowych funkcji tego narzędzia jest zdolność do odwoływania się do historycznych danych OSM, co pomaga unikać duplikatów i zapewnia poprawniejsze, dokładniejsze mapy. Raz usunięte przejście nie zostanie ponownie dodane przez okres kilku lat. Przygotowałem uproszczony diagram procesu, który lepiej wizualizuje cały proces (zobacz w pełnym ekranie):

Diagram procesu narzędzia

Moje projekty opierają się na przejrzystości i współpracy. W związku z tym, to narzędzie zostało opublikowane na GitHubie, jako w pełni otwarte - oraz darmowe - oprogramowanie! Proszę o wsparcie go gwiazdką ⭐️ lub/i darowizną 🙂.

🚸 AI-Powered Pedestrian Crossing Mapping: A Revolution

Posted by NorthCrab on 3 August 2023 in English. Last updated on 15 August 2023.

🗺️🦀 Hello OpenStreetMap community,

I am excited to share with you my latest invention, osm-yolo-crossings — a new tool that harnesses cutting-edge AI technology to autonomously detect and map pedestrian crossings (zebras) in OpenStreetMap. After the successful AI building import in Poland, it’s now time to expand and improve pedestrian safety!

Application banner

Leveraging the power of YOLOv8 object detection, this tool is designed to ensure that we no longer miss pedestrian crossings on our maps. With an impressive >99.7% precision rate, it’s able to import around 88% of all detected crossings. The tool discards the remaining 12% due to low confidence levels. Thanks to smart filtering, this system is incredibly efficient. For instance, it can map the entirety of Poland in just about two months using a single server without GPU. This is AI working smart, not hard!

See an example changeset.

YOLOv8 object detection example

One of the key features of this tool is its ability to cross-reference historical OSM data, which helps avoid duplicate entries and ensures cleaner, more accurate maps. Once removed crossing will not be re-added for a few years. I prepared a simplified workflow diagram, to better visualize the complete process (view in full screen):

Tool's workflow diagram

The core principles of my projects are transparency and teamwork. As such, this project is released on GitHub, as a fully open-source - and free (as in freedom) - software! Please support it with a star ⭐️ or/and donate here 🙂.

Rewolucja w imporcie budynków w Polsce za pomocą sztucznej inteligencji

Posted by NorthCrab on 24 July 2023 in Polish (Polski). Last updated on 15 August 2023.

🗺️🦀 Witam społeczność OpenStreetMap,

Z zadowoleniem ogłaszam mój najnowszy projekt, osm-budynki-orto-import - w pełni autonomiczne narzędzie do importu budynków, które obecnie funkcjonuje w Polsce. Jest to mój kolejny krok w kierunku uczynienia OpenStreetMap (OSM) bardziej dynamiczną i wydajną platformą.

Podgląd zbioru danych

Narzędzie to zostało zaprojektowane w celu uproszczenia i zwiększenia dokładności procesu importu budynków. System wykorzystuje oficjalne dane budynków w połączeniu ze zdjęciami ortofotograficznymi w celu weryfikacji poprawności danych przed ich zaimportowaniem.

Sercem tego projektu jest zaawansowany model wizji komputerowej, którego precyzja sięga 99,7%. Moim zdaniem dokładność ta przewyższa możliwości większości przeciętnych mapujących, zapewniając szybszy i bardziej niezawodny sposób mapowania struktur.

Wyjątkową cechą tego narzędzia jest jego zdolność do porównywania historycznych danych OSM, aby zapobiegać ponownemu importowaniu wcześniej usuniętych budynków. Funkcjonalność ta gwarantuje, że po usunięciu budynku z OSM, nie pojawi się on ponownie w przyszłych automatycznych importach.

Głównym celem tego projektu jest ograniczenie monotonnych i powtarzalnych zadań, co pozwoli mapującym skupić się na bardziej skomplikowanych i wartościowych zadaniach.

Z myślą o propagowaniu przejrzystości i wzajemnej współpracy, cały projekt został udostępniony w serwisie GitHub. Wraz z bazą kodu udostępniłem również zbiór danych kompatybilny z CVAT zawierający 6000 wpisów klasyfikacji. Zasób niezwykle przydatny dla innych programistów.

Jak zawsze, wasze opinie odgrywają kluczową rolę w ciągłym rozwoju i udoskonalaniu projektu. Jeśli uznasz ten projekt za przydatny i zechcesz wesprzeć jego dalszy rozwój i utrzymanie, informacje na temat darowizn znajdziesz na stronie https://monicz.dev/#support-my-work (strona w języku angielskim).

Daj gwiazdkę na GitHub ⭐️.

Revolutionizing building import in Poland with AI

Posted by NorthCrab on 24 July 2023 in English. Last updated on 15 August 2023.

🗺️🦀 Hello to the OpenStreetMap community,

I am happy to announce my latest project, osm-budynki-orto-import — a fully autonomous building import tool currently in operation in Poland. This is my next step towards making OpenStreetMap (OSM) a more dynamic and efficient platform.

Dataset preview

This tool is designed with the objective of making building import process simpler and more accurate. The system utilizes official building data in conjunction with ortophoto imagery to validate the accuracy of the data before importing it.

At the heart of this tool lies an advanced computer vision model, with a precision as high as 99.7%. This accuracy is, in my opinion, superior to the capabilities of most average mappers, providing a faster and more reliable way of mapping structures.

One unique feature of this tool is its ability to cross-reference historical OSM data to prevent re-importing of previously deleted buildings. This function ensures that once a building is removed from the OSM, it will not reappear in future automatic imports.

The primary goal of this project is to significantly cut down on monotonous and repetitive tasks, freeing up the mappers to focus on intricate and high-value mapping assignments.

In the spirit of promoting transparency and collaboration, the complete project has been open-sourced on GitHub. Along with the codebase, I have also shared a CVAT-compatible dataset with 6000 classification entries, a resource which could be highly beneficial to other developers.

As always, your feedback plays a crucial role in the ongoing development and refinement of the project. If you find value in this project and wish to support its ongoing development and maintenance, you can find my donation information at https://monicz.dev/#support-my-work.

Give it a star on GitHub ⭐️.

OSM Relatify: Prosty transport publiczny

Posted by NorthCrab on 18 June 2023 in Polish (Polski). Last updated on 11 August 2023.

Witaj społeczności OpenStreetMap,

Po publikacji osm-revert, pragnę podzielić się szczegółami na temat mojego nowego projektu, OSM Relatify. Narzędzie to ma na celu uproszczenie procesu edycji relacji transportu publicznego w OpenStreetMap (OSM).

Podgląd aplikacji

OSM Relatify został zaprojektowany tak, aby mapowanie transportu publicznego było bardziej dostępne i wydajne. Chociaż obecnie skupia się na relacjach autobusowych, wizja OSM Relatify obejmuje szerszy zakres zadań mapowania transportu publicznego.

Jedną z kluczowych cech OSM Relatify jest inteligentna logika wyznaczania tras. Automatycznie tworzy poprawne relacje autobusowe z podanych przystanków i dróg, biorąc pod uwagę takie czynniki jak ulice jednokierunkowe i ronda. To znacznie skraca czas i obniża trudność związaną z zarządzaniem relacjami autobusowymi, czyniąc to narzędzie cennym zasobem zarówno dla nowych, jak i doświadczonych maperów.

Przyszłe aktualizacje mają na celu rozszerzenie możliwości OSM Relatify, w tym obsługę dodatkowych typów transportu publicznego i ulepszone możliwości edycji. Więcej szczegółów na temat planowanych funkcjonalności można znaleźć tutaj. Głównym celem jest uczynienie z OSM Relatify wszechstronnego rozwiązania do mapowania transportu publicznego na OpenStreetMap.

Już dziś można korzystać z OSM Relatify pod adresem https://relatify.monicz.dev.

Krótki wideo-poradnik: https://files.monicz.dev/osm/osm-relatify-poradnik.mp4.

Zachęcam wszystkich do zapoznania się z OSM Relatify i podzielenia się swoimi doświadczeniami. Wasze opinie mają kluczowe znaczenie dla udoskonalenia narzędzia i kierowania jego dalszym rozwojem. Jeśli uznasz ten projekt za przydatny i zechcesz wesprzeć jego dalszy rozwój i utrzymanie, informacje na temat darowizn znajdziesz na stronie https://monicz.dev/#support-my-work (strona w języku angielskim).

Daj gwiazdkę na GitHub ⭐️.

OSM Relatify: OpenStreetMap public transport made easy

Posted by NorthCrab on 18 June 2023 in English. Last updated on 11 August 2023.

Greetings OpenStreetMap community,

Following the release of osm-revert, I’m here to share details about my new project, OSM Relatify. This tool is designed to simplify the process of editing public transport relations within OpenStreetMap (OSM).

Application preview

OSM Relatify is built with the goal of making public transport mapping more accessible and efficient. While it currently focuses on bus relations, the vision for OSM Relatify is to encompass a broader range of public transport mapping tasks.

One of the key features of OSM Relatify is its smart routing logic. It automatically constructs bus routes from given bus stops and ways, taking into account factors like one-way streets and roundabouts. This significantly reduces the time and complexity involved in managing bus relations, making the tool a valuable asset for both new mappers and seasoned contributors.

Future updates aim to expand the capabilities of OSM Relatify, including support for additional public transport types and enhanced editing features. You can find more details about the planned features here. The goal is to make OSM Relatify a comprehensive solution for public transport mapping on OpenStreetMap.

You can access OSM Relatify today at https://relatify.monicz.dev.

I encourage everyone to explore OSM Relatify and share your experiences. Your feedback is crucial in refining the tool and guiding its development. If you find value in this project and wish to support its ongoing development and maintenance, you can find my donation information at https://monicz.dev/#support-my-work.

Give it a star on GitHub ⭐️.