NorthCrab's Diary

OpenStreetMap NextGen Development Diary #2

Posted by NorthCrab on 24 March 2024 in English.

Welcome to my second OpenStreetMap NextGen development diary. I am ready to highlight this week’s progress — and there was a lot 👷!

You can subscribe to my diary updates on RSS here:
https://www.openstreetmap.org/user/NorthCrab/diary/rss

Finished Settings Page

Last week I showcased initial progress on the new settings page for OpenStreetMap. This week, I’ve finished it. Here’s what’s new. Keep in mind that if you believe some features should be done differently, there will be a period of public testing where I’ll collect this kind of feedback 😉.

New settings page screenshot

Click here to view it in full screen.

Aside from the improved layout, I’ve introduced two notable changes. First, the application language selection has been redesigned; it’s now a simple dropdown selector. I believe the current system (where you input locale names by hand) is difficult for less technical people to understand, and I want to make the website more accessible to everyone!

New privacy controls screenshot

Secondly, OpenStreetMap finally gains account privacy controls. Want to opt out of activity tracking but still send crash reports? No problem! Today’s opt-out implementation relies solely on the Do-Not-Track browser setting. Even when opted out, you’ll still connect to analytics servers (for some reason). The NextGen codebase supports Do-Not-Track headers, the new Global Privacy Control, and does not connect to analytics servers when opted out. Account configuration takes precedence over browser configuration.

Seamless 180th Meridian Handling

This week I’ve invested time in smoothing out handling of the 180th meridian in both the frontend and backend. When browsing the map around the 180th meridian, there are no more visual glitches when crossing it. Additionally, the map and notes layers will no longer throw exceptions around it. The new position handling script is also significantly smaller compared to the existing Ruby codebase. The backend API now seamlessly handles 180th meridian requests.

The current API integration when requesting an area from 160° to 200°:

Send 1st request from 160° to 180°
Send 2nd request from -180° to -160°
Merge results

The new integration:

Send 1 request from 160° to 200° (or from -200° to -160°)

Splitting, wrapping, and merging are done under the hood by the API, significantly reducing code complexity in many consumer applications. Less complexity means fewer bugs and easier integration with OpenStreetMap.

System Apps and Test Users

As I actively work towards creating the best developer experience, this week I focused on two new features: system apps and test users.

System apps are OAuth2 applications managed by the application itself, not by users. Currently, there are two system apps: iD and Rapid — the supported web editors.

INFO:     | 2024-03-23 23:43:45 | Registering system app 'iD'
INFO:     | 2024-03-23 23:43:45 | Registering system app 'Rapid'

In the current Ruby codebase, you need to follow all these steps just to get your own iD instance running. OpenStreetMap-NextGen does this automatically on application launch for you — you’re welcome 🙂.

Debug login panel screenshot

Test users are just that: users for testing. I like when applications provide developers with multiple users ready to be used, rather than requiring them to go through the registration process. Contributors can now log in to any of these users with a single button press!

Progress on Notes and Changesets API 0.6

Changeset read API response screenshot

I’m actively working on the migration of existing API features. This week, I successfully tested CRUD (Create, Read, Update, Delete) operations on notes and changesets. There were a few issues that needed correction, but nothing notable.

And More…

All of my work is publicly accessible on this GitHub repository. While my development diaries focus on selected highlights, there are many more details to read through. You can browse the commits history.

Project Sponsors 🦀

I want to take a moment to personally thank this week’s project patrons. It was super motivating to see new supporters! I really appreciate all of you! 🫰

Currently, my work is sponsored by 5 patrons on Liberapay, including three private donors. I have one public donor on GitHub Sponsors and have also received a private donation via PayPal.

~1847430, gileri, 快乐的老鼠宝宝/LaoshuBaby

If you’d like to join my development sponsors, you can find me on Liberapay or GitHub Sponsors. Currently, all contributions go directly towards the development of OpenStreetMap NextGen 🚀.

Disclaimer

Please note that this project is not affiliated with the OpenStreetMap Foundation. It’s the result of my voluntary work and personal choices.

Discussion

Comment from mmd on 24 March 2024 at 10:32

Thank you for the update. As before, I did some local testing here on my box, to better understand how user interactions look like (screenshots are not that good at that).

I think the Profile picture part looks cleaner and easier to understand, and I can see some benefits in splitting up the current “My Settings” page in smaller pieces. I’ve noticed some small details like restricting the permitted file types to images, which helps to improve user experience at essentially zero development effort.

Of course, there are still some rough edges, such as this interesting Internal Server Error: “PIL.Image.DecompressionBombError: Image size (204615540 pixels) exceeds limit of 178956970 pixels, could be decompression bomb DOS attack.”, but that’s ok at this point in time.

I also managed to create some notes via the API 0.6 endpoint, but not on the map itself. Here’s what I could capture in the browser console:

“POST /api/0.6/notes HTTP/1.1” 422 Unprocessable Entity –> input is null

It looks like the payload is lacking some relevant details here. I’m pretty sure I’m missing some pipeline step here. Any sort of documentation would be much appreciated.

-----------------------------11619737919638056413629656613
Content-Disposition: form-data; name="lon"


-----------------------------11619737919638056413629656613
Content-Disposition: form-data; name="lat"


-----------------------------11619737919638056413629656613--

Also the single click Debug Login didn’t work here, it just doesn’t do anything, not even a message on the browser console. Manually logging in with “user1” and password “openstreetmap” did the trick, though.

I’ve also noticed Cython-build being a bit unstable at the moment, but didn’t spend time to investigate it. I will try this again in a few weeks maybe.

By the way, setting up users on Rails and registering apps could be fully scripted. Registering a new OAuth2 application is essentially a one-liner:

Doorkeeper::Application.create(name: "Local iD", owner_type: "User", owner_id: User.find_by(:display_name => "mmd2").id, redirect_uri: "http://localhost:3000", scopes: ["read_prefs write_prefs write_api read_gpx write_gpx write_notes"])

Results could then be used to automatically update settings.local.yml. My guess is that all preparation steps, including user creation and app registration could be done in about 20 LOC without any hidden magic involved. Nobody bothered enough to do it so far, still it’s fairly easy to do.

Comment from NorthCrab on 24 March 2024 at 12:54

As before, you are testing parts of the application that are under development. I really can’t help you with troubleshooting work-in-progress software.

Here’s a resource to help you: the Systems Development Life Cycle (SDLC). This concept is similar to how OpenStreetMap-NextGen is developed and might clear things up for you.

Comment from Woazboat on 1 April 2024 at 16:32

I am fairly certain that mmd is aware of software development lifecycles and that his comments were intended as feedback and not as requests for assistance with troubleshooting.

Comment from NorthCrab on 1 April 2024 at 16:40

@Woazboat, Hey! I originally also thought so, but since it happened 2nd time, I just wanted to clarify it further. Honestly it’s not a big deal, people are busy and sometimes forget. 🙂👍

(If the feedback is: some parts of the website don’t work properly; then yes, I know. If it were otherwise, the website would already be open for public access.).