OpenStreetMap

Divide and map. Now.

Posted by qeef on 1 January 2020 in English (English)

What?

There is a place to be mapped. The area is usually too large for one guy. Involving a team of mappers is, therefore, evident. And there is an excellent approach to solving big problems called divide and conquer. Therefore Divide and map. Now. – the damn project.

The goal is to split a large area to squares a human can map. Then, lock a square, work on it, and mark ready for review. Lock again, review the square, and mark as done. The project is here to avoid overwriting work between mappers.

If this sounds familiar, you probably know the HOT’s Tasking Manager. (HOT stands for the Humanitarian OpenStreetMap Team.)

Why?

There are issues with the HOT Tasking Manager. Mainly performance issues. Then, slowly forgetting about essential functionality. Unwanted updates as the necessity of email address (I really hate this!). And only one repository for the whole project.

These are technical issues, however. What about community issues? Why Google disk when there is OpenStreetMap wiki? Why Slack where you must be invited and logged to (just!) see the history? Why all these sign-up Google forms to contribute? I will try to guess an answer – because HOT is a company (even non-profit) and does not know how to be an open community.

Before you ask “If you are so smart, why you do not contribute to Tasking Manager?”, I tried 1, 2, 3, 4. However, there is no real discussion about the future of the Tasking Manager. The future is already decided; openness means that you can see the source code and fix bugs. Nothing more.

Who?

Let’s talk about the potential damn community. It starts with mappers. I will split mappers to novices, experts, reviewers (or validators), and mentors. Mappers are the most critical and the largest group, so the tool they use should be efficient, stable, and comfortable.

Novice mappers perhaps have different needs than experts or reviewers. Mentors, on the other side, probably need to be as close as possible to novices to help them. All of them use the tool differently.

The next are guys with a place to be mapped. Their goal is to point to an area, provide the necessary information, and let mappers do their job – completely different tool usage.

The last (but not least) part of the damn community are developers. They should communicate with the whole community and deliver the tool.

How?

I will try to explain how to create the tool that satisfies Who, do What, and avoids Why. The damn project tends to be proof of concept.

Novice mappers, along with mentors, probably use a web browser. Make a web application then. That could include web chat, maybe? However, remember the primary goal! Let novice mappers map and make mistakes. Let mentors reach the novices as quickly as possible and mark their mistakes efficiently.

Expert users use JOSM. JOSM plugin is the only possible solution here. Maybe other users use a different specialized tool? Make the plugin for that tool, too.

Another web application can help guys publishing places to map. But not necessarily a web app. What about some script? Maybe with some support of so popular machine-learning? Why creating areas to map couldn’t be done by computer? And then just confirmed by some (perhaps another) tool?

You need to serve all these clients with a centralized data store. That’s the backend server.

For each client, create a communication channel between users and developers, namely between novice users, mentors, and mapping web app developers. Then between JOSM users and JOSM plugin developers. Finally, between all the clients’ developers and backend server developers. Everyone involved should also be able to communicate with everyone else.

Always support new clients. One tool just can’t fit all mappers.

Damn project

The damn project is composed of multiple repositories. The central application is damn_server 5 source – backend server written in python using the async FastAPI framework.

There are damn_client (web application, 6 source) and JOSM damn_plugin 7 as main damn clients. There is damn_manager (web application, 8 source) for creating areas to map.

All the repositories are in damn-project group on Gitlab. There is damn-project community chat on Gitter. When needed, new channels on Gitter will be created.

The damn project is proof of concept. It suffers from performance issues because the damn server runs on basic DigitalOcean droplet with 1GB RAM, 1vCPU, 1TB transfer, 25GB SSD disk. There are, of course, more issues.

Comment from EditConscript on 9 January 2020 at 13:21

メンターとエキスパートの役割に自分が合っていると思う コミュニティに貢献する準備ができている I find myself to fit in the role of Mentor and Expert (use of JOSM) as of this point, ready to contribute to the damn community when it’s damn time the damn developers got the damn servers going

Comment from qeef on 9 January 2020 at 13:37

Thanks.

The list of running instances is at damn project website. However, just the “developer” instance of the server is only running currently. By “developer” I mean cheap, low-performance server.

I would be really happy to test the damn project with some local community on self-hosted premises, however.

Comment from tordans on 9 January 2020 at 16:51

Hi qeef, just in case … do you know https://mappingnorthkorea.com/map / https://github.com/MRVDH/mapping-north-korea which is also a kind of Tasking Manager Tool, but just for north korea.

Also a quick feedback: Personally, I am not a fan of the name. It sound negative, which is a bad starting point for something good IMO :-)

Best, Tobias

Comment from qeef on 10 January 2020 at 23:17

Hi tordans. Thanks for the link. I think I saw it somewhere but couldn’t use it as it is too specific. And I don’t know node.js.

I feel like the most problematic thing is to name things. Also, someone already pointed out that the name is problematic in a religion point of view. I will think about it. Until then, please consider the project damn awesome, thanks :)

Comment from Artemis64 on 20 January 2020 at 09:27

Hi tordans and qeef, Creator of https://mappingnorthkorea.com here. I’ve actually been working on a newer version of mapping north korea that is not limited to North Korea. You will be able to create your own iterations, sectors etc anywhere in the world. I originally set it up as an experiment, but ran into scaling issues with node.js and mongodb. The new version will have a backend in .NET Core instead of node.js. I’ll make sure to send it to the Weekly OSM blog once it’s live. 👍

Comment from qeef on 21 January 2020 at 08:53

Hi @Artemis64, that sounds good! When it’s ready, it would be awesome to benchmark somehow your new version, damn server, and HOT Tasking Manager. I would be interesting :)

Login to leave a comment