I run load testing with locust.io on the damn server, and I think it’s usable for mapathon with 100 mappers.
The damn server is part of the Divide and map. Now. – the damn project I have published at the beginning of the year. And I am still interested in the performance. I was wondering if I can simulate a mapathon event.
I run the damn server on $5 per month VPS of do.co. The server runs Debian 10.2 x64, 1 vCPU, and 1GB / 25GB Disk. There is nothing special about the deployement I described in the last diary.
For load testing, I prepared the area. I uploaded a big area (JOSM measurement plugin says it’s 42 852 square km) with the damn manager, that split it to 654 squares.
Then, I simulated a mapathon in the locust file. The “participating” test mapper can choose from:
Get the area and all its commits. (In fact, all commits wouldn’t be used in a real client twice, since since for an area is implemented.)
Request a random square for mapping. If the square is available, work on it for between 1 and 5 seconds and then stop with need more mapping (30 %), split the square (10 %), or ready for review (60 %). (In the brackets, there is a probability of stop reason used.)
Request a random square for review. If the square is available, work on it for between 1 and 5 seconds and then stop with need more mapping (30 %), please, review again (10 %), or done (60 %).
When (1), (2), or (3) finishes, wait for between 1 and 2 seconds and choose again. (2) and (3) are chosen five times more often than (1).
I tested for ten minutes. There were ten new mappers per second until a maximum of 100 mappers.
The total request count was 7 310. And there was no error!
I tested for ten minutes again. There were 50 new mappers per second until a maximum of 500 mappers.
The total request count was 8 499. There were 2.35 failures per second.
I don’t like failures, so what were these errors? Connection refused, connection closed without response, and not found url. I think that I just overloaded the $5 per month server.
First, I don’t think that the average response time of 6 seconds is fantastic. However, I don’t believe that mappers at mapathon work on a square for 1 to 5 seconds, either.
The damn server currently online is not ready for big loads. It looks like one vCPU can’t handle 500 mappers. However, the damn project aims at local communities. Moreover, I have no ambitions to run a production server on my own.
During load testing, I found some database access bugs and fixed them.
The last thing – I would like to see some similar (comparable) load testing of HOT Tasking Manager. I gave up on trying. I made the Tasking Manager run but had problems with login. And I have no idea how to run queries from 100 test mappers anyway. (No, I don’t think that 1000 times GET / is comparable load testing.)