OpenStreetMap
It's not because you have accurate data that you have to upload all of them in OSM
Posted by Pieren on 23 February 2015 in English.After reading this blog post: https://blog.amigocloud.com/sub-meter-data-collection-with-an-iphone-into-openstreetmap/
I checked one of the created ways done after a survey with centimeter accuracy: http://www.openstreetmap.org/way/329799637
Unsurprisingly, all surveyed nodes have been imported in OSM. But OSM doesn’t need a node every 30 centimeters, even not every meter if the angle is null.
Someone to contact Ragi Burhum, AmigoCloud CEO, to explain that JOSM is also providing a “simplify way” function ?
Discussion
Comment from Sanderd17 on 23 February 2015 at 11:44
Ugh, it’s even worse than that, he even tagged all nodes: http://www.openstreetmap.org/node/3367453367
Since he wants high-precision data, the default settings for simplify-way are a bit too coarse. So someone who knows to enter finer settings, that would be very nice.
And of course, null-angle or almost null-angle nodes should be deleted, but sharp turns can still benefit from a node every 30 cm. But this is something algorithms (like the one in simplify-way) do rather well.
Comment from Sanderd17 on 23 February 2015 at 11:45
Also, it looks like most of the features he added were already present in the data, but just not tagged with landuse=*, so they weren’t rendered.
Comment from SK53 on 23 February 2015 at 13:23
And we have really accurate position of the pins on the golf course, which will of course get moved every few days to keep green wear constant.
Comment from Richard on 23 February 2015 at 13:37
Given JOSM users’ proclivity to demand that every single possible error is trapped by iD (cf https://github.com/openstreetmap/iD/issues/1461) and P2, I think we should quit ragging on Ragi, and instead JOSM should refuse to upload unnecessarily detailed ways like this. It’s the responsible thing to do. ;)
Comment from Rovastar on 23 February 2015 at 14:22
Although there are some problems with all the nodes being tagged….. I see no problem at all is mapping to this level of detail. In fact, I don’t even think it is that excessive. A few hundred nodes for a large non complex shape of grass? Add a few thousand nodes maybe some from the admin team can say this was pushing OSM database over the edge as now it is about 2.75 billion nodes + a few thousand.
Is it really because some are sort of import……
Why again are we potentially putting off people contributing? Are we really telling people what they can and cannot map here and in what detail? What if someone said you should not map tree types or lock gates with as much detail as some do?
I am not sure where anyone is getting the information that the information was already in OSM. Sure some was added the same day from a different user and I imagine that it was part of the same project given the timing.
Comment from Vincent de Phily on 23 February 2015 at 14:48
Commented on the changeset.
Comment from Rovastar on 23 February 2015 at 15:01
Ok I have tidied up that golf course. Removed the tags on the nodes and deleted the additional dup ways.
Oh to explain JOSMs Simplified Way well … IMHO useless for any sort of detailed mapping. You lose a ridiculous amount of detail. Maybe some of you have never used it…..
Comment from Vincent de Phily on 23 February 2015 at 15:21
edit -> preferences -> advanced -> simplify -> 0.3 or nearby is much more reasonable than the default 3.0.
Comment from Rovastar on 23 February 2015 at 15:26
So that would be every 30 cm which is what he had the data anyway apparently. So it would be the same as the import therefore not quite sure of the point of it. Why add more barriers to entry for OSM? Is OSm for old timer dinosaurs only? shrug The real issue here is some don’t want detailed mapping.
Comment from Sanderd17 on 23 February 2015 at 15:54
The simplify way algorithm uses (most likely) the Douglas-Peuker algorithm. That algorithm is clever enough to not just evenly space nodes along the line, but it removes the nodes that influence the shape as little as possible. This is done by comparing how big the area difference is when you delete or keep a node.
As such, the end results are that it deletes the nodes that don’t change the shape, but it keeps the nodes close enough in a sharp turn. Of course, finding the right precision value is a bit of trial-and-error, as it differs per case.
Comment from Rovastar on 23 February 2015 at 16:32
Still there is no need to use this. Over time we will get more detailed information in OSM. (I know some of you hate this idea but it is true) Moaning about users uploading detailed information really isn’t needed or I would argue desired. What harm does it really do? This really isn’t that detailed. All simplifying algorithms lose information. I do not see any need to lose this information. The only reason is to put a burden on the editor.
Comment from Richard on 23 February 2015 at 17:24
“All simplifying algorithms lose information”
ORLY?
If you have a polyline [1,0],[2,0],[3,0],[4,0],[5,0],[6,0], you can simplify that without any loss of geometry to [1,0],[6,0].
If you have a polyline [1,0],[2,0.0000000001],[3,-0.0000000001],[4,0],[5,0.0000000001],[6,0], you can simplify that (units permitting) with minimal loss of geometry to [1,0],[6,0].
“Minimal” is in the eye of the beholder, but given the width of the features we survey in polyline form in OSM and the positional accuracy of GPS units, it’s clearly >0.
Still, as both Vincent and I are only editor coders, what the hell do we know about how these things actually work, eh, Rovastar?
Comment from Rovastar on 23 February 2015 at 21:20
Are you really saying that I cannot come up with a set of coordinates that lose data? Please at least make any challenges that require me to use some of my intelligence when throwing down the gauntlet.
Comment from Rovastar on 23 February 2015 at 21:20
Are you really saying that I cannot come up with a set of coordinates that lose data? Please at least make any challenges that require me to use some of my intelligence when throwing down the gauntlet.
Comment from Richard on 23 February 2015 at 23:50
Splendid. Ok. Here’s your challenge.
Write some polyline simplification code that reduces [1,0],[2,0],[3,0],[4,0],[5,0],[6,0] to [1,0],[6,0], yet is completely lossless.
Any programming language of your choosing. It’s very doable.
Second task, since you ask for “challenges” (plural): adapt that simplification algorithm to permit minimal loss of geometry within the bounds of feature width and GPS accuracy. You may choose to implement an adjustable threshold as part of this.
Comment from Sanderd17 on 24 February 2015 at 08:24
There are different factors involved when it comes to precision. I isn’t enough that your sensor is precise.
First of all, it matters how precise you hold the device. If you have a 2cm accurate sensor, it’s rather easy for the human hand to wiggle 2 cm, and thus cause an extra precision node. In fact, I see that in this data, some points are even there when there’s no wiggle at all (or something lower than we’re able to represent in OSM).
Then you also have the changing nature of things. When grass lives in a good environment, it will take several cm of the sandbox per year. If the grass doesn’t grow well there, more ground will turn into sand over the years. So if you map something natural up to 5cm precise, it will only be true for a year or two. It’s even worse for water, which may change on a daily basis.
Thus, to answer your question: I don’t think OSM will ever become as precise as the data mapped there. There’s a natural limit to our precision. Any data we add beyond that natural precision will just give us the fake idea of improving the data (and it puts a burden on the data customers).
Comment from Pieren on 24 February 2015 at 09:18
If you don’t want to loose all your accurate surveyed points, they are still in version 1 of the way & nodes. Or another solution would be to upload your surveyed points as GPX files. These trace files are preserved in OSM and nobody will modify them.
Comment from SomeoneElse on 25 February 2015 at 19:18
Given the plate tectonics that’s active in the area
http://www.nhm.org/site/sites/default/files/mineral/san_andreas_fault-480px.jpg
I just hope that he’s going to keep it up to date in the future :)
Comment from philsevansdesigh on 26 February 2015 at 14:32
I will be seeing Ragi today and am sending him this link.
Comment from Ragi Yaser Burhum on 26 February 2015 at 15:25
Hi all,
Sorry my edits are a bit controversial. I’d love to make it less controversial next time, so some guidelines for how detailed it should be would be appreciated.
See, when we go test our software, we are literally walking several miles a day with an end result of beautiful data. Nevertheless, it is sitting in our computers, and after we run our tests, it just stays there. Trails, golf courses, parks, etc. I wanted to encourage the rest of folks at AmigoCloud to share their data to OSM so other people can benefit from it.
Riviera Country Club was extra interesting to me because before last week it was effectively empty in OSM and I wanted it to be better than the Google Maps version.
Of course, we I can post process the data in our pipeline. I can easily run st_simplifipreservetopology with a specific tolerance X and do some other PostGIS magic to space out the vertices by an X threshold before export. I can skip JOSM’s simplification functionality and tweak the results exactly as desired.
I’d just love to know what that X is. 1m? 5m? 10m? Whatever is good for cartographic output? Something a bit higher? The beauty of OSM, as we all know, is that it is obviously way more than cartographic output by now. Any particular link in the OSM wiki that can help me with choosing these thresholds?
I walked a few trails this weekend and I was thinking about uploading that data, but I also want to be cautious since I don’t want to piss anybody off. Some people have explicitly asked me for end to end tutorial on how they can use their iPhone / Android to do exactly this, but I don’t want to publish a workflow that is not sanctioned and will cause headaches for people in the community.
Any concerns or questions, feel free to contact me directly or here. My contact is my first.name@last.name.com
Comment from Harry Wood on 9 March 2015 at 12:33
Interesting discussion. We have some brief guidelines on simplifying for imports, but this is mostly for larger scale imports. We’ve had problems with unsimplified shapefile imports where tremendous number of nodes were plonked in, despite relative inaccuracy of the data. In this case it clearly is very accurate survey-based data. In any case though, to put “too much” data in, is to screw up the data for anyone making normal use/normal edits. But how much is too much.
If we were to run an analysis of the golf bunkers worldwide, look at the nodes-per-metre, draw a histogram of that. I wonder what detail level mappers typically go up to. It’s the kind of thing people do map in quite a lot of detail. And they’re always curvy shapes of course. I guess finding a “high end” 90th percentile value from the existing database would have been a good way to decide on what to simplify this data down to (too late now, but could be fun thing to calculate)
I suppose the principle I’m suggesting here, is that we should encourage a detail level which is high, but not so high that other dedicated mappers are not attaining the same level elsewhere in the world using normal OpenStreetMap contributing techniques.