Here is some background:
The WGS84 Coordinate Reference System uses an ITRF datum, locked with the Earth’s rotation.
The tectonic plates we’re living on, are moving, up to 70 to 80 mm/year for the Australian plate, 20 to 25 mm/year for the Eurasian one.
The tectonic plates have internal distortions (we do ignore them here)
If you have a very accurate GNSS receiver which shows you WGS84 coordinates, and place it on a survey mark, because this mark moves, you won’t get the same coordinates year after year. If you surveyed a mark in Australia 13 years ago, today’s coordinates are 1 meter off. So, accurate WGS84 coordinates are not accurate at all if you don’t know the epoch, just as a time value is unuseful if you don’t know the time zone.
Working with dynamic coordinates would be a nightmare for the surveyors. To make their life easier they use a datum associated with their continental plate, locked at a point in time. And for a few decades now, many countries have chosen an ITRF datum and “locked” it at a specific date. With such a datum, the coordinates of survey marks remain the same over the years.
USA is using NAD83
Australian is using GDA94 (ITRS epoch 1994.0)
Europe is using ETRS89 (ITRS epoch 1993.0)
France (in europe) is using RGF93, a more accurate ETRS89 realization. We consider that RGF93 = ETRS89.
At first, don’t forget that a WGS84 datum without epoch can’t be accurate at a submeter level.
Secondly, if you use a standard GNSS receiver, it doesn’t matter, as your accuracy is too low (3 to 5 meters). If you use RTK, like I do, your datum is the one used by the base station, and it’s usually plated-fixed.
Thirdly, if your aerial imagery has a low resolution like 50cm/pixel, it will be very difficult to see small details.
However, we have more and more high-res imagery. In some places, you can see some survey marks and check the coordinates. Let’s go to Australia, next to the Mount Stromlo Observatory. This node is a GNSS antenna, part of the IGS network.
You can go to the Geoscience Australia website to find the GDA94 coordinates of this antenna and enter them in JOSM with the “Lat Lon Tool”.
Then, if you switch between the 3 good imagery (ACTmapi Imagery 2017, ACTmapi Imagery 2018, LPI Nsw Imagery), you will see that the antenna, the node with the GDA94 coordinates, and the 3 imagery layers, are aligned:
Ok, but what are the WGS84/IRTF coordinates for this antenna? Luckily, you can download the data from this GNSS station and process them with a PPP software. The Australian online AUSPOS service gives you the PPP results with GDA94, GDA2020 and IRTF (@epoch data) coordinates. Here are the locations, more than 1.5m to north-east:
The conclusion is that these imageries are not using an IRTF datum, but the local one: GDA94. Therefore any OSM data created using these layers are not in WGS84.
Now let’s go to France, to this node : https://www.openstreetmap.org/node/670517301
This is a survey mark with coordinates using RGF93 datum and the accuracy is better than 10cm.
The alignment is good. We can say that this imagery layer use RGF93, not WGS84, or we would have seen a 70 to 80 cm offset. I’ve checked check some other survey marks in the country…all aerial imagery are RGF93/ETRS89.
This is our first problem: Some OSM data are not using WGS84 coordinates ; the offset could be as high as 2 meters.
Our second issue is: How do we guarantee a good accuracy over the years?
Here are a couple of options.
This is simple, but we don’t want too many datums: perhaps we should select a single datum for each tectonic plate.
Aerial imageries with the same local datum remain aligned.
We should inform data consumers, so that they can decide to convert the coordinates to WGS84 if they want to.
We will need to move a lot of data when a new datum replaces an old one (more on that later)
We must convert data with local datum (Australia, France, …) to WGS84. This is not an easy task, but this is feasible with a plate motion model.
The date a node is created in OSM doesn’t inform about the original epoch.
Should we keep the old aerial imageries to their original location, or transform them to today’s location? It could be difficult for the editors (Josm, ID, …) to do this if there is a translation and a rotation.
When you load data in an editor, it should check all nodes coordinates, find the epochs, and dynamicaly move them to the actual localization with an Helmert transformation. It would be the same problem if you want to reuse the data. Another way is to create a bot which moves all the nodes in the database regularly (twice a year?).
Prone to errors
Whatever the decision is, we should check what is the real datum of all high-res imagery. To do so we need more reference points that are visible on one or more imagery. These could be a survey mark on the ground with public and accurate coordinates, or a GNSS antenna on or near the ground (we can’t use an object on a roof) with observation data freely available. I’ve already started a list on the wiki, feel free to add more reference points. A new tag on these nodes would be useful.
Let’s go back to Australia: The Auspos service provides GNSS coordinates with 3 datums : GDA94, WGS84 and GDA2020. What is the last one? As you know, the Australian plate is moving pretty fast. In 2020 the offset between GDA94 and the real location will be more than 1.8 meters. They decided they will use a new datum from 2020: GDA2020
It means that new high-res aerial imagery in this country will have a big offset relative to the old imagery.
They will use a dynamic reference frame too, but I’m not sure it will be used by the GIS world: ATRF
Another country will change its datum : the U.S.A., with the plate-fixed NATRF2022, in replacement of NAD83.
On Earth, everything is moving, up to 80mm/year.
A fix point on a tectonic plate will see its WGS84 coordinates changing over the years, but the same point have static coordinates with a plate-fixed local datum.
WGS84 coordinates without epoch can’t be accurate.
Contrary to what is stated, not all OSM data are using a WGS84 datum. In some country, the offset could be as high as 1.8 meters.
Some aerial imagery are not using a WGS84 coordinate reference system, but a local datum.
What should OSM do? Keep the dynamic WGS84 coordinates, or use some local datums as the surveyor’s world do?
Thank you to naomap for his help with the translation.
Comment from GetlostMaps on 17 July 2019 at 22:14
Great write up thanks.
Comment from Warin61 on 17 July 2019 at 22:35
Until GPSes and phones start using plate tectonic references there will be no consumer demand for anything other than WGS84/WGS84 like references.
And, as you point out, the differences are too small for consumers to recognize above the error of their equipment.
So technically correct, but of no use to the majority of users.
Eventually I would like to see OSM move to plate tectonic references. But it will not happen soon.
Comment from Harry Wood on 18 July 2019 at 00:23
Past April fools jokes come back to haunt us: https://blog.openstreetmap.org/2017/03/31/osm-plate-tectonics/#comment-115756 .
As I mentioned at the time, plate tectonics are big concern for OpenStreetMap in Australia, where by “big” I mean …not as big as many other concerns :-)
I did find this post interesting to be fair, but… it’s still not a real problem.
Comment from Stalfur on 18 July 2019 at 01:20
Iceland is moving west, east and south - parts of it going each way.
Comment from Cameron Shorter on 18 July 2019 at 01:39
Some great research you present here StephaneP. Thanks.
Us Australians are wrestling with upcoming misaligned maps in web-mapping because we have:
* GDA2020 is defined as having moved 1.8 from GDA94
* GDA94 is defined the same as WGS84 (which was the case in 1994)
* GDA2020 is defined as the same as WGS84 (which will be the case in 2020)
* Web-mapping use cases incorrectly assumes WGS84 hasn’t changed.
We are discussing on the proj email list: https://lists.osgeo.org/pipermail/proj/2019-July/thread.html thread: [PROJ] Static/Dynamic Webmapping Problem version 2.0
I’ve been drafting a positional paper - covering many of the same concepts: https://docs.google.com/document/d/15uBX2qICODRkiHXeksT0zEy6TuysanIxuc4anV0o5F0
I’m expecting that our solution will involve creating an EPSG code for a static realisation of WGS84 which aligns with existing proj transformations within each region and epochs.
Comment from aharvey on 18 July 2019 at 06:30
Great write up StephaneP, glad to see this getting more attention in the OSM world, indeed it’s an important problem.
Which solution to quite simply store not just lat,lon, but also time, as in time when those lat/lon coordinates where recorded from the GPS, or time epoch when the imagery was captured?
Then either client side or server side, all those coordinates from different datums could be transformed into a single epoch based on the local transformations and plate shifts.
I think this is called a dynamic datum, but the only real change OSM need to make is storing that time value for each coordinate.
Comment from SimonPoole on 18 July 2019 at 09:33
OSM geometries already have a timestamp and if the datum used to determine the coordinates was actually WGS84 there wouldn’t be a problem.
The authors point, which doesn’t get across so well, is that in some circumstances we are using sources that are not actually using WGS84 and are offset to it. This makes after the fact correction based on shifts relative to WGS84 difficult.
Comment from StephaneP on 18 July 2019 at 09:39
What do you mean with “plate tectonic references”? There are 2 Coordinate Reference System type: Earth fixed (WGS84), and plate fixed (local datum like GDA94, NAD83, ETRS89 ….)
The problem is that Osm is mixing the 2 types: Plate fixed coordinates in a earth fixed database.
If the GNSS receiver you use today is not accurate, some recent smartphones include a dual frequency receiver. Sub-meter accuracy in our pocket is just a matter of a few years.
I had completely forgotten this April fool :-)
If we do nothing with the Australian data, we will see Osm contributors going to waste thousands of hours fixing everything with GDA2020 aerial imagery whereas they could use this time to add new data. I think it’s a big concern.
Here is a 1.8m offset:
Comment from imagico on 18 July 2019 at 09:50
From a theoretical perspective this is interesting and useful read. From the practical perspective of mappers and data users in OSM this is less significant than it might seem after reading this post in isolation.
For almost all sources of coordinate data used in OSM (be that standard non-differential GPS or any kind of satellite imagery) the inherent tolerances in such data are much larger than the systematic errors you are discussing here. But more important than this fact alone is that mappers to a large portion are very much unaware of most of these error sources. Even systematic offsets in imagery (which is what is discussed most frequently in OSM of these) are often ignored.
In other words - measures to tackle the data consistency problems you are discussing here would necessarily be preceded by measures recording the absolute accuracy of the data we record. And that would again need to be preceded by measures that allow and enable mappers to actually assess the absolute measuring precision of their coordinates with an accuracy that comes somewhere close to the range you are talking about here. And we are far away from even the basic technical ability to achieve this for any significant fraction of the earth surface or the mapper community.
At the moment we are struggling even with the basic problem that huge location errors in some coordinate data sources (in particular high resolution satellite imagery) in the order of 50m and more systematically dilute the quality of coordinate information in OSM despite the fact that we have the technical ability already to do better than this.
Comment from StephaneP on 18 July 2019 at 09:57
Your draft is very interesting. I will follow it and the proj mailing list too.
we are using sources that are not actually using WGS84 and are offset to it
we are using sources that are not actually using WGS84 and are offset to it
I think that the first thing to do is a “state of the aerial imagery”. We have to know where the data are not using WGS84 coordinates, and the first step is to add some reference marks in every country with high res imagery: https://wiki.openstreetmap.org/wiki/Survey_points_on_aerial_imagery
Comment from jidanni on 20 August 2019 at 10:44
Alas, even if I had the exact coordinates of those markers in the pictures… well…
Comment from Warin61 on 20 August 2019 at 23:39
Usually when adding data there will be other data that is already in OSM. The imagery can be shifted to match the data already in OSM and then the imagery can be used without causing too many problems.
If the data already in OSM is wrong and you add new data that demonstrates this shift then the resulting map will show this as a distortion between the two data sets giving an incorrect impression of the land scape. This to me is worse than having the entire map shifted by some meters as most would accept that shift as being an error that is easily allowed for.
‘Exact coordinates’ can be had from trig points, state survey marks and a number of other places. But you need to know everything about the survey - when it was done, to what accuracy, to what datum etc etc and that should be avalible together with the location information. If that extra data is not avalible then it is not an ‘exact coordinate’.
Comment from Eric S on 8 October 2019 at 20:58
In France, in Grenoble’s suburb, I used my Garmin Etrex 30 (GPS+Glonass) with my bicycle to record tracks.
I drove precisely at the junction between light and dark areas:
Now, you can see the report in JOSM:
Background are orthophoto and cadastre both in RGF93 (i.e. local datum that was mostly aligned with WGS84 in 1993).
So, even with a consumer product, we can touch the problem.
Comment from kgjenkins on 8 May 2021 at 05:29
Thanks for this great explanation! Would it be possible to fix the image links? It seems they can be opened directly (in new tab or downloaded) but are not loading within the article. (Maybe there is a referrer constraint on the server?)
Comment from StephaneP on 8 May 2021 at 06:04
It works here. If you use some ads blocker or privacy addon inside your browser, check them.
Comment from kgjenkins on 8 May 2021 at 06:42
@StephaneP Thanks for checking. Ends up being a browser issue due to mixed content, with some browsers (Chrome, Edge) blocking http images from this https page.
Comment from StephaneP on 8 May 2021 at 06:50
@kgjenkins that makes sense. I’d like we can embed images directly on the osm.org servers. After a few years a big part of diary images are not correctly linked anymore.