This will be my third and final installment in a series of articles discussing device I’ve used when preforming surveys when contributing to OSM.
In the first article, I talked about accuracy limitations associated with GPS signals while the second gave an overview of four specific mapping devices I’ve personally used. In this article I’ll compare the accuracy of three devices previously discussed using GPS traces.
To test device precision, I recorded GPS traces along a hike with a variety of line-of-sight obstructions to the sky with each device recording at the same time. The traces were assigned colors to help differentiate each device:
The arrangement of the devices are as follows:
The EcoDroidGPS and S10e location information were sampled every one second within a GPX track using the app OsmAnd. The 66sr records as an “activity” with location information sampled every one second. The resulting GPX track was downloaded from the device.
Before arriving to the trailhead, I had to walk along the shoulder of a bridge spanning a medium size river. The bridge is long enough to offer a best case scenario for accuracy. There is a good 45 degrees of sky visible from most angles in the center of this bridge.
The result of a stretch of road with minimal interference is quite predictable, all three traces are stacked right on top of each other without significant deviation.
The trail begins as a wide gravel path visible from satellite imagery.
In this section of the trail, I stayed to the far right which is more accurately portrayed by the green EcoDroidGPS and purple Garmin traces. It’s interesting to note that the orange S10e trace follows the curve of the others, but is offset about 4 meters to the left before rejoining the green and purple tracks towards the top of the picture.
At this point, I took one of the side trails which starts in a mild wetland area before breaking to thick woods. In fact, from the satellite imagery, the trail is not visible.
Looking at the traces, the green EcoDroidGPS and purple Garmin traces agree with each other, especially during the first part of the trail where the tree cover is not as dense. At this point, the orange S10e is surprisingly more precise than the previous gravel portion, but still lazily tracks the actual path.
This section of the trail is an out-and-back, so we actually see two pairs of each trace as I return along a path I’ve already taken. The path tread here is maybe half a meter wide which means I should be walking the same path both ways.
Here we see both the greatest deviation between devices and location, despite walking the same path in a different direction.The purple Garmin trace is the most consistent while the green EcoDroidGPS brackets it on each side. The orange S10e trace preforms the worst in this situation.
Which device should be used when surveying for OSM contributions? I think the answer depends on your situation and what you’re interested in mapping. A smartphone struggles with mapping smaller ways, but is sufficient when you can include satellite imagery, existing data, or even using multiple GPS traces and averaging the results. For greater accuracy, if you don’t mind the antenna mast sticking out of a backpack, the EcoDroidGPS is a phenomenal value. Since it uses a USB receiver, it can be upgraded as better receivers come to market. The Garmin is a perfect example where spending more money on a single GPS technology does not necessarily equate to better accuracy. It is certainly better than the smartphone, but not much is gained over the much cheaper EcoDroidGPS.
For me personally, I’ll be transitioning to using the Garmin, mostly because I already own it. I tend to view it as an extremely robust outdoor navigator whose data can later be used in OSM contributions. It isn’t as fragile as a phone or the EcoDroidGPS outdoors and I expect the accuracy to increase over time as features are added to the device. If it does, I’d like to post about that as well.
For everyone else, if you aren’t in the market for a more precise device to use in your surveys, wait for a smartphone upgrade. Multi-band and multi-GNSS devices will become common and will represent the next appreciable increase in precision, something that will simply be included in new phone models going forward. If a smartphone isn’t your thing, the effect of including them in a smartphone should help increase the availability of this technology and lower the cost from other vendors.
Comment from lectrician1 on 6 November 2020 at 04:21
Comment from Radleyx1x on 6 November 2020 at 11:57
Call please and I’ll explain
Comment from ChristianA on 6 November 2020 at 14:29
I did some similar tests around 2013, comparing a smartphone and the Garmin eTrex30. Then I could clearly see that the smartphone did fairly well out in the open, but as soon as I entered a forest it lost accuracy much more than the eTrex did.
Do you have any idea wether your new Garmin outdoor GPS might be better than my old eTrex (from 2012)?
Comment from bobwz on 6 November 2020 at 16:38
@ChristianA The eTrex uses only two satellite constellations at a time GPS and GLONASS while the new units can use GPS, GLONASS, Galileo, QZSS, and IRNSS (QZSS and IRNSS are only applicable to Asian countries). In addition the GPSMAP 66sr and the cheaper GPSNAP 65s both use multi-band and multi-GNSS for positioning. With more satellites at its disposal and an external antenna stub, it should absolutely be more accurate than the eTrex30.
With that being said, I’d recommended waiting on getting the dedicated Garmin devices with multi-GNSS until the firmware improves a bit. While the unit is quite accurate in its current state, I don’t think it’s worth the price tag if you primarily will only use it for mapping. If you’ll use the navigation features while hiking or other outdoor activities it may be worth it and the precision would be a bonus.
The 66sr specifically is expected to gain RINEX logging which means that data can be pulled from the device to achieve nearly centimeter accuracy with post-processing, but this is not yet available despite what is listed on the Garmin product page.
If you’re up for a smartphone upgrade, see if any of the new phones you’re weighing have multi-band or multi-GNSS. I would guess you’d see precision like you see from the green EcoDroidGPS tracks in this article.
Comment from philippec on 6 November 2020 at 17:58
Multi-GNSS smartphones are already very common.
I don’t know multi-band, I only know what is called dual frequency.
Garmin has only recently released a dual frequency thing.
Comment from Dzertanoj on 6 November 2020 at 19:00
This comparison by design can’t bring any conclusive results because of two fundamental metrological factors.
- Unknown accuracy can only be measured against the reference data with known (higher) accuracy. It is impossible to measure it having three devices with unknown accuracy.
- Smoother track doesn’t necessarily means better accuracy - it could be caused by filtering that doesn’t improve anything rather than visual appearance. In fact, filtering can create false perception of the higher accuracy while distorting data instead of improving it. Smoother appearance is highly deceiving.
So, making comparisons like that might be fun, but it’s far from being practical, leave aside being scientific. And “scientific” doesn’t mean “fancy”, it means “conclusive and reliable” (and that’s the only way to achieve that).
Since the test happened in the US, there should be some high-accuracy GIS data available from DOT or another agency that could and must be used as a reference to measure everything else against it.
Comment from bobwz on 6 November 2020 at 19:58
@Dzertanoj You are absolutely correct this was not a scientific survey. Nor was that the intent. When dealing with GNSS location data where one must be absolutely confident in the accuracy of a survey, at least two devices are necessary (as seen from professional surveyors) to account for exactly the interference you described. This article series relayed my personal experience when using only one device that is relatively available to the average OSM mapper.
By comparing the tracks created by three devices recording at the same time, it shows how the devices respond during the same atmospheric and meteorological conditions. Differences in the track are more likely to be due to each device’s properties and their processing rather than variances in atmospheric conditions. I did omit details how each device sampled location data at one second intervals, but I’ve added that in based on your feedback.
I do not present these devices as substitutes for professional surveying equipment or teams when you need to know where exactly a specific coordinate is on planet Earth. Instead, these accounts are my experience as a hobbyist contributor to OSM for others who may be wandering into the single-device accuracy rabbit hole as I did.
Regarding post-processing the information, this is the next area I’d like to discover. As far as I know, neither the S10e nor the EcoDroidGPS make available their raw satellite observations for post-processing against data provided by continuously operating reference stations. The 66sr should be able to provide this data through RINEX logging, but this feature is not currently available in the firmware. Do you have information on how this is done? I would be super interested in learning how RINEX logs are processed to further increase accuracy.
Comment from philippec on 7 November 2020 at 12:17
Garmin seems to have its own terminology.
Comment from bobwz on 7 November 2020 at 14:43
@philippec Which terms did you find unique to Garmin? While initially researching the device, some terms are indeed Garmin products, but others were generally available words. If you’re talking about RINEX, it’s a data exchange format like GPX not unique to Garmin devices. I had first heard of it for this device and I’m eagerly awaiting its activation. Here is the Wikipedia page for RINEX.
Comment from philippec on 7 November 2020 at 14:48
Dual frequency may in the future, who knows become more than 2 frequencies, so call it multi band, that could be called multi GNSS, or multi GPS, for the common folk. :)