Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race

[sky-runner]

@Dimitrios-Kanellopoulos

Here is an example from a race that I did with Suunto 9 back in January. I used an online tool to export FIT file to CSV, then added a column which is a distance delta from the previous distance, then created a graph with timestamp on X axis and distance delta on Y axis. Perhaps that isn’t the best possible analysis, but something I could do very quickly. You can see a very distinct cutoff at 10 meters for the most of the activity. There were just a few points with 6-9 meters between the points.
It is interesting that it dropped to 3 meters between the points at the end, but that is where my watch had issues with GPS and it wasn’t getting GPS signal for most of the time.

The overall duration of this was about 2 hours. The terrain was fairly flat but nearly 100% of it was under tree cover.

[sky-runner]

Here is a similar graph from another run. Yes, occasionally the distance between the points is less than 10 meters, but for the most part it is 10 meters or greater - there is a clear threshold visible on the graph:

[ollemelin]

Alright then. I’ve done some more tests with SB9 and my conclusion is that SB9 measures about 20-30 meters short on every km compared to Maps, Garmin devices and various apps like strava. Strava, garmin are spot on with maps most of the time.

I’ve tried different kinds of GNSS-combos on SB9 but with same results.

What do you think? Sameting wrong with my device? Some days ago i ran with a friend with Spartan baro and my SB9 came short also with him.

[cosme.costa]

@ollemelin said in Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race:

Alright then. I’ve done some more tests with SB9 and my conclusion is that SB9 measures about 20-30 meters short on every km compared to Maps, Garmin devices and various apps like strava. Strava, garmin are spot on with maps most of the time.

I’ve tried different kinds of GNSS-combos on SB9 but with same results.

What do you think? Sameting wrong with my device? Some days ago i ran with a friend with Spartan baro and my SB9 came short also with him.

I usualy run with a friend that uses an Ambit 3 Run and my S9B is always short on distance. I think that is something that Suunto knows (from what I recall in others posts). For me is not a big issue, I can live with that. What I’m not so sure is that Garmins are spot on, I guess it depends on the model. If I see the tracks of some people using Garmin (mainly Fenix) they are OK but not spot on, and if the track is not ok can’t be the distance 8Except if they do corrections when uploading the the activity to their servers). And when you say Strava is spot on, what do you mean? planning a route or recording it with a phone?

[ollemelin]

@cosmecosta

I just don’t really get the reason why it is this way. I like suunto and the new app is really great, but this measurement issu is really bugging me

Thinking about buying an stryd-pod…

[Dimitrios Kanellopoulos]

20-30 per km = 200-300m per 10km = 800-1200m per 40 km to put it in perspective.

@ollemelin are those results from trails ? Or road ? How would you categorize the “conditions/visibility” of GPS.

Owning a stryd in general terms will make you stop thinking of any GPS based watche’s accuracy. However by far I dont say that like: Buy stryd. It’s just because it will give you the correct, consistent cross watch results. You will stop worrying about a bad GPS day etc etc. Get an older version of 100e Your Suunto, Garmin, Polar, COROS will love it

[ollemelin]

@Dimitrios-Kanellopoulos

It’s not a GPS visibility problem. The measurement is consistent on track, road and trail. In surveying we call it ”systematic error” and the device needs calibration.

Ofc the error seems small long distance but its really bad for shorter distances. It’s bad when apps like strava or runkeeper makes a better job.

Does it matter for my running development? No! But i still wish Suunto to fix this issue

[Dimitrios Kanellopoulos]

@ollemelin I dont argue. By far. From what you said the error is huge for long distances.

But you didn’t reply on the conditions of the “systematic” error.

[ollemelin]

@Dimitrios-Kanellopoulos

Sorry! I misread your answer.

Yesterday i ran in perfect conditions. Clear blue sky. Straight road and no trees. 20-30 meters short compared to map, garmin 735 and strava.

But I also run trails with trees etc and about same drift Every time

[Dimitrios Kanellopoulos]

@ollemelin Roger that.

Let’s see what a new gps firmware can bring.

In theory and according to our tests and fieldtesters the distance on simple paths is ok. On some cases of trails it can be short. According to recent researches and academic papers this doesn’t vary so much as you you say but I don’t argue. I have seen this feedback personally many times.

I can only ask you to “wait” I don’t have anything else to propose atm unfortunately.

However it would be super interesting even if you , your friends etc tested stryd on the same route / run.

Have a great weekend, I am signing off at least for today

[ollemelin]

@Dimitrios-Kanellopoulos

Thanks. I have some extra money to spend so I think I will buy an stryd. Maybe i can test in my work aswell

[Łukasz Szmigiel]

@Dimitrios-Kanellopoulos just out of curiosity - is there anything being done to those strange offsets that occur in first 10 - 15 minutes of workout, especially if one starts the workout fast (without giving the watch some time after the GPS arrow is green) and / or in demanding gps conditions?

Such offsets do autocorrect after some time and change of direction (ie. 90 deg) but this behavior is strange. Is it because fusedspeed has priority if signal is below certain threshold?

[Dimitrios Kanellopoulos]

@Łukasz-Szmigiel I didn’t know about what you just said but yeah it sounds possible because I can think about a reason behind it, that actually has many times caused a headache.

---

When the “GNSS lock” / “SOLID Green arrow” happens it means that the signal is quite good for you to go on. It aint perfect. Depending on the environment it will vary.

Now consider that a quick start (for running mainly) means arm movement, a lots that is, and a “good” GNSS lock.

I think we can all agree it can drop the “signal strength”/“lock quality” quite easily, as in contrast to cycling with little arm movement. Why would it then?

Here is an extra case: Hard conditions (eg canyon) to get a lock, and after a weak lock, you go run in the forest in the canyon. … You might loose GPS.

I hope it makes a little sense. What I wrote above applies to all brands, and is less/more effective depending on the antenna position. I hope it helps

[Łukasz Szmigiel]

@Dimitrios-Kanellopoulos thanks for explanation. I was suspecting fusedspeed and the way firmware interprets GNSS signals to avoid zigzagging between readings.

Here’s a run from today, fresh, for reference. I’ve started running right after obtaining GNSS lock (GPS + Beidou) but pressed start after ~50 meters. Conditions were meh at best - it was raining all day and it’s still overcast. QS link.

I’ve marked actual path in pink:

1. Here you can see start and finish (~35 minutes later). Start is offset while finishing path is spot on, I’d say up to 3 meters.
   

2. Continued from last screenshot.
   

3. And right before I’ve entered the park when it auto corrected after a short path on embankment. This is interesting as I’ve entered a wooded area but nevertheless the accuracy improved and the offset was gone from that moment.
   

Just to be clear - I’m not complaining, I just find this behavior fascinating and I’d like to dig deeper into it and perhaps know how to reduce it when accuracy may be more important than on a regular “everyday run”.

[ollemelin]

@Łukasz-Szmigiel

Yeah, I think a Quick start is one reason behind the issues im having. My wifes garmin 735 takes a good bit longer to aquire gps signal. Ideal i would like to have a big screen before I press start reling me how good signal is from 1 to 5 :).

With that said, I still have a lot more spikes in my data compared to others in same conditions. Maybe garmin have an algoritm that smooths over those spikes. I don’t know.

I did order a stryd yesterday so hopefully that will lower my blood pressure. I live in Sweden and there are trees and forests everywhere so GPS-readings are never going to be that precise anyway.

[Łukasz Szmigiel]

@ollemelin if you can, try waiting for a little bit longer on a start screen before you actually hit “Start”. I used to stand in place where sky visibility is good and slowly rotate with my arm in front but I no longer do it since the firmware that introduced Beidou is good enough to begin the workout without any extra rituals

Some extra data about GNSS quality on a start screen would be a nice addition but I doubt it honestly. Suunto seems to be on a “don’t bother users with too much data” side and I think it’s wise.

I did have a glimpse on my last workout on QS and Runalyze and it really seems that Suunto is doing its magic with fusedspeed which results in those strange offsets in recorded paths. This is pretty logical - if GNSS is poor and the path would be wobbly but the pace takes priority, the line has to be straight. So the error at the beginning of a turn will persist until the direction changes above certain threshold.

1. This is the start area - you can see that GNSS quality was poor by looking on altitude diagram. It should be around 120 meters but is way higher and dropping as I run (I have an S5, so there’s no baro). But the pace is rock steady regardless of poor GNSS readings. Also, after enabling “Show points” in QS, it seems that the actual points on the map are more spread apart in this area than in the open - so the watch must behave differently if it’s unsure of it’s position.
   

2. Now take a look on the same area on Runalyze and its pace chart - it oscillates like mad between 5 and 8 min/km.
   

So it really seems to me that Suunto is doing good job to provide as accurate data as possible during the run but since it’s impossible to do this without sacrificing something, it chooses to do a recorded track offset in order to provide more accurate tempo and speed readings.

I have never seen anything like it before with different hardware - it’s more common to have a zig-zag pattern when GNSS reception is poor, as the path is constantly drifting and it’s being reflected on the record.

But if you think about it - it’s better to have more accurate predictions about pace and speed than a GNSS path that is constantly wobbling from right to left. But the effect is a constant offset until something dramatic happens (I assume it’s either a sudden change in direction, speed or GNSS precision increase).

Neat and practical programming.

Btw. @Dimitrios-Kanellopoulos, kudos for “Show points” in QS - it’s great for analyzing such events. You can clearly see that the watch has two modes of saving data where the dots are either more or less condensed - I assume it’s related to GNSS reception.

[freeheeler]

I think something went bananas today… some segments were right to the point and some completely off

[Dimitrios Kanellopoulos]

@TELE-HO COROS + VANTAGE + SONYS had issues yesterday. Logically SOny chip issue. If you need links …

[Mff73]

@Dimitrios-Kanellopoulos
I had bad GPS yesterday as well, but no Sony in my Spartan GPS Fix was long, need to sync again AGPS, and track is bad
But no complain, i enjoyed my almost nighty, and totally rainy workout

[surfboomerang]

@Dimitrios-Kanellopoulos
Only Sony? Can this be chip specific? GPS errors affects all chips right?