Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race

[isazi]

@Łukasz-Szmigiel recently found out about Trimble, sounds interesting.

[A Former User]

I am seriously thinking about going back to using my old Ambit3 instead of Suunto 9.
With the new GPS firmware I no longer find the distance accuracy acceptable on the local trails where I run.

Today I did a trail run with a few of my friends. All of them currently use Garmin 5X and their distances were consistent between 7.40 and 7.48 miles. My Suunto has measured 7.11 miles. When I ran the same route with A3P a few years ago it measured 7.47 miles, so it is clearly Suunto 9 is the outlier in this case. The run consisted of a fairly short and steep uphill and a much faster downhill. The uphill part went straight up on a 30-40% slope, and the downhill part had a lot of switchbacks. On the uphill part my Suunto matched other watches, but lost about 6-8% compared to others on the 5 mile downhill.

And here is why (this is a comparison with my previous A3P run):

I see a similar picture in a lot of switchbacks. You can see how it stops short before the end of switchback, then cuts through. There are other places on the track where smaller turns are completely invisible on Suunto 9 track.

Here is the entire comparison: https://quantified-self.io/user/JMRgYAdyBBXBsMbxUHGVRwlKoKq2/event/8ntW1ekBcZZMA46TQEC0

@Dimitrios-Kanellopoulos, is there any chance Suunto would consider different algorithms for trail running vs. road running, for example eliminating the 10 meter threshold when saving points for activities such as trail running. Or even better, giving us an option. If the answer is no I’d need to seriously consider an alternative. Fenix 6 is too inaccurate too, but I think that going with Fenix 5X might not be a bad idea.

[fejker]

@silentvoyager whow, 30-40% slope … you are machine-like. You actually run at that grade? I would have trouble hiking at 2 km/h at that grade.

[A Former User]

@fejker said in Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race:

@silentvoyager whow, 30-40% slope … you are machine-like. You actually run at that grade? I would have trouble hiking at 2 km/h at that grade.

I hiked uphill all the way, then ran downhill. That is pretty normal for trail running

[A Former User]

Here is another evidence about how Suunto 9 “swallows” distance on trails - a screenshot from FitFileExplorer application:

This screenshot is from a downhill part of the run I mentioned above. Notice how the distances updates only every 10-20 meters. Normally it takes about 3 seconds to cover every 10 meters. However in the middle part it takes 21 seconds to increase the distance by another 10 meters while speed and cadence remain nearly constant. So clearly the trail had a turn or a series of short switchbacks at that point but from the watch point of view I barely moved.

Where is FusedTrack technology when we need it the most in a case like this? Why FusedTrack isn’t enabled in Performance mode?

[Dimitrios Kanellopoulos]

@silentvoyager hi this is basically one of the last times I am going to mention the distance and fit files or expoorts. I hope it will be understandble for you.

Imagine a tunnel. You run. When the watch determines the distance it will update it.

We could also not write at all the distance at the fit file or exports like gpx export does in many cases.

That said , if from “comparing with friends” is your conclusions based I am sorry to say this is not something valuable or claimable as an argument.
I don’t know what systems they used, were they all in the same arm ? All followed same footsteps ?

I don’t even know if you had at the same arm the ambit or different arms and the s9 was on the left , thus taking the short corner from your screenshots.

You want to test better and provide good feedback ?

Wear the ambit and s9 same arm , 10cm at least apart via eg a buff and do a route that has the equal amount of left to right corners.

I will be deploying on qs soon a feature to show the navstar distance as well so you know what is used in terms of fused speed.

So I would strongly suggest to either test right either try not to waste your time.

[A Former User]

@Dimitrios-Kanellopoulos Thank you for explaining! I get your tunnel exampe. (Although in a real tunnel the watch would start updating the distance from accelerometer once it loses GPS reception, but that is another topic).

What I see on the map and in the FIT is that the position and the distance are updated only when the distance from the previous point increases by more than 10 meters. It is very rare to see two points within 10 meters on any tracks that I looked at. Do you think that is a coincidence that the watch can’t update the position/distance sooner due to poor GPS reception or a deliberate decision in the algorithm, or some combination of both?

I claim that because of that the watch loses distance accuracy on trails.

[Fenr1r]

@Dimitrios-Kanellopoulos said in Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race:

Wear the ambit and s9 same arm , 10cm at least apart via eg a buff…

Given the different antenna placement of the two designs, should one be worn slightly rotated around the wrist/forearm so that the sky views match?

[Dimitrios Kanellopoulos]

@Fenr1r if you have the no bump/patch antenna, then even how the watch is positioned in terms of horizon is important.

assumme “-” is the watch screen pointing to the sky.

/ -> will offset to the left mostly
\ -> will offset to the right

That is also usually the trend that happens when worn left or right. There is a difference and very noticable.

@silentvoyager I dont argue that the new FW has less distance. A US fieldtester has been pointing this out and he did not agree with the new FW so much.

I will add today later (currently testing ) on QS a GNSS distance stat. Also will provide some insight. I do trails as well, I use stryd. I find it curious that also for me the new GPS FW of the s9 reports less distance but: Stryd always and still reports much less.

I will provide more examples later.

On the good side of the news there is a new GPS FW tested with Beidou as well.

[Fenr1r]

@Dimitrios-Kanellopoulos It certainly makes sense: there seem to be around 50 degrees of difference in antenna angle (assuming both are basically in line with the axes of their mounts). That’s a fair bit of sky one of them’s losing if they’re buckled conventionally.

So twist the more comfortably twistable watch so its antenna matches the less comfortably twistable? Loosen strap and stick a little wedge into the air-gap? Or (given that more data is usually better than less, even considering similarly oriented test subjects) just turn the S9 to match the A3P when worn on the left arm?

[Dimitrios Kanellopoulos]

@Fenr1r you could say that that would help. For example, no joke, test done, wearing the watch on the head would dramatically improve reception.

What many people, forget is that the problem is the actual movement of the “antenna” in regards to it’s own axis orientation. Take for example track and field that your arm moves rapidly, walks that you might put the hand in your pocket etc.

Even running and raising the hand to contantly monitor the pace for example, will improve the GNSS tracking. You can perform a simple test via using the routes screen and if you are off the drawn path (eg GNSS is a bit out ) raising the screen to face the sky can improve after a few.

That are my observations from testing and reading, I am not a professional on this.

[Brad_Olwin]

@Dimitrios-Kanellopoulos I agree with you on the Stryd. Mine is consistently less distance on the trails, ones that I have run for many years with many watches.

[nseslija]

@Dimitrios-Kanellopoulos that is probobly the reason why the bike rides with the watch on the handlebar is always really really good…

[Dimitrios Kanellopoulos]

@nseslija yup

[Fenr1r]

@Dimitrios-Kanellopoulos Yup. Got very accurate comparative results once by strapping two to the top of a rucksack. Bugger to tell the time, tho’.

Not just watch angle, of course. Satellites can shift over the course of a race/trek, messing with the accuracy for different people (with the same watches) at different stages on a route. And if a significant number of satellites are low in the the sky, every obstacle in LOS makes a relatively crappy HDOP worse. I seem to remember reading somewhere that it’s not just trees, buildings and hills … enough people between you and a satellite can worsen the accuracy. The sort of numbers you might get at starts or bottlenecks on group runs.

I wonder if the other linked metrics recorded in the FIT files have corresponding levels of (in-)accuracy. Beyond a certain decimal place, extra granularity in any statistic related to distance might be wasted.

[A Former User]

@Dimitrios-Kanellopoulos said in Suunto 9 with the recent GPS firmware was among least accurate GPS watches in today's 25K trail race:

I will add today later (currently testing ) on QS a GNSS distance stat. Also will provide some insight.

I’ve tried the beta version of QS and can see the distance graph. If I zoom all the way in I can see how distance is incremented, and can find some spots where it remains flat for a long time despite moving with a steady pace and cadence, for example this where you can see distance remaining the same for 23 seconds:

In general the distance seems to always increment by 10-13 meters. I am curious whether that comes directly from the GPS chipset or an algorithm in the watch that skips some position / distance updates in order to smooth the track?

[A Former User]

I did another race with a bunch of other runners and had another opportunity to compare my Suunto 9 result (and a few other Suunto 9 results) to other watches. The method I used is the same. I open Strava FlyBys and look at matches activities, then open them and look at the distances and quality of the tracks. This was a looped race course so it is a bit easier to compare the GPS performance based on how clean repeated loops are. This method isn’t very scientific, I know, but it gives a good enough overall idea. I’ve done that multiple times and I can clearly see some patterns now.

My subjective conclusion is that Suunto 9 is among the worst now. The only watch that produces worse tracks is Garmin Vivoactive. All cheap entry level Garmin watches clearly outperformed Suunto 9 both in terms of the track and in terms of the distance. Garmin 6X has also showed a better, albeit a bit noisier track, and much more accurate distance. Even Coros Apex, arguably, performed better.

Here are some images for comparison:

Garmin 6X - 3 loops, 14.93 mi (4.97 per loop). I can tell that the track is a bit noisy, so the accurate distance is probably accidental. Suunto 9 used to be the same with the previous firmware.

Suunto 9 - 3 loops, 13.87 mi (4.62 mi per loop). It lost GPS 2 or 3 times on the 3rd loop, but as far as I can tell that didn’t affect the distance in the negative way because the 3rd loop was actually the longest.

Suunto 9 - 3 loops, 13.51 mi (4.50 mi per loop). This is from another runner so I don’t know what firmware was used.

Here is for comparison 6 loops made with A3P - 29.24 mi (4.87 mi per loop). This is the cleanest track among all that I looked at.

Close-up of Fenix 6X. Not very clean, but it got the turn at the top left corner right:

Close-up of my Suunto 9. Look at the same corner. Also look how badly it cut the corner at he sharp U-turn closer to the finish:

Close-up of another Suunto 9. This is just horrible - one of the worst tracks in the entire race:

Yes. We can continue to pat ourselves on the backs and tell ourselves that Suunto is the most accurate. But I’d say the reality is far from that.

[Dimitrios Kanellopoulos]

@silentvoyager perhaps time to change brand and pat your self in the back?

With this kinda feedback and attitude friend you are only enclosed to your own mind.

Its only been a constant complain here

I really dont know what else to say with the attitude you carry here.

We are constantly trying to improve the product, I personally try to figure out what goes wrong but its always a blocker from your side.

You don’t like Suunto app (Which this forum WAS about), you have to agree with all the complains here, why MC is better said 1nce per week in each debate.

So I have to ask. What are you trying to get out of here? What is the purpose of posting and keeping this atitude?

Its ok to suggest , and post an issue but for the love of god you could put some effort to be a bit less :

Yes. We can continue to pat ourselves on the backs and tell ourselves that Suunto is the most accurate. But I’d say the reality is far from that.

We are people here reading the comments and feedback not machines and at some point in time the only thing bad sentiment brings is well, giving up.

Honestly you could just have ommited the

Yes. We can continue to pat ourselves on the backs and tell ourselves that Suunto is the most accurate. But I’d say the reality is far from that.

and this would have been a great post, I could help (as I have contacted you many times) with providing test FW or getting in the test team but sorry to say this is not an attitude one can have communicating a subject.

Sorry to say so.

[cosme.costa]

I’m not going to say that the S9B is the most accurate in class (track wise) but in some occasions it is really accurate, usually I have better tracks than the ones that I had with my SSU. I have to say that only have it for two months and that I like it a lot.

Despite all of this sometimes I have the same feeling than @silentvoyager with the S9B,regarding distances. I do some short Cross races, usually 3 to 5 loops to a non permanent circuit with lots of turns and switchbacks. Usually all of the participants got shorter distances than the ones published by the organization, compared with the other participants the S9 is always in the shorter distance area. Yesterday I raced and the official distance was 5 km, the S9 counted 4.6 km and the other participants (with Garmins and Polars ) got among 4.7 and 4.8 km. Is this a lot? I do not know.

On the other hand, today I went for a morning trail run and for the first time I got the message that I have lost the GPS signal. I have lost it for nearly 5 km in a nearly 13 km run. First of all, I have to say and as it has been said in another posts that the message is very annoying and shouldn’t be permanent, it also should have a different vibration pattern than the auto-lap, maybe two small vibrations (I do not use sound in the watch). The run that I did today I have done it 15 to 20 times so I know more or less where are all the 1 km split, but today the first km alert was later than it should, this was strange to me and makes me believe that the AGPS was not OK and made also lose the GPS. Anyway the next km was consistent + the offset of the first km so I stopped to pay attention to the watch, I was running with headlamp in trails so I prefer to focus in the running. But then I got a vibration identical to the 1 km auto-lap too early that was when I saw the GPS lost signal message. Later on, when I saw the track I’ve realized that I lost the GPS 3.5 km aprox before I was aware of it (maybe I have the message before but because it has the same vibration pattern than the auto-lap I wasn’t aware). 
In this route I have never lost the signal, not with the Ambit 3 Vertical, not with the SSU, not with the Sigma bike GPS and not with the S9B until today. The sky during the run was completely clear and open, only stars and the big moon, after 1.5 km from where I have lost the signal there is nearly 1 km in an open area where the watch should have regained the signal but it didn’t. Is for that that I believe that this GPS signal issue is an isolated case and maybe due a bad AGPS.
Then the total distance was pretty accurate (Fusedtrack, Good job here Suunto!! ) and the track where I have signal too.
I think the GPS firmware/algorithm is not the best yet but, seeing the last updates, Suunto is in the good direction and I believe that sooner than latter we will be very happy (I still am)

[Dimitrios Kanellopoulos]

@cosmecosta in regards to this we are bringing more FW updates and yes there are a lot of issues mentioned here recognized and fixed already.