Validity of Current Smartwatches for Triathlon Training: How Accurate Are Heart Rate, Distance, and Swimming Readings?

1. Introduction

Since 2016, the American College of Sports Medicine has listed wearable technology as one of the top three fitness trends worldwide, demonstrating the enormous impact wearables are having on the sports community [1,2,3,4,5,6,7,8]. Using photoplethysmography, a key sensing technology, it is possible to track physiological parameters such as heart rate, heart rate variability, heart rhythm, pulse wave, respiratory rate, oxygen saturation, and sleep quality [9]. The technical possibilities continue to increase, as wearables are becoming progressively smaller, and better sensors, such as accelerometers, gyroscopes, or global navigation satellite system (GNSS) receiver chips, are being built into the watches. This increases the comfort for an athlete who wishes to track their training automatically with virtually no additional technical effort. On the other hand, external sensors, such as temperature sensors or additional acceleration sensors, are also being included in wrist, trunk or shoe pods to record additional data. In 2022, 492 million wearables were sold worldwide, with a predicted upward trend [10]. Today, wearable technology offers the advantage of balancing training load and monitoring health status at the same time. Athletes, coaches, and sports scientists have identified its potential to prevent injuries, improve performance, and prolong an athlete’s career. However, the validity, accuracy, and reliability of this evolving technology may be, in parts, controversial and uncertain. [11] In a survey of triathlon coaches, it was found that some coaches were concerned about athletes’ overreliance on technology and that the data it produces can impact their perceptions on performance [12]. The enormous potential and constant development of smartwatch technology regularly necessitates the review of current and new models. Various models have already been assessed, however not with a triathlon-specific focus in mind [13,14,15,16,17,18]. Notwithstanding, the accuracy of the measured parameters is crucial to achieve the desired benefits.

Hence, the goal of the present study was to determine the measurement accuracy of ten different sports watches in regard to optical heart rate (i.e. pulse rate) measurements and GNSS-based distance tracking, as well as in the acquisition of swimming data. The following questions were addressed: How accurate is the optical heart rate measurement during a 3-minute run on a treadmill? How accurate are the GNSS measurements during a 4,000 m run on a 400-m tartan stadium track, a 3.4 km hilly outdoor run, and a 36.8 km profiled bike ride? Are there differences in the GNSS measurements among the various watches? Furthermore, do such differences in the measured distances depend on whether the watch is worn on the wrist facing the inside or outside of the running track/course curvature? Lastly, how accurate are the lap counts and arm stroke counts in a 50-m Olympic-size swimming pool? These questions focus on typical single-discipline triathlon training, which is usually divided into separate sessions of swimming, cycling, and/or running. An assessment for use in brick sessions or competitions was not carried out, as it would require a multisport function, which is not supported by all of the selected smartwatches.

2. Materials and Methods

2.1. Smartwatches

Ten different smartwatches currently available on the market (as of May 2024) from nine manufacturers were included in this study (Table 1). The smartwatches were selected to cover a wide range of manufacturers with adequate market shares, ranging from entry-level products at a recommended retail price of 120 €, such as the Mi Watch (XIA), to an advanced triathlon-specific model, the Garmin Forerunner® 955 Solar (GAF), at 650 €. In an overview of global sales figures for wearables from 2014 to 2022, in total eight leading manufacturers were listed, seven of which are included in this study: Apple, Samsung, Xiaomi, Huawei, Fitbit, Garmin and Fossil (order according to rank) [19]. Looking at the most popular smartwatch brands in Germany in 2023, the top seven are represented in this study, supplemented by Fossil on rank 10. Moreover, Polar was included in this study, owing to the popularity of their smartwatches in Germany [19] and because Polar watches were analysed in previous studies [15,16,18].

As regards movement tracking, a multi-GNSS analysis is more accurate than a single GNSS [20]. For this reason, the GNSS function of the watches was set to utilise both the American GPS and the Russian GLONASS systems in parallel to achieve better comparability where applicable. If this was not available, the watch’s default settings (commonly GPS only) were used. The possible GNSS options are shown in Table 1, based on the information from the respective manufacturers. Notably, Fossil does not specify the supported GNSSs in the Fossil Gen 6 Smartwatch (FOS). However, as its Snapdragon Wear 4100/4100+ processor supports GPS, GLONASS, Galileo, and BeiDou (BDS), that smartwatch is expected to do accordingly [21]. To compensate for any variability in production, GNSS data were simultaneously collected using two separate watches of the same model, worn unilaterally by the same individual when running or mounted to the handlebar when cycling.

For all measurements and watches, the most suitable sport mode for the planned measurement was selected; for running this was either outdoor running or track running. For watches that did not provide a specific track running mode, the regular outdoor running mode was selected. For road cycling, the outdoor cycling mode was used. If the watch did not offer this mode, the outdoor running mode was chosen instead, which was necessary with the FOS watch. Because of the higher average movement speed in cycling, it can be assumed that this differing choice of mode did not significantly influence the cycling distance measurements. For swimming, the indoor swimming mode was selected for any watch, and the lap length was set to 50 m. If a software update for a device or its corresponding application became available during the measurement period, it was installed to provide the most current device software status available.

Table 1. Overview of the ten investigated smartwatches, their manufacturers, and available GNSS options.

Smartwatch	Manufacturer	Abbreviation	GNSS Options
GTS3	Amazfit	AMA	GPS, GLONASS, Galileo, BDS, QZSS	[22,23]
Watch SE	Apple	APP	GPS, GLONASS, Galileo, QZSS	[24]
Versa 4	Fitbit	FIT	GPS, GLONASS	[25]
Gen 6 Smartwatch	Fossil	FOS	GPS, GLONASS, Galileo, BDS	[21,26,27]
Forerunner® 955 Solar	Garmin	GAF	GPS, GLONASS, Galileo, BDS, QZSS, IRNSS	[28,29,30]
Venu® 2	Garmin	GAV	GPS, GLONASS, Galileo	[31,32]
Watch GT 3	Huawei	HUA	GPS, GLONASS, Galileo, BDS, QZSS	[33,34]
Ignite 2	Polar	POL	GPS, GLONASS, Galileo, QZSS	[35,36]
Galaxy Watch 4	Samsung	SAM	GPS, GLONASS, Galileo, BDS	[37,38]
Mi Watch	Xiaomi	XIA	GPS, GLONASS, Galileo, BDS	[39]

GNSS: Global Navigation Satellite System; GPS: Global Positioning System (USA); GLONASS: Globalnaya Navigazionnaya Sputnikovaya Sistema (Russia); BDS: BeiDou Navigation Satellite System (China); QZSS: Quasi-Zenith Satellite System (Japan); IRNSS: Indian Regional Navigation Satellite System (India).

Table 1. Overview of the ten investigated smartwatches, their manufacturers, and available GNSS options.

Smartwatch	Manufacturer	Abbreviation	GNSS Options
GTS3	Amazfit	AMA	GPS, GLONASS, Galileo, BDS, QZSS	[22,23]
Watch SE	Apple	APP	GPS, GLONASS, Galileo, QZSS	[24]
Versa 4	Fitbit	FIT	GPS, GLONASS	[25]
Gen 6 Smartwatch	Fossil	FOS	GPS, GLONASS, Galileo, BDS	[21,26,27]
Forerunner® 955 Solar	Garmin	GAF	GPS, GLONASS, Galileo, BDS, QZSS, IRNSS	[28,29,30]
Venu® 2	Garmin	GAV	GPS, GLONASS, Galileo	[31,32]
Watch GT 3	Huawei	HUA	GPS, GLONASS, Galileo, BDS, QZSS	[33,34]
Ignite 2	Polar	POL	GPS, GLONASS, Galileo, QZSS	[35,36]
Galaxy Watch 4	Samsung	SAM	GPS, GLONASS, Galileo, BDS	[37,38]
Mi Watch	Xiaomi	XIA	GPS, GLONASS, Galileo, BDS	[39]

GNSS: Global Navigation Satellite System; GPS: Global Positioning System (USA); GLONASS: Globalnaya Navigazionnaya Sputnikovaya Sistema (Russia); BDS: BeiDou Navigation Satellite System (China); QZSS: Quasi-Zenith Satellite System (Japan); IRNSS: Indian Regional Navigation Satellite System (India).

2.2. Participants

A total of 30 individuals (15 males/15 females) with various training backgrounds, ranging from recreational athletes to trained long-distance triathletes (age: 29.4±7.4 years; height: 175.6±8.8 cm; body mass: 70.2±8.2 kg), participated in the heart rate measurement part of the study (Table 2). For the running, cycling and swimming parts, nine trained athletes of this cohort were recruited because the study design allowed for multiple measurements to be completed by the same individual without loss of statistical power. The measurements were conducted in and around Leipzig, Germany, and Magdeburg, Germany, from April to June 2023.

2.3. Heart Rate Measurements

The arterial blood pressure in rest was measured manually and bilaterally with a sphygmomanometer (boso med 1, Bosch + Sohn GmbH & Co. KG, Jungingen, Germany) by a physician. The resting heart rate was determined by electrocardiography (ECG), which was also used to rule out any deviations in the cardiac currents that could have led to an incorrect measurement. Except for two extrasystoles for one subject, which did not lead to exclusion due to their statistical insignificance, no abnormalities were detected that could have contributed false measurements. In particular, heart rate and pulse rate could be treated as equivalents for all subjects. In addition, the resting ECG and blood pressure measurements were examined to detect any pathologies that would have contraindicated a stress test. If the ECG results were normal and the systolic blood pressure was ≤ 160 mmHg, the treadmill test was started.

For this purpose, the participants completed 5 x 3 min intervals at a self-selected running speed (min.: 8 km/h) on a motorised treadmill (h/p/cosmos saturn®, 250/100, h/p/cosmos sports & medical GmbH, 83365 Nußdorf–Traunstein, Germany or Star Trac 10 FreeRunner™, MERCOR Fitnesskonzepte GmbH Leipzig, Germany, respectively). An 1.5% incline was set to compensate for the absence of air resistance. While running, optical heart rate data (i.e. pulse rate data) were collected continuously by two different smartwatch models in parallel using their photoplethysmography sensors, with one watch being worn on each wrist. As reference, true heart rate was measured electronically using a chest strap heart rate monitor (Garmin HRM-PRO, Garmin AG, Schaffhausen, Switzerland). The validity of the chest strap measurement has been confirmed in previous studies [40,41,42,43]. The measurements using the smartwatches and the chest strap started simultaneously and ended after 3 minutes of running at the target speed. All heart rate measurements were conducted with a data rate of at least 1 Hz.

2.4. Tracking of Running and Cycling Distances

To evaluate the accuracy of the distance tracking measurements, the smartwatches were tested on reference routes for running and cycling. To avoid environmental influence [44], the same location and the same date and time were used under a clear sky. Before the GNSS measurements were performed, a bike computer (Sigma BC 8.12, SIGMA-ELEKTRO GmbH, Neustadt, Germany) was calibrated with the tyre size. The rolling length of one tyre’s circumference was measured by marks on the ground while the rider was sitting on the bike (tyre size: 25x700c at 7,5 bar/109 PSI). This calibration was crosschecked and confirmed by measuring the length of a five-laps course on a 400 m stadium track on lane 1 as described below. With this calibration, all running and cycling distances were measured as reference values by riding the same course at least twice on the bike (with the tyre pressure kept at its calibration time value given above). Additionally, the true lengths of the running and cycling routes were crosschecked using OpenStreetMap and Google Maps. These three distance results per course, i.e. by bike ride, OpenStreetMap and Google Maps, differed only marginally in the second decimal place (3.41 km vs. 3.4(0) km and 36.84 km vs. 36.8(0) km). Notably, this small difference was inevitable as the bike computer’s display provided two decimal places, whereas the map material yielded only one.

For each run, 4 smartwatches were worn simultaneously to reduce the number of runs: two on the left forearm and two on the right forearm. The positions were numbered—(T1) proximal forearm, left; (T2) distal forearm, left; (T3) distal forearm, right; and (T4) proximal forearm, right—and noted for each watch and trial. Five runners (3 males, 2 females) ran 4,000 m on a 400 m standard tartan stadium track. The runners were instructed to stay in lane one, which is exactly 400 m in length, at a distance of 30 cm from the inner line [45]; running was maintained in this lane (with the exception of at maximum 6 quick overtaking manoeuvres per 4,000 m). Additionally, an outdoor run of 3.41 km on an asphalt road in profiled terrain was performed. The tests on the stadium track and the asphalt road were performed in triplicate by five runners, each with 4 smartwatches in positions T1–T4, resulting in 5 ⋅ 3 ⋅ 4 / 10 = 6 separate measurements per watch. The watches were rotated among the runners, as well as the forearm position, after each run. For the cycling measurements, four cyclists (3 males, 1 female) completed a fixed road bike course of 36.84 km on asphalt roads in both directions with five smartwatches attached the handlebars of each bike or to the cyclist’s forearms (the latter if a non-zero pulse rate was required for correct operation), resulting in 4 ⋅ 5 ⋅ 2 / 10 = 4 separate measurements per smartwatch model.

2.5. Tracking of Pool Swimming Activities

Three different swimming protocols were performed with the smartwatches in a 50 m Olympic-size pool to determine the accuracy of swimming lap counts and stroke rates. The following measurement protocols were carried out three times per watch, divided among six swimmers (3 males, 3 female), with a different watch worn on each forearm: (1) 200 m individual medley (butterfly–backstroke–breaststroke–front crawl) with stroke transitions every 50 m, (2) 400 m freestyle (performed as front crawl), (3) 200 m individual medley with stroke transitions every 25 m. During the freestyle sequences, the swimmers were filmed by a moving camera to provide a video-based independent validation of the number of strokes detected by the smartwatches.

2.6. Data Analysis and Statistics

The data collected from all smartwatches was transferred to the mobile phone apps of the respective manufacturers via Bluetooth. The following apps were used: Fitbit, Fossil Smartwatch, Garmin Connect, Apple Health, Huawei Health, Mi Fitness, Polar Flow, Samsung Health, and Zepp. Afterwards, the data were transferred manually to Microsoft® Excel® for Microsoft 365 (Microsoft Corporation, Redmond, USA). All mathematical analyses and statistical tests were performed with Microsoft® Excel® for Microsoft 365 (Microsoft Corporation, Redmond, USA), MATLAB R2023a (MathWorks Inc., Natick, USA), and JASP (JASP Team (2023), Version 0.17.1, Amsterdam University, Amsterdam, Netherlands)

To evaluate the accuracy of the heart rate measurement, the mean heart rate and the maximum heart rate were each determined for 3-minute intervals and then compared to the heart rate values from the chest strap reference measurements. The 3-minute interval per stage was chosen because of its commonality and significance in performance diagnostics. Derived descriptive statistical parameters comprised minimum deviation (from reference), maximum deviation, mean absolute error, mean absolute percentage error, median, and interquartile range. Pearson and Spearman correlation coefficients were calculated between measured and reference heart rates, along with their coefficients of determination (R²) and levels of significance.

To verify the accuracy of the GNSS-based distance measurements, the distances measured by the smartwatches were compared to the true reference distance. In particular, descriptive statistics included the arithmetic mean, minimum, maximum, mean absolute error, mean absolute percentage error, standard deviation, and interquartile range. To test for statistical significance of possible deviations between measured distances and the reference value, t-tests were carried out with effect sizes characterised by Cohen’s d. To compare the accuracy of the watches among each other, a one-way repeated-measures ANOVA was conducted with the smartwatch model being the independent variable. In addition, a t-test was conducted for the 4,000-m stadium runs to investigate if wearing the watch on the left or right forearm (i.e. inside or outside the lane) had an impact on the measured distance.

For the swimming tests, the number of metres swum by the participants, the number of strokes used, and the SWOLF index were evaluated. The SWOLF value is the time in seconds plus the number of strokes required to swim a given distance, i.e. SWOLF = time in seconds/lap + strokes/lap. Descriptive statistics comprised: arithmetic mean, minimum, maximum, mean absolute error, mean absolute percentage error, standard deviation, and interquartile range.

For all statistical tests, the level of significance was set to p<0.05. If not stated otherwise, results are given in terms of mean ± standard deviation (SD).

3. Results

3.1. Heart Rate

The mean absolute errors (MAEs) of the average heart rate measurements, as measured for all of the smartwatches, were between 4.2 and 11.8 beats per minute (bpm). This corresponded to mean absolute percentage errors (MAPEs) of 3.1% to 8.3%. The coefficient of determination of the average heart rate was between 0.220 and 0.705. The Spearman correlation coefficients ranged from 0.436 to 0.841 (p<0.001), indicating medium to large associations. In detail, the model-specific results can be found in Table 3, and the respective box plots are shown in Figure 1, with the regression analysis in Figure 2. All regression lines have a lower gradient than the reference lines. Reviewing the data for all 30 runners, the MAEs for each runner and all 10 watches combined ranged between 1.1 and 19.1 bpm, which corresponds to MAPEs of 0.8%–17.0%.

The maximum heart rate measurements showed similar MAE values compared to the average heart rate, i.e., between 4.4 and 10.9 bpm, resulting in MAPEs between 3.1% and 7.3%. The coefficients of determination for the maximum heart rates were also in a similar range as the average heart rate, i.e., between 0.179 and 0.743. The Spearman correlation coefficients equally showed only small differences, ranging between 0.482 and 0.851 (p<0.001). The results are presented in Table 4, the box-and-whisker plots in Figure 3, and the regression analysis in Figure 4. All regression lines had a lower gradient than that measured by the heart rate monitor. Based on the analysis of the data for all 30 runners, the MAEs for each runner and all 10 watches combined ranged between 0.9 and 29.1 bpm, which is equivalent to MAPEs of 0.6%–24.7%. In summary, the maximum heart rate measured by the smartwatches showed greater variation than the average heart rate.

The most accurate smartwatches in terms of both the average and the maximum heart rates were the APP and the HUA with MAPEs of 3.1%/3.1% and 3.3%/3.4%, respectively.

3.2. GNSS-Based Distance

3.2.1. Stadium Track Running Tests

The results of the 4,000 m track running tests are summarised in Table 5 and Figure 5. Altogether, seven smartwatches varied, on average, by less than 80 m (<2%) from the true distance. The mean MAPEs were between 0.8% and 12.1%. The FIT severely underestimated the true distance in every measurement (3,515 ± 381.20 m, p = 0.026, t(5) = -3.12, Cohen’s d = -1.272), whereas the FOS (4,284 ± 150 m, p = 0.006, t(5) = 4.66, d = 1.903) and XIA (4,140 ± 123 m, p=0.038, t(5) = 2.79, d = 1.139) returned significantly too high values. For all three watches, the effect sizes in terms of Cohen’s d indicate large mean deviations from the true value. Overall, a strong effect of the smartwatch model on the measured distance was confirmed by the ANOVA (p<0.007, η² = 0.509). The post-hoc comparisons showed that 9 (out of 45) deviations from the true distance were significant among the individual smartwatch models (p<0.001–0.013), all of which were for the FIT compared to the other models. Regarding body side dependence, the mean distance measured by all smartwatches when worn on the inner wrist (T1 and T2), was 3,946±272 m (n=30), as compared to 4,052±249 m (n=30) when worn on the outer wrist (T3 and T4). Comparing these values, the difference of 106 m (2,7%) was not significant (p=0.094).

3.2.2. Hilly Outdoor Running Test On Asphalt

The results of the outdoor running test on a hilly asphalt course with a true length of 3,410 m are summarised in Table 6 and Figure 6. In general, these measurements were more accurate than those for the 4,000 m track test, with the MAPEs ranging from 0.2% to 7.5%. The most accurate smartwatch, in this respect, was the GAF, which had an MAPE of 0.2%. Three smartwatches had SDs less than 10 m: APP, GAF, and GAV. Four significant deviations (p<0.001–0.049, t(5) = -11.62–4,66,) were found, as well as large effect sizes in terms of Cohen’s d of 1.90 to -4.74, for AMA, APP, FOS, and SAM. The ANOVA confirmed that smartwatch model had a large effect on the measurement accuracy (p<0.001, η² = 0.263). The post-hoc analysis showed 5 (out of 45) significant differences when comparing the 10 models among each other (p<0.001–0.021), affecting the FIT and POL. Comparing the measurements taken on the inside wrist (i.e., the right wrist in the clockwise direction along the loop) with those on the outside wrist (i.e., the left wrist), the difference of 33 m (1,0%) between 3,384±181 m (n=24) and 3,417±112 m (n=24) was smaller than that for the 400 m track and not significant either (p = 0.195).

Regarding the tracked elevation profile, all watches that offered this analysis readily via an online platform (i.e. AMA, FIT, GAF, GAV, HUA and POL) tracked the relative elevation changes adequately throughout the course (Figure 6 (d)). However, AMA und HUA returned strongly imprecise absolute elevations (error of approximately 60 to 120 m).

3.2.3. Road Cycling Course

The results for the road cycling course with a true length of 36.84 km are shown in Table 6. The route and elevation profile are shown in Figure 7. All ten smartwatches underestimated the reference distance cycled by participants but were still more accurate than the GNSS measurements recorded while running, with nine watches exhibiting MAPEs below 1%. The most accurate watch was the POL, with an MAPE of only 0.03%.

Seven significant deviations were observed for the ten watches (p <0.001–0.023, t(2–3) = -44.33– -4.29,), with a large effect sizes in terms of Cohen’s d of -22.167 to -2.144. The three watches that had larger standard deviations, or fewer measurement values that could be analysed (POL, FOS, and FIT) due to self-aborted measurements by the watch were not significant. The ANOVA confirmed that smartwatch model had a large effect on the measurement accuracy (p<0.001, η² = 0.775). The performed post-hoc analysis showed 12 (out of 45) significant differences among the 10 models when compared to each other (p=<0.001–0.049). Similar to the hilly outdoor running test, absolute elevations and cumulative vertical metres climbed were substantially inaccurate for AMA and HUA (Figure 7 (e)), apparently due to an altitude offset of approximately -84 m and -156 m, respectively, and because of highly noisy or smoothed elevation profiles.

3.3. Pool Swimming

Altogether, the smartwatches offer various swimming modes which are compared in Table A1. The FOS, however, lacks a swimming mode; therefore, only nine watches were included in the analysis of the distances tracked during the swimming tests. An evaluation of the 400 m front crawl trial, as presented in Table A2, shows that seven smartwatches recorded correct measurements. Two watches had MAPEs of 4.2% (Fit and POL), as, in each case, one of the three recorded 450 m instead of 400 m.

The results of the 200 m individual medley trial with stroke transitions every 50 m show that the distance tracked had a higher propensity for error when the swimming stroke changed than during a continuous front crawl stroke. Six smartwatches measured the distances correctly, and the three watches that recorded incorrect distance had the following MAPE values: AMA, 8.3%; FIT, 16.7%; POL, 141.7% (Table A3).

For the 200 m individual medley with stroke transitions every 25 m (in the middle and at the end of the lap in the 50 m Olympic-size pool), none of the watches were able to measure the distance correctly (Table A4). The MAPE was between 41.7% and 91.7%.

Five watches provided a functionality to analyse the SWOLF index, i.e., were able to also detect the number of strokes per lap. These values were collected during the 400 m front crawl trial. The evaluation of the number of strokes at 50 m as part of the SWOLF index is shown in Table 8. The MAPE was between 0.4% and 29.5%.

4. Discussion

The aim of this study was to compare the accuracy of 10 different smartwatches when measuring the heart rate during running exercises; the distance during running, cycling, and swimming tests; and the stroke rate during the swimming trials.

The results of the heart rate measurements were divided into two parts: the average heart rate measurement and the maximum heart rate measurement. The average heart rates had MAPEs between 3.1% and 8.3% while the MAPEs of the maximum heart rates ranged from 3.1% to 7.3%. The heart rate readings, therefore, were more accurate for the maximum values. A factor that could contribute to differences here is the time range, and hence number of data points over which the various parameters were calculated. For the maximum heart rate, a single maximum spike value was used, while for the average heart rate, values constantly recorded over a 3-minute interval were averaged. Another contributing factor could be that the measurement intervals between the watches and the chest strap reference were a few seconds apart in some cases. All regression lines had a gradient lower than the reference for all watches. High values tended to be underestimated, and low values tended to be overestimated. The accuracy of the heart rate measurements performed at the wrist fell short of those achieved with the chest strap. Similar average heart rate MAPEs were recorded compared to values in a study by Chow et al., in which the MAPE deviations in the heart rate measurements during a treadmill exercise were 2.5%–8.3%, although that study analysed the heart rate second by second [47]. Interestingly, some runners exhibited only slight deviations from the reference values, and this was consistent for all tested smartwatches. For other runners, all or nearly all of the 10 smartwatches showed substantial deviations. This can be seen in the MAPE values calculated for each athlete, with values of 0.8%–17.0% for the average heart rate and 0.6%–24.7% for the maximum heart rate. If these values for the average heart rate measurements are divided into three ranges—below, within, and above the MAPE values—for the individual smartwatches, the following distribution emerges: nine runners had MAPEs between 0.8% and 3.1%, sixteen runners had MAPEs between 3.2% and 7.7%, and five runners had MAPEs from 10.1% to 17.0%. This indicates that it is not only the accuracy of the smartwatch that influences the result but also the individuality of the runner wearing it. A possible reason could be due to an interference of the photoplethysmography-based measurement concept, whereby an external movement alters the signal strength of the light reflected by the tissue instead of solely the blood flow. In this case, false external frequencies might be measured, such as the step frequency. Future studies could, thus, investigate whether there are non-physiological associations between the supposed measured heart rate and the step frequency.

Three significant deviations for the 4,000 m run on the 400 m track were observed. The most accurate watch (GAF) recorded a maximum deviation of 120 m, whereas the least accurate watch (FIT) displayed a deviation of 1,020 m. The GAF has a track running mode, and it showed the least deviation, with an MAPE of 0.8%. Overall, the MAPEs were between 0.8% and 12.1%. The showed results that fell within the range of MAPEs that were recorded in a study by Budig et al., with an MAPE of 1.8% for one smartwatch model [13] and MAPEs of 1.4%–1.9% for two models [18], as well as another study by Gilgen-Ammann, with MAPEs of 0.9%–4.1% for eight watch models [15]. In addition, the difference between all measurements taken on the inside curve of the wrist (3,946 m) and on the outside (4,052 m) was 106 m. Considering the geometry of the 400 m track, the theoretical difference would be 7.04 m per lap for a runner using the second lane and not the first. For the 10 laps comprising 400 m each, this difference would be 70.4 m with a track width of 1.22 m [45]. The distance between a runner’s two wrists measured at rest is certainly smaller; however, in motion, distance between wrists may result in values that differ by up to one lane span as the upper limit. The measured difference of 106 m between the inside-facing watch and the outside-facing watch is, therefore, greater than the difference between lane 1 and lane 2 as 70.4 m over 10 laps. Smartwatch models are available that can differentiate wrist sides, and this could be advantageous for more accurate measurements of distance at curves. Presently, the various manufacturers do not reveal whether they use this information for a more accurate measurement. Future research is indicated to calculate this effect more precisely, even though its practical importance seems limited within the framework of triathlon training.

For the results of the 3410 m hilly running course on asphalt, which showed four significant differences, the mean MAPEs were between 0.2% and 7.5%. Results in a similar range were found for the MAPEs in the study by Budig et al., with 2.8% for one smartwatch [13] and 0.7%–4.8% for two models [18], and Gilgen-Ammann reported MAPEs of 3.5%–8.5% based on eight watches [15]. However, the measured values varied greatly among some of the watches. With different routes taken in a variety of environments, the MAPEs could not be properly compared because of disturbance variables (e.g., forest, urban canyons, and mountains), but they were in a similar range. The difference in the results between the watches attached facing the inside of the curve and those attached facing outward was 33 m, which is less than that of the 400 m stadium track. However, the outdoor running track had fewer curves than the stadium track.

During cycling, MAPEs between 0.03% and 4.2% were calculated. Nine models had an MAPE <1.0% (the exception was the FIT, with only one result). This corresponds to a deviation of fewer than 10 metres per kilometre. This level of accuracy required that the tyre circumference and air pressure, combined with the weight of the rider, be set very precisely using a classic bicycle odometer to achieve a greater accuracy than some of the GNSS measurements. Nevertheless, seven out of nine measurements showed a significant deviation from the true reference distance. This can be explained by the fact that all of the watches, without exception, measured the reference distance as too short on average and, additionally in almost all of the individual measurements. This can likely be explained by the fact that GNSS distance measurement consists of many individual points. On a curved surface, this means that the measurement consists of many short straight lines as the lower boundary of the actual pathlength of the curved trajectory. At higher speeds (e.g., when cycling), this measurement inaccuracy increases as the short straight lines become longer. Comparing the MAPE values with those in Budig’s studies (MAPE of 0.5% for one smartwatch model and MAPEs of 0.3% for two model) [13,18], nine of the ten watches in this study were in similar ranges. Although they had similar route lengths of 31.5 and 36.7 km, they differed in their locations and, possibly, in the elevation profiles.

According to the findings of the swimming trials, the distance of 400 metres was measured correctly by seven out of nine of the watches. The MAPEs were between 0.0% and 4.2%, with only two watches measuring one excess lap (FIT and POL). These MAPEs were lower than those recorded in Lee’s study, in which the MAPEs were between 0.0% and 20.6%, split among the different speeds [48]. Two different watches were used in that study. However, similar results were obtained in Budig’s study, reporting MAPEs of 0.4% and 4.6% for two models in the 500 m breaststroke [18]. In the 200 m medley swim, six out of nine watches recorded this distance without error. This corresponds to MAPEs of 0.0%–141.7%. Upon the addition of a stroke transition after 25 m in the centre of the lane, no smartwatch recorded the lap count correctly. The stroke transition in the middle of the lap was usually counted as the start of a new lap. This resulted in MAPEs between 41.7% and 91.7%. In another study by Brunner et al., swimming stroke changes performed in the centre of the lap were described as a mixed style, and greater accuracies were also reported [49]. This could not be confirmed with the models in the present study. When measuring the number of strokes, as a main contributor to the SWOLF index, the MAPEs were between 0.4% and 29.5%. These values per lap were recorded for five watches. The range of values are comparable to those in a study by Lee et al., in which the MAPE values were 6.2%–17.6% [48].

5. Limitations

There are some limitations to the present study that need to be named. First, the heart rate measurements were carried out under controlled laboratory conditions on a motorised treadmill at a constant speed, with no external disturbance factors or changes in the running speed. The heart rate readings were, thus, as expected, mostly constant over time. Moreover, heart rates were evaluated only in terms of the means and maximum values and not by continuous comparison. However, before further field studies can be carried out, tests under laboratory conditions are required.

Second, regarding the GNSS measurements, the number of runs and bike rides per watch was limited due to the availability of athletes at the same time, bearing in mind that the measurements had to be carried out at the same day and roughly the same daytime to ensure identical GNSS satellite conditions for all bouts. Future research should try to increase the number of parallel measurements by increasing the number of identical smartwatch models and athletes available. Furthermore, it could not be determined for all watches which of the global satellite systems—GLONASS, Galileo, or BDS—were used in addition to GPS to determine the position. Despite that, conclusion can still be drawn as to whether the measured distances are valid.

Third, this study investigated only the accuracy and thus practicability for the three triathlon disciplines during training in a separate manner. A multisport functionality for brick sessions or triathlon competitions could not be evaluated across the smartwatches as it was available only for a small minority of the models tested.

6. Conclusions

The results of this study show that the tested smartwatches differed substantially in their accuracies. In particular, optically measured heart rates can deviate considerably from the true values. Comparison with an ECG or a chest strap may be helpful before relying on such wrist-based measurements. As regards distance tracking in running, a conventional GNSS approach still cannot be recommended for measuring the distance run on a 400 m stadium track. Despite all the technical improvements over the last years, manually or automatically counting the number of laps appears to be still the more accurate approach. However, a smartwatch with specific track mode functionality (as provided by GAF) can overcome this issue and provide sufficiently accurate results for most practical purposes in triathlon and long-distance running exercise. Depending on the smartwatch model, using GNSS for outdoor measurements of distances run or cycled is sufficiently accurate in the context of long-distance running and triathlon exercise control. Not only when measuring personal bests, errors can also occur, and the results should always be critically scrutinised. As for swimming, most of the tested smartwatches were able to record the distance swum in the front crawl 400-m pool swimming trial with sufficient accuracy and were also able to correctly count stroke transitions when performed at the end of a lap. However, our results indicate that current smartwatches are not suitable for the demands of frequent stroke transitions in swim trials. In addition, the stroke rate was reproduced accurately by only a small subset of the watches (HUA and SAM). In essence, all non-temporal values measured by current sports smartwatches should be critically assessed for validity before being used in exercise control, but once their accuracy is confirmed, they can be a useful tool in training management for triathletes and coaches.