Evaluating the Usefulness of a PNT Solution Using DGNSS-SBAS for Canoe Slalom: Simulated and Real-World Analysis

1. Introduction

The rules of canoe slalom [1] require athletes to race in a time trial fashion (fastest time wins), over a predetermined course consisting of 18-25 gates, set over a section of river length 200-400m containing natural and/or artificial obstacles. Each gate consists of two poles 1.2-4.0 m apart, which are colour coded for the direction the athlete must pass. 6-8 gates are red in colour and negotiated in an upstream direction, the remainder are green and negotiated in a downstream direction. In all instances, to be deemed successful, negotiation of a gate must entail the head, shoulder and part of the boat passing through in the correct direction.

As such, canoe slalom performance is defined by the athletes cognitive abilities [2] combined with technical skill [3,4] and physical components [5]. The motor co-ordination of which is based upon the ability to process external stimuli [2], react through the appropriate timing of force magnitude and its rate of application to the footrests, seat, and paddle shaft [6]. For a video demonstration of a competition run https://doi.org/10.6084/m9.figshare.7379948.v2. These qualities culminate in athlete trajectory, where time to negotiate gate sequences is a product of path length and speed, and correlates highly with canoe slalom performance [3]. Investigations into short sections of a course, via aerial video tracking, within a race run, have shown that the shortest trajectory usually leads to the best canoe slalom performance [3,7] but little is known about a complete competition run. In this instance, further research needs to explore technological solutions to facilitate more invasive athlete position tracking to better understand strategies used by top athletes [3] in competition, and also to monitor the development of canoe slalom paddlers performance status.

Global navigation satellite systems (GNSS) are typically used in such instances but are deemed to provide low power signals making them susceptible to noise [8]. These errors typically centre around clock and/or system errors as a result of multipath environments which reflect, delay or interfere with the signal(s) [9]. Particularly problematic for sports performed in nature or built up areas are signal obscuration by buildings, mountains, or bush, at the receiver end [10]. Consequently, a single constellation channel device is likely only within 5m error 95% of the time [11]. The utilization of multiple GNSS constellation channels enhances both static [12] and kinematic [13] positional accuracy. Furthermore, the integration of Satellite-Based Augmentation Systems (SBAS) enables centime-level accuracy in static mode and decimetre-level accuracy in kinematic mode [14]. In addition, the incorporation of inertial measuring units (IMU’s) and ground-based reference stations further enhances overall position accuracy, precision and signal integrity [15].

Typically, in team sport situations, tightly coupled integration units combining GNSS with IMU’s are used to overcome such limitations to record speed and distance data in athlete movement with good (<5%) reliability [16,17,18]. However, these Positional Navigational Timing (PNT) solutions tend not to provide information regarding positional accuracy, availability, integrity, and service continuity. Such systems, utilising GNSS channels (GPS (L1 and L5 band) and GLONASS (L1 band)) + external IMU’s have been effectively used in canoe slalom for measures of speed [19], but data processing is out of reach for most consumers at this time, and no information regarding the positional or kinematic accuracy, precision, and integrity, or service continuity has been provided. Considering that athlete positioning is such an important aspect of canoe slalom performance, reflecting strategy and technique [20,21], it would be highly beneficial to be able to quantify this aspect of performance accurately, and with precision. Recently, a single-frequency (GPS) PNT solution with a sampling rate of 10Hz reported a discrepancy of -8.9% between the measured total distance and the actual total distance travelled during a simulated dry-land slalom. This resulted in a 3.2% deficit compared to aerial tracking in the same dry land simulation, and a 5.5% difference when comparing the PNT solution to aerial tracking in a real-world slalom trial [22]. This is not surprising as the accuracy of GNSS degrades significantly when conditions become challenging [23] such as non-linear movement patterns of the athletes, and multipath impact [24]. Additionally, sports such as canoe slalom involve frequent direction and acceleration changes [5] where positional sample frequency ≥20Hz is essential to obtain useful trajectories to distinguish differences in performance [24]. The latter has been overcome in sports similar in nature to canoe slalom, such as slalom skiing, where real-time kinematic (RTK) positioning was shown to provide accurate positional data (<10cm) along with sport-specific performance parameters such as distance travelled, speed and acceleration [24]. DGNSS with real-time base station corrections could be used to determine athlete position to sub-meter accuracy [25]. As such, it is likely that differential methods to GNSS could be used in a real-world scenario for slalom canoeing to enable performers or coaches to quantify trajectory from a performance or developmental perspective.

The purpose of this study was to assess the positional accuracy, availability, integrity, and service continuity of a commercially available PNT solution that uses DGNSS-SBAS multiple GNSS channels while sampling at a frequency of 20Hz during a simulated dry-land slalom and real-world on-water slalom. The primary objective was to assess the PNT solution during the dry-land slalom while observing the efficacy to differentiate between trajectories. Furthermore, the simulated results were compared with a non-controlled, on-water, real-world slalom test to validate the findings. Based on a minimum required accuracy for the sport, and pilot work with the PNT solution being used, it was hypothesised that the PNT solution would provide useful (<0.3m) horizontal accuracy and reliability under both simulated and real-world conditions for slalom canoeing.

2. Materials and Methods

The primary objective of this study was to evaluate the positional accuracy, availability, integrity, and service continuity of the PNT solution. Therefore, the focus was primarily on assessing the performance of the device rather than analysing specific test protocols or athlete performances [22]. Additionally, because of an inability to precisely follow a set trajectory through slalom gates of known distance it was decided to perform multiple (n=25) runs over two different trajectories during a dry-land simulation. These trajectories, namely the short or more direct path, and long or less direct path, were chosen to simulate performer variability (Figure 1A) typical of slalom canoeing [22]. This was then followed by an assessment of the device’s performance during an uncontrolled on-water slalom for real-world comparisons (Figure 1B).

This sample size was based on previous research [22] conducted in a similar context, where aerial video tracking showed significant differences but remained practically useful for canoe slalom. Using this data, a priori statistics were generated (G*power V 3.1.9.7, Heinrich-Heine University,Dusseldorf, Germany) for t-tests of mean differences between two dependent matched means. Calculations were based on an equivalence or non-inferiority test rather than a traditional hypothesis test.

It was decided that a difference of 1.0±0.5m in trajectory was acceptable, resulting in a required sample size of n=6. However, considering the sample size used in previous studies [22], our need for increased reliability, and the potential for greater variability in real-world settings, a sample size n=25 was chosen.

For all data collection one DGNSS device (Arrow 100 GNSS receiver, Eos Positioning Systems, Inc.^®, (Eos), Montreal, CA) was used in combination with one experienced (35 yrs. national/international experience), male kayaker in accordance with the University Human Ethics Committee.

The GNSS receiver was positioned on the participant’s canoeing helmet, over the frontal-sagittal border of the parietal and frontal bones of the skull, to minimise satellite reception impediment [11], but also because international federation rules [1] stipulate the head must pass through the gate-line. As such, placement on the boat would likely augment any discrepancy or misrepresentation of athlete trajectory, limiting data value. The PNT solution utilised in this study made use of SBAS SouthPAN (Satellite-Based Augmentation System South Pacific) in combination with all available GNSS constellation channels, including GPS, GLONASS, Galileo, Beidou, and QZSS. The device could receive differential corrections through NTRIP (Networked Transport of RTCM via Internet Protocol) GNSS data obtained from PositioNZ base station(s). Specifically, the differential corrections were sourced from the mount point DNVK00NZL0. The mount station was located approximately 41 km from the location of both trials. Prior, unpublished data showed that this distance for these venues did not affect the accuracy, availability, integrity, and continuity of the PNT solution outside the optimal range required (Table 1).

Where, Hrms = Horizontal Root Mean Square, PDOP = Positional Dilution of Precision, HDOP = Horizontal Dilution of Precision.

GNSS signals and measurement extraction were processed via EOS Pro Tools application (Eos Positioning Systems, Inc.^®, (Eos), Montreal, CA) using propriety algorithms. This application provided UTC Time, Longitude, Latitude, Height above sea level, Speed, Hrms, Vrms, 3Drms, Satellites in view, satellites used, PDOP, VDOP, HDOP, and differential age throughout the study. For real-time data logging, EOS pro Tools connected via a third-party mobile iOS app called ArcGIS QuickCapture (ESRI, Redlands, CA, USA). The app facilitated real-time logging of position data at a sampling frequency of 20 Hz, resulting in a high-resolution dataset for analysis.

• All post-processing of logged raw data took place in ArcGis Pro (v3.2.0, ESRI, Redlands, CA, USA) where five steps:
• Load a georeferenced aerial image (DJI Matrice 300RTK, Zenmuse P1 35mm with 45 MP Full-frame sensor, Shenzhen, China).
• Add slalom gates to the georeferenced image as a layer on top of the actual gates within the image.
• Load data from the PNT solution stored within the ArcGis Cloud.
• Perform a geographic transformation from NZGD_2000 to WGS_1984_UTMZone_60s.
• Export to data to statistical analysis software (Graphpad prism (V6.0f)).

2.1. Test 1: Dry land Slalom.

The dry-land slalom simulation was performed in a car park during a holiday period where multipath interference was considered low. The environment was devoid of reflective surfaces or buildings exceeding a height of 20m. Pilot work conducted prior to the simulation confirmed excellent signal reception at the venue, with an average of 25 ± 1 satellites, 95% CI 24.6-24.7. The horizontal accuracy was found to have high precision, with a horizontal route mean square (Hrms) of 0.10 ± 0.00 m, (95% CI 0.099 – 0.102). Additionally, the stability of the positioning solution was demonstrated by a Positional Dilution of Precision (PDOP) value of 1.12 ± 0.04 (95% CI 1.12-1.13) over a 25 s duration of slalom.

From a stationary start, at a predetermined position marked on the surface as start, and when the device had met the predetermined Hrms threshold of less than 30 cm, the participant followed one of two painted lines on a black asphalt surface configured to replicate a typical slalom trajectory dictated by a set course of gates (Figure 1A). Slalom poles were replaced by two surface markers 1.2 m apart as per International Canoe Federation Slalom Rule 27 for spacing of gate poles. Each trial finished when the participant reached the finish marker and remained stationary until data logging ceased. The total length of each trajectory as per tape measure method of measurement was 47.4 and 57.9 m for the short and long trajectories, respectively. The participant completed each trajectory on foot (n=25), where time, longitude, latitude, speed (m∙s^-1), direction of travel (°), compass reading, horizontal accuracy (Hrms), PDOP, Horizontal Dilution of Precision (HDOP), and the number of satellites being used were recorded at a frequency of 20 Hz.

2.2. Test 2: On-water Slalom

The aim of this test was to confirm the validity and reliability of test 1 in a real-world setting for canoe slalom compared to the simulated dry-land slalom. The test was performed on a flatwater, grade I slalom venue (Centennial Lagoon, Palmerston North, New Zealand) where the participant negotiated the set course (Figure 1B) and completed 25 trials. The course was situated in an environment with no buildings within a 50 m radius. The tree cover was limited, with no trees located within 25 m of the course. The highest nearby tree had a height of less than 30 m. As per the simulated trial speed was not controlled but the participant was instructed to vary the speeds and trajectory as might be expected with different performance levels of canoe slalom athlete.

2.3. Data Analysis

Descriptive statistics, including the number of samples or data points (N=), mean ± standard deviation (95% confidence interval), and frequency distribution data were calculated for each trial. These calculations were used to determine overall mean ± SD for various parameters, including the time taken to complete each trial (s), the distance covered (m), the number of satellites used as an indicator of availability and quality of satellite signals received by the receiver, the horizontal accuracy measured by the Hrms, and the integrity assessed by the geometric quality of satellite configurations represented by the HDOP and PDOP metrics.

Subsequently, the dilution of precision data was categorised based on a rating system [23,26]. In this system, ratings were assigned as follows: values less than 1 were considered ideal, values between 1 and 2 were categorised as excellent, values between 2 and 5 were classified as good, values between 5 and 10 were deemed moderate, values between 10 and 20 were considered fair, and values greater than 20 were categorised as poor.

For the dry land slalom total distance (m) covered (test 1) was compared with the known distance for each trajectory and the known measure of that trajectory path using a two-tailed one sample t-test. Differences between all grouped dependent variables (Mean, SD, N) for the short and long trajectory were assessed using a non-paired t-test with Welch’s correction if data did not pass D’Agostino and Pearson’s normality test.

Comparison between dry-land trajectories and the real-world on-water slalom for all mean trial dependent variable data was performed using a one-way ANOVA with Holm-Sidak’s multiple comparison’s test with single pooled variance. If the data did not pass the normality test, then a Kruskal-Wallis non-parametric one-way ANOVA, was employed. Post-hoc pairwise comparisons were conducted using Dunn’s multiple comparison test to assess differences between paired tests. All statistical analysis were performed using Graphpad Prism (V6.0f) with the alpha value set at p=0.05.

3. Results

3.1. Test 1: Dry-land Slalom

The participant completed 25 runs for each trajectory (short/direct and long/less direct, Figure 2B) which were significantly different for distance (46.95±0.31 and 57.21±0.22 m, t₍₅₇₎=131.5, p<0.0001) and time (t₍₅₇₎=3.998, p=0.0002) taking 51.21±5.88 and 57.52±5.13 s, respectively. The starting threshold for the horizontal accuracy (Hrms) was met on every occasion and resulted in a mean ± SD (range) of 0.22 ± 0.04 (0.21-0.24) m and 0.23 ± 0.06 (0.20-0.25) m, for the short and long trajectories, respectively.

Overall, the PNT solution data, processed via ArcGIS Pro, consisted of 24,008 data points, with no data set errors determined through visual inspection of trajectory plots. The number of satellites within the constellation, and thus usable by the device were 20 ± 1 (17-21) and resulted in HDOP 0.68 ± 0.03 (0.63-0.78), and PDOP 1.36 ± 0.07 (1.20-1.58). According to the classification [26] of dilution of precision, HDOP was ideal, VDOP excellent, and PDOP excellent. The overall Hrms value was 0.23 ± 0.02 (0.18-0.28) m.

When data were analysed and compared between the short and long trajectories (n=10,310 and n=13,698), the mean±SD (95% CI) for Hrms were 0.23 ± 0.04 (0.21-0.25) and 0.24 ± 0.05 (0.22-0.26) m, respectively. The Hrms values were normally distributed (K2=1.149, p=0.563 and K2=1.456, p=0.483) and showed no significant difference (t₍₅₀₎=0.395, p=0.695). The number of satellites used within the constellation by the device had normal data distribution (K2=2.838, p=0.242 and K2=3.97, p=0.137) but differed significantly (18 ± 2 (17-19) and 19 ± 1 (18-19), t₍₅₀₎=2.504, p=0.016). The subsequent effect of satellite geometry led to differences for the normally distributed (K2=4.572, p=0.102 and K2=0.021, p=0.990) HDOP data (0.74 ± 0.09 (0.70-0.78) and 0.69 ± 0.03 (0.68-0.70), t₍₅₀₎=2.673, p=0.011) and not normally distributed (K2=16.31, p=0.0003 and K2=4.944, p=0.084) PDOP data (1.45 ± 0.22 (1.36-1.54) and 1.31 ± 0.10 (1.27-1.36), t₍₅₀₎=2.749, p=0.009) for the short and long trajectories, respectively. As per the overall classification of precision of error, the HDOP was ideal, and PDOP excellent. Frequency distribution of horizontal accuracy and dilution of precision data is shown in Figure 3.

A one-sample t-test with comparison to the known total distance (tape-measured distance) of the two dry land slalom course trajectories (short 47.3 and short 57.8 m) showed significant difference (Figure 4A) for the short trajectory (46.95 ± 0.30 m, 95% CI discrepancy ^-0.48 - ^-0.22 m, CV=0.65 %, t₍₂₉₎=5.741, p<0.0001), and the long trajectory (57.21 ± 0.22 m, 95% CI discrepancy ^-0.69 - ^-0.50 m, CV=0.38%, t₍₂₉₎=12.91, p<0.0001). However, the overall percentage differences from the measured distance were small for both the short (-0.74 ± 0.64%) and long (-1.09 ± 0.42%) trajectories (Figure 4B).

3.2. Test 2: On-Water Slalom

The participant performed n=25 on-water slalom runs on the same course with a mean±SD (range) duration 53.35 ± 3.31 s (46.35-64.00) with a total of 1,329 data samples. Data was calculated using 26 ± 2 (24-31) satellites within the constellation, where athlete trajectory was 96.25 ± 1.52 m (94.14-102.1) with an average speed of 1.81 ± 0.09 m∙s^-1 (1.60-2.05). Upon visual inspection of geographical trajectory, no errors were identified in the dataset for the on-water test. The mean ± SD (95% CI) accuracy at the start of the test was determined to be 0.315 ± 0.109 (0.265-0.364) m. The overall Hrms was 0.35 ± 0.05 (0.349-0.354) m with a corresponding HDOP 0.56 ± 0.03 (0.555-0.558), and PDOP 1.13 ± 0.03 (1.123-1.236). Frequency distribution of Hrms and dilution of precision data is shown in Figure 5 where the classification [23] of dilution of precision was ideal for HDOP, while it was excellent for PDOP.

Comparison between dry-land simulation and on-water tests using a Kruskal-Wallis one-way ANOVA presented significant differences for the number of satellites used within the constellation (H_(3,75)=22.15, p=0<0.0001, Figure 6A) no significant differences for Hrms (H_(3,75)=4.036, p=0.133, Figure 6B), while significant differences were found for HDOP (H_(3,75)=24.71, p<0.0001, Figure 6C) and PDOP (H_(3,75)=39.49, p<0.0001, Figure 6D) even though all data was classified as ideal for HDOP and excellent for PDOP. Post-hoc analysis showed no differences between dry-land simulation courses (p>0.05) for any of the dependent variables while there were significant differences between the both dry land course simulations and the on-water test for satellites used, HDOP and PDOP data (Figure 6).

4. Discussion

The purpose of this study was to assess the positional accuracy, availability, integrity, and service continuity of a commercially available PNT solution that uses DGNSS-SBAS multiple GNSS channels during a simulated dry-land slalom and real-world on-water slalom. The main findings indicate that the PNT solution achieved accurate trajectory distances in a non-ecological environment, as they were within 1.5% of a known distance. The coefficient of variance was less than 0.5% suggesting consistent and reliable results. The dilution of precision measurements demonstrated ideal-excellent levels of position accuracy, while the horizontal root-mean-square was within 0.3m, indicating precise horizontal positioning. Furthermore, when considering real-world validity, the study revealed improved results for dilution of precision data due to an increase in the number of satellites used and presenting no differences for positional accuracy, availability, and integrity of the PNT solution.

In order to assess method validity, a dry-land simulation typical of slalom canoeing was performed [22], where residual analysis of all data points were visually inspected within ArcGis pro via a georeferenced aerial image overlay as per Figure 1A & 2B. Visual data inspection of each trial identified no data set errors for positioning in relation to the known trajectories used (Figure 2B), showing exceptional continuity of the PNT solution. The high number of satellites used (~18) in the dry-land simulation supports previous work that relates increased number of satellites in view, and used by the receiver, to increase the geometric dilution of precision [27]. The subsequent dilution of precision measures (Figure 3) supports the PNT solutions ability to use high quality satellite constellation and geometry in the horizontal plane where HDOP was ideal (<1) and reflects low levels of atmospheric or multipath interference, and/or receiver noise [8,26,28]. This is further supported from a 3D perspective through the PDOP values which are slightly higher and suggest further research needs to investigate altitude changes from a slalom perspective on a whitewater course where a gradient, while small, is present. In this data set, where no elevation changes occurred, the primary focus was on horizontal accuracy. The reported Hrms (0.23 ± 0.02 m) , while not as good as that reported in skiing [25], still provides a reasonable level of certainty around geographical positioning of athletes during the slalom simulation. This is reflected by the high overall accuracy, with a deviation of less than 1.5% when compared to the actual distance. Additionally, the precision was equally high with a CV of less than 0.5% observed over 50 trials. It is likely that the distance of the base station mount (41 km) from the receiver could influence the accuracy. However, pilot work with this PNT solution found that the distance used in this study still provided useful data. If more accuracy is desired due to no base station nearby, then using a device with a mobile RTK (real time kinematic) function would be preferable. Future research needs to explore the relationship between base station distance from slalom venue and PNT accuracy requirements for slalom in this regard.

Within slalom, performance is determined to a certain extent by line choice or trajectory, which is influenced by cognitive planning and/or reactions to external stimuli in the moment [2,3]. Therefore, it is crucial not only to assess the ability of PNT solutions to accurately log geographical positions but also to differentiate between the various trajectories taken. The inclusion of two trajectories over the same dry-land simulated slalom course, with successful negotiation of all gates (Figure 1A), allows for quantification of the PNT solution and facilitates a comparison of athlete trajectories (Figure 2B). In this instance, even small positional changes across the entire course, as observed in the dry-land short and long trajectories, resulted in a significant difference of 10.5m in total distance travelled. Such a difference would have substantial effect on performance where a meaningful change is reported to be 0.4-0.6% [20]. The mean ± SD total distance travelled using the PNT solution was found to be less than 1.5% for both the short (direct path) and long (less direct path) trajectories (Figure 3). This level of accuracy surpasses what has been reported in many team sport investigations [16,18,29]. These results suggest that the PNT solution could provide valuable information to coaches and athletes regarding overall performance. Furthermore, the narrow 95% CI for horizontal accuracy suggests that more specific segmental data could be obtained, as previously demonstrated through aerial analysis of single gate sections [3].

However, frequency distribution analysis (Figure 3) emphasises the importance for analysts to understand the integrity and continuity of PNT solutions, as well as how data is interpreted. Despite maintaining high numbers of satellites used (Figure 3A), and ideal PDOP, HDOP values (Figure 3C-D), the spread of data (0.10-0.45m) in the Hrms (Figure 3B) indicates the need for cautious interpretation. Future work should focus on using this to assess performance, along with developing and validating analysis systems that can provide coaches and athletes invaluable information.

As the data from the dry-land simulations provided positive results regarding accuracy, precision, and functionality of the PNT solution in a controlled environment, testing on-water in a real-world uncontrolled situation took place (Figure 3B). In this instance an even greater number of satellites were utilised (Figure 5A). While data was collected at a similar time of day, the dry land and on-water trials were performed on different days, which may mean there were less usable satellites available [30] or less signal obscuration at the receiver or in this case the athlete end [10] for the dry land trials. Even with much higher numbers of usable satellites in the on-water trial (Figure 6A), there were no significant differences between the horizontal accuracy (Hrms) of the dry-land verses the on-water trials. Importantly, while not significant, there was a definite trend of increased horizontal accuracy on-water (Figure 6B) which is supported by a significant decrease in dilution of precision variables (Figure 6C-D). Even so, the use of dry-land simulation to test tracking devices in canoe slalom is warranted, and in this case the PNT solution provided high levels of accuracy, availability, integrity, and service continuity. On-water testing likewise supports its use as a quantitative tracking tool where future research needs to focus on the efficacy of DGNSS in canoe slalom along with methods of analysing segmental data