Less Is More: Higher Skilled Sim Racers Allocate Significantly Less Attention to Track Relative to Display Features Than Lower Skilled Sim Racers

1. Introduction

Simulated (sim) racing strives to recreate real-world motor racing experiences in a virtual environment (Paiva, 2015). Sim racing is one of the oldest esports and recently has gained much more popularity (Lefebvre et al., 2024). Tudor (2020) notes how sim racing came to replace real-world motorsport during the Covid-19 pandemic with Formula 1′s virtual grand prix series garnering approximately 7.5 million viewers over the course of eight events. When compared to real-world motorsport, sim racing lacks vestibular feedback as there are no g-forces acting upon the sim racer. Thus, in addition to noisy haptic feedback provided sometimes through their wheelbase and pedal set, sim racers must rely more on visual feedback to navigate their way along the virtual racetrack (Lappi, 2015). As a result of the high virtual speeds attained during sim racing, this visual feedback must be processed quickly, necessitating efficient visual attention behaviour.

Visual attention is important in many performance-based environments and the way visual attention is typically measured is through the use of eye tracking technology. The number of fixations, their duration and where they are allocated in one’s field of view all shed light on the amount and types of visual information processed by the observer (Williams & Ericsson, 2005). These gaze behaviours have been used to differentiate expertise in many performance contexts, including surgery (Law et al., 2004; Wilson et al., 2010), medicine (Dreiseitl et al., 2012; Bertram et al., 2013), and sport (Spitz et al., 2016; North et al., 2009). In sport in particular, perceptual-cognitive skills are a feature that differentiates expert and novice performers, with experts demonstrating the ability to more rapidly process information in the environment and produce relevant movements for successful task completion (Marteniuk, 1976). For example, in a recent review by Brams and colleagues (2019), it was found that in 10 out of the 19 studies examined, experts across various sports displayed a greater number of fixations at more locations (faster visual search rate) in comparison to intermediate or novice level performers. Interestingly, in the remaining visual search rate studies examined in the review, experts were found to have fewer fixations or fixations on a smaller number of locations (slower visual search rate). Therefore, it would seem that there is a lack of consensus on the visual behaviours of experts and novices, but findings from work by Vickers (1992; 1996) that expert performers exhibit fewer and longer duration fixations in comparison to non-experts when performing self-paced tasks seems to suggest that task type may contribute to these patterns.

Sim racing may be classified as either an X or Y type task depending on whether one is racing against opponents on the same track or performing a time-trial, whereby they are attempting to achieve their fastest lap on an empty track. Race situations are very much externally-paced. The actions of competitors must be anticipated and reacted to decisions need to be taken to initiate or defend overtakes. In contrast, during a practice or qualifying situation, the driver has time to prepare for a fast lap on their out lap (lap after exiting the pit garage) and during their fastest lap attempt, they dictate when to brake, turn in, and accelerate out of corners. Although a number of studies exist examining visual attention in real-world driving (e.g., Underwood et al., 2003) and simulated driving tasks (e.g., Robbins et al., 2019; Tuhkanen et al., 2023), no study has examined how visual attention is allocated by experts and novices in a simulated racing environment.

The purpose of this study is to determine if simulated racers of high and lower skill can allocate their visual attention differently when completing a lap of a racetrack. As we intend to examine a self-paced time trial, we firstly hypothesize that low skilled sim racers will have greater attention allocation than high skilled sim racers, evidenced by longer average and total fixation durations, and a greater number of fixations. However, to mitigate the effect that a longer laptime may have on attention allocation opportunity, we also hypothesize that, when metrics have been adjusted for lap time, that high skilled sim racers will allocate attention for longer durations in fewer locations than low skilled sim racers. This will be evidenced by high skilled participants exhibiting lower fixations per second and higher average and total fixations durations as a proportion of lap time compared to lower skilled participants. Thirdly, we hypothesise that high skilled sim racers will attend more to task relevant stimuli (e.g., the track ahead; TRACK) relative to irrelevant stimuli (e.g., heads up display (HUD) elements in-game), than less skilled sim racers. This will be evidenced by a higher ratio of fixations (TRACK:HUD) on track for high skilled sim racers compared to lower skill sim racers.

2. Materials and Methods

Participants

A total of 104 participants (Age = 30.84±9.51) were recruited at a sim racing convention (ADAC Sim Racing Expo Nuremberg, 2022). Approval for the study was authorized by the research ethics board at the University of Limerick in accordance with the Declaration of Helsinki.

Materials

Two Playseat Sensation (Playseat, NL) sim racing setups with Logitech G Pro Wheelbases and Pedals (Logitech; CH) were used for data collection. Both racing setups were powered by PC and information was displayed on 55-inch (Samsung) monitors with a refresh rate of 120Hz. We used the racing title Assetto Corsa Competizione (v1.8.18) (Kunos Simulazioni, 2018), in-game and hardware settings can be found in Supplementary File 1. MoTeC (v.1.1.5.0085) telemetry software (MoTec; AU) was used to capture participants telemetry data to ascertain their lap times. To collect gaze data, participants wore Tobii Pro Glasses 3 (Tobii AB, 2022) which sampled data at a rate of 50Hz. The eye tracking glasses were connected to a separate PC where they could be controlled via the Tobii Pro Glasses 3 Controller app.

Protocol

Participants filled out a demographic questionnaire that also gathered information regarding their previous sim racing experience. Following this, participants were then instructed to get into the optimal seating position for themselves in one of the sim racing setups. They then were given the eye tracking glasses to wear and a one-point calibration was performed. This one-point calibration has been found to be accurate to within 1.60˚ (Onkhar et al., 2023), where accuracy refers to the angular distance from a target.

Participants then drove 8 laps around the Brands Hatch circuit using the McLaren 720s GT3 (setup on the car could not be changed) as fast as they could. Following the completion of their eighth lap, the eye tracking recording was stopped.

Data processing

Following exclusions for incomplete eye tracking data, 88 participants remained. From this pool of participants, two groups were formed based on lap time. The fastest lap time each participant achieved was obtained from their telemetry data. Thereafter, we conducted a percentile split of fastest lap times across all 88 participants whereby the top 25% (high skilled) and bottom 25% (low skilled) lap times were segregated into two groups (n = 22 in each group) of examining visual attention behaviour.

In each participant’s eye tracking file two events were inserted to mark the beginning and end of the gaze data corresponding to their fastest lap while reviewing the video footage of their recording using Tobii Pro Lab v1.207 (Tobii AB, 2023). Using these two events, a time of interest (TOI) was created which included all gaze data during a participant’s fastest lap.

To ascertain where participants were looking during their TOI, areas of interest (AOIs) were overlaid on a still image (Figure 2), and fixation mapping to the still image was applied for each participant. The assisted mapping feature in Tobii Pro Lab compares the snapshot image with each frame in the recording. The algorithm looks for similarities in contrast and colour between the snapshot image and video recording. Gaze points are then automatically mapped from each frame to the snapshot according to a similarity score, where a similarity threshold can be set (in this case 50%), indicating how confident the algorithm is with the mapping. All automatically mapped points were reviewed and mapped manually if the fixation point in the video recording did not match the automatically mapped points following the assisted mapping. AOI’s for the heads-up display (HUD), which included lap time display, circuit map, in-car information screen, tyre and brake temperature display, and speedometer, and the whole view outside the car (i.e., the track ahead; TRACK) were created.

Before exporting the data from Tobii Pro Lab, the Tobii I-VT attention filter was applied to the data within the TOI. Further details on this filter can be found in Appendix B. The attention filter was used as it is recommended by the eye tracker manufacturer (Tobii AB, 2022, p.172) when using eye tracking glasses under dynamic conditions. Thus, exported calculations were based off the attention filter. Only data within the TOI outlined above was exported for further analyses.

Variables exported from Tobii Pro Lab for analysis included the following: total whole fixation duration (TFD; sum of all individual fixation durations across the fastest lap), average fixation duration (AFD; average duration of an individual fixation across all fixations for the fastest lap), fixation count (FC; total number of fixations across the fastest lap), these same variables normalised to each participant’s lap time (TFD_n; AFD_n; FC_n) and the ratio of fixations allocated on the track vs. heads up display (TRACK: HUD). Calculations for each normalised metric and the TRACK:HUD metric are provided below.

Normalised total fixation duration (TFD_n)

Eq. 1                                 

Normalised Average fixation duration (TFD_n)

Eq. 2                                 

Fixations per second (FC_n)

Eq. 3                                 

TRACK:HUD fixation ratio

Eq.4                                          FC_TRACK:FC_HUD

Data analysis

Statistical analyses were carried out using SPSS v.28.0 (SPSS, 2021). After removing outliers (fixations and average fixation durations exceeding 1.5 times the interquartile range; the data of 3 participants from the low skilled group were excluded), a Shapiro-Wilk test and investigation of Q-Q plots were performed on the dependent variables to verify the normality of the data.

To investigate whether low skilled sim racers displayed greater attention allocation than high skilled sim racers when metrics were unadjusted for lap time, we compared fixation count, average fixation duration and total fixation duration between high and low skilled sim racers using independent t-tests.

To investigate whether high skilled sim racers displayed greater attention allocation when metrics were adjusted for lap time, we compared fixations per second, average, and total fixation durations as a proportion of lap time between high and low skilled sim racers using independent t-tests.

To investigate whether high skilled sim racers attended to more task-relevant stimuli, we compared TRACK:HUD fixations between high and low skilled group using an independent t-test.

Sidak corrections were applied during the analyses as multiple t-tests were conducted for each hypothesis (significance alpha level p < .007). Effect sizes are reported using d and results as means ± SD.

3. Results

3.1. Skill level comparisons

Higher and lower skilled sim racers significantly differed in the number of hours that they raced per week (t(39) = 5.321, p <.001, d = 1.666), with high skilled sim racers spending 18.45±10.43 hours sim racing per week compared to the lower skilled groups 4.42±5.18 hours (Table 1). Furthermore, high and low skilled sim racers’ fastest lap times were found to be significantly different (t(39) = -8.820, p <.001, d = -2.762), with high and lower skilled sim racers averaging lap times of 87029.36ms and 96479.74ms respectively (Table 1).

3.2. Raw fixation metrics (Non-normalised)

Lower skilled sim racers displayed a higher number of fixations (FC) over the course of their lap compared to high skilled sim racers, although this was not significant (t(39) = -1.299, p = .101, d = -.407)(Table 1; Figure 2a). Non-normalised average fixation duration (AFD) was found not to be significantly different between higher and lower skilled sim racers (t(39) = -.696, p = .245, d = -.218)(Table 1; Figure 2b). Finally, when examining total fixation duration (TFD), lower skilled sim racers had a significantly higher total fixation duration than higher skilled sim racers (t(39) = -8.336, p <.001, d = -2.661)(Table 1; Figure 2c).

3.3. Normalised fixation metrics

When fixation count was corrected for lap time (FCn), no significant difference was found between high and low skilled sim racers (t(39) = -.236, p = .407, d = -.074)(Table 1; Figure 3a). Normalising average fixation duration (AFDn) resulted in no significant difference between higher and lower skilled sim racers (t(39) = .226, p = .411, d = .071)(Table 1, Figure 3b). Lastly, when normalising total fixation duration (TFDn), a significant difference was still present, with lower skilled sim racers fixating for a higher proportion of their lap time than higher skilled sim racers (t(39) = -2.239, p = .013, d = -.729) (Table 1; Figure 3c)

3.4. Fixation allocation ratio

When examining TRACK:HUD fixation ratios, we found low skilled sim racers had a significantly higher ratio of their fixations on track in comparison to HUD elements when compared to high skilled sim racers (t(39) = -3.093, p = .002, d = -.969)(Table 1; Figure 4).

4. Discussion

In this study, we set out to compare the visual behaviours of high and low skilled sim racers during a time trial task where participants tried to achieve their fastest lap time. By testing our first hypothesis, we found that, low skilled sim racers had a higher number of fixations and had a significantly greater total fixation duration compared to high skilled sim racers. When testing our second hypothesis, by correcting for the lap time duration of both groups, we found no significant difference in fixations per second or average fixation duration, but did still see higher total fixation durations among lower skilled compared to high skill racers. Lastly, when testing our third hypothesis, we found that low skilled sim racers had a significantly greater ratio of fixations on-track compared to HUD elements than high skilled sim racers. We discuss the implications of our findings below.

As expected, low skilled sim racers displayed greater attention allocation when lap time was not corrected for. The longer duration of their lap times, which were significantly different from the lap times of the high skilled sim racers, allowed for an increased number of fixations (Figure 2a), average fixation duration (Figure 2b), and total fixation duration (Figure 2c).

Interestingly, this pattern of behaviour was maintained even after correcting for the increased temporal opportunity low skill sim racers had to fixate due to their slower lap times. This suggests that to complete the same task, low skill racers required greater attentional resources. Much of the literature on self-paced tasks and visual behaviour has found that experts tend to have less fixations of a longer duration than non-experts (Brams et al., 2019 - systematic review; Vickers, 1992;1996). Our findings contradict this previous body of evidence by the fact that the fixation counts were no different between our skill levels. We note however that in previous studies the findings pertained to the comparison of extreme groups (i.e., beginner/ novice and expert/high skill) while the performance of our low skilled group equated more to a skilled as opposed to a novice cohort. In other words, skill level in this current study may be homogenous given that low skilled sim racers committed over 4 hours per week to race training (Table 1). Overall, our findings evidence that higher skill sim racers appear to need significantly less attention throughout their fastest lap.

Contrary to our final hypothesis we note that that low skilled sim racers had a significantly higher ratio of fixations on track compared to high skilled sim racers (Figure 4). That is not to say that either group attended more to HUD elements relative to the track. This can be seen by the fact that the average fixation ratios of both groups fall above the ratio of the AOI areas of the two elements (see dotted line in Figure 4). Instead, we note that lower skilled racers attend relatively more to track compared to HUD elements. This suggests that lower skilled sim racers may not have the attentional resources to gather and utilise other on-screen information that may be relevant to higher levels of performance. Interestingly, a lower ratio of TRACK:HUD fixations for high skilled sim racers did not hinder performance as evidenced by their faster lap times. This potentially points to the importance of HUD elements for optimal sim racing performance. For example, lap time information, which contains current lap time versus best lap time, can inform where time may be gained throughout a racers lap. This pattern of attentional allocation among our high skilled group can also be seen in research studying expertise in first person shooter video games. Furukado and Hagiwara (2023) found that high skilled Valorant players attended significantly more to the in-game mini-map (giving the player HUD information on locations of opponents) than less skilled players. Thus, selective attention allocation may be an important feature of successful performance.

Although previous research has examined driver head and eye movement in real (Land & Tatler, 2001) and virtual settings (van Leeuwen et al., 2017), the current study is the first to examine visual attention in a virtual racing setting. In doing so, we provide new insight into how sim racers of differing skill levels allocate attention, which may be used for the talent identification and training of sim racing expertise. The implementation of gaze metrics has already been applied to training in surgery (Wilson et al., 2011), and sporting domains (Vine & Wilson, 2010; Vine et al., 2011). As many highly skilled sim racers now transition into real world racing, visual attention training could aid their performance. Eye movement variables that were not assessed in this study, such as saccades and gaze distribution, should be investigated in the future as the total area of the scene being viewed may also be a fruitful avenue to explore expertise (Jeong et al. 2024).

5. Conclusions

Overall, the findings of the current study differ from existing literature on visual behaviours and expertise but provides explanations as to why sim racers of differing skill level do not show differences in their length of attention allocation. Furthermore, the importance of where attention is allocated is highlighted for successful sim racing performance. Thus, our findings contribute to a relatively new field of study, where much more research is needed to cognitively profile sim racers of differing expertise.