Cognitive Assessment and Training in Extended Reality: Multimodal Systems, Clinical Utility, and Current Challenges

1. Introduction

Extended Reality (XR), which encompasses Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), has revolutionized the landscape of cognitive assessment and training [1,2]. These immersive technologies have enabled the creation of highly interactive environments that simulate real-world tasks, offering a more ecologically valid approach to understanding cognitive processes [3]. Unlike traditional neuropsychological tests, which are often limited to static, isolated tasks in controlled settings, XR allows for the dynamic simulation of real-world complexities [4,5]. This increased ecological validity is essential for accurately measuring cognitive functions such as memory, attention, and executive function, all of which are deeply intertwined with real-world behaviors and tasks [6,7].

One of the primary advantages of XR in cognitive science is its ability to offer rich multimodal feedback [8]. By integrating advanced technologies such as eye-tracking, electroencephalography (EEG), galvanic skin response (GSR), and body tracking, XR platforms provide real-time, continuous data on both cognitive performance and underlying physiological states [9,10]. This multimodal approach delivers a more comprehensive picture of cognitive processes, allowing for a deeper understanding of how individuals interact with their environment during cognitive tasks [11]. In contrast to traditional assessments, which often capture only isolated data points, XR’s capacity for real-time feedback offers dynamic insights into the cognitive and emotional states of individuals as they engage in complex, real-world tasks [12].

The immersive nature of XR not only enhances user engagement and motivation [13]. but also addresses a common challenge in traditional cognitive tasks: the learning effect that arises when assessments are administered repeatedly. Traditional tasks, often tedious and repetitive, can lead to disengagement, particularly in older adults and individuals with cognitive impairments [14]. In contrast, XR transforms these tasks into interactive and engaging experiences, with immersion in virtual environments stimulating higher levels of user participation and attention, which is especially beneficial for extended cognitive training sessions [15]. Importantly, XR-based assessments can adapt and personalize test conditions in real-time, modifying difficulty levels based on user performance and cognitive load [16,17]. This adaptability helps mitigate the learning effect, as each session can present varied challenges, preserving the assessment’s validity over repeated administrations. This flexibility makes XR an effective tool for cognitive training and rehabilitation across both clinical and educational settings [18].

The applications of XR extend beyond cognitive training to include a wide range of clinical assessments and interventions [19]. In clinical settings, XR technologies are being used to assess cognitive impairments related to neurodegenerative diseases, brain injuries, and developmental disorders [20,21]. The ability to simulate daily activities and real-world challenges within a controlled environment allows clinicians to observe cognitive performance in a context that is more representative of everyday functioning [22]. This capability is particularly important in neurorehabilitation, where XR can be used to design interventions that target specific cognitive deficits while providing a safe and controlled environment for practice [23]. Additionally, the continuous data collection provided by XR systems enables clinicians to track progress over time and adjust interventions, accordingly, offering more personalized and responsive treatments [24].

Despite the immense potential of XR technologies, several challenges hinder their widespread adoption, one significant limitation is the underutilization of XR’s full capabilities [25]. Many existing XR applications focus predominantly on visual or auditory feedback, while neglecting the integration of other sensory modalities such as haptic feedback [26,27]. This underutilization limits the potential for fully immersive experiences and reduces the range of biometric data that can be collected [28]. Enhancing the sensory feedback in XR environments could greatly improve the user experience and provide more detailed insights into cognitive performance [18].

Another challenge is the issue of cybersickness, which remains a major barrier to the extended use of immersive XR technologies [29]. Cybersickness can manifest as nausea, dizziness, and discomfort, particularly during prolonged exposure to VR environments [30]. This issue is especially problematic in clinical and educational settings, where long sessions may be necessary for effective cognitive training or assessment [31,32]. Addressing cybersickness requires not only improvements in the design of XR systems but also a deeper understanding of how individual differences—such as age, gender, and prior experience—affect susceptibility to these symptoms [33,34,35,36]. Mitigating cybersickness will be crucial for the broader adoption of XR technologies in cognitive science [37].

Finally, accessibility and usability remain significant barriers to the full integration of XR technologies into cognitive assessment and training [38]. While the cost of XR hardware has decreased over time, high-quality systems remain prohibitively expensive for many clinical and research institutions [39]. Similarly, XR frameworks has increased the accessibility of software packages to facilitate the development of VR to non-programmer researchers; however, the successful development of reliable, complex and realistic VR environments and tasks still requires a technical knowledge and competency [40,41,42]. Furthermore, the technical complexity of these systems can pose challenges for clinicians and educators who may lack the expertise to operate them effectively [43]. To ensure that XR technologies can be widely adopted, it will be essential to develop more affordable, user-friendly systems that can be easily implemented in a variety of clinical and educational settings [44].

This review aims to explore current applications, usability, and limitations of XR technologies in cognitive assessment and training. It will examine how XR can enhance ecological validity, improve user engagement through immersive interfaces, and leverage multimodal systems for real-time data collection. Additionally, the review will identify key challenges in the field, such as the underutilization of available XR modalities, issues related to cybersickness, and barriers to widespread accessibility and usability.

2. Ecological Validity in XR-based Cognitive Assessment and Training

2.1. Definition and Importance of Ecological Validity in Cognitive Science

Ecological validity refers to the extent to which the conditions and tasks used in cognitive assessments reflect real-world scenarios [45]. In cognitive science, ecological validity is essential for ensuring that findings and measurements gathered in controlled environments can be generalized to everyday cognitive functioning [46]. Traditional cognitive assessments, such as paper-and-pencil tests or static computerized tasks, often lack this quality [47]. These assessments tend to isolate specific cognitive functions, such as memory, attention, or problem-solving, but they do so in environments that do not accurately represent the complexity and unpredictability of real-world situations [48].

The importance of ecological validity lies in the understanding that cognitive performance is context-dependent [49]. Human cognition is influenced by various environmental, social, and situational factors, and how individuals perform in a lab setting may not accurately reflect how they function in daily life [50]. This issue is especially relevant in fields such as neuropsychology and cognitive rehabilitation, where assessments guide interventions designed to improve real-world outcomes [51].

2.2. XR’s Potential to Enhance Ecological Validity

Extended Reality (XR) technologies, particularly Virtual Reality (VR) and Mixed Reality (MR), offer immense potential for enhancing the ecological validity of cognitive assessments and training [1,52]. XR environments simulate real-world conditions, allowing for the creation of immersive, dynamic environments that more accurately reflect everyday challenges [53]. This ability to closely replicate real-world tasks enables researchers and clinicians to assess cognitive abilities in a contextually relevant way, providing insights that are more transferable to daily life [54].

For example, in a VR-based cognitive assessment, a participant might be required to navigate through a virtual city, make decisions, solve problems, and interact with the environment [4,55]. This scenario would require the participant to engage multiple cognitive functions—such as attention, memory, executive function, and visuospatial reasoning—in a way that more closely mirrors the cognitive demands of real life [56,57]. XR environments allow for more holistic assessments of cognitive abilities, as they engage participants in tasks that are not only dynamic but also highly relevant to everyday life [1].

2.3. Examples of XR-based Cognitive Tasks with Real-World Relevance

Several XR-based cognitive tasks have been developed to assess real-world cognitive abilities in immersive and contextually relevant settings [6,7]. In the case of everyday memory, immersive VR cognitive tasks have been shown to evoke autobiographical memories that are closer to daily-life memories than neuropsychological or computerized tests [58]. Furthermore, assessing episodic memory in XR naturalistic environments promotes the recall of everyday actions, which benefits memory codification and retention processes often impaired in several neurodegenerative diseases [59].

Other prominent example is the use of XR environments to assess prospective memory, which involves remembering to perform a task at a specific time or in response to a specific event [60]. In traditional lab settings, prospective memory tasks might involve simple instructions, such as remembering to press a button after a set period [61]. However, in an XR environment, prospective memory tasks can be embedded within complex, realistic scenarios, such as managing a virtual household or navigating through a shopping trip [56,62]. These more elaborate settings provide a better approximation of real-world cognitive demands enabling the study of prospective memory task in specific populations [63].

Another example is the use of XR to assess executive function [37]. Executive functions—such as planning, decision-making, and problem-solving—are critical for managing complex, goal-oriented behavior in everyday life [64]. XR-based assessments can simulate situations that require the use of executive functions in a realistic manner, such as planning a route through a virtual environment or managing multiple tasks in a virtual workspace [55]. These types of tasks provide a more functional and ecological assessment of executive function than traditional tasks, offering a closer approximation to daily-life demands [50].

The ecological validity of XR is particularly relevant in assessing language within a naturalistic and interactive environment featuring virtual avatars in social interactions, enabling the simultaneous examination of eye gaze, body movements, facial expressions and natural speech [65,66]. Traditional tests examine language with repetitive, standardized and isolated tasks, missing social interactions and multimodal data [65]. For example, combining VR, EEG and eye-tracking in naturalistic environments such as supermarket and restaurant, can help predict language comprehension and identify neurophysiological markers [67].

Additionally, XR-based tasks have been applied to the assessment of attention, spatial cognition, and working memory [68,69]. For example, XR environments can simulate everyday tasks such as driving, cooking, or working in an office, all of which demand sustained attention, the ability to shift focus between tasks, and the use of spatial memory to navigate through complex environments [1,55]. By allowing cognitive abilities to be assessed in real-world-like contexts, XR technologies offer a more ecologically valid measure of cognitive performance that reflects the demands individuals face in their daily lives [4].

3. Usability, Acceptability, and User Experience in XR

3.1. Usability of XR Devices and Systems for Cognitive Assessment

Usability is a critical factor in the successful implementation of Extended Reality (XR) technologies for cognitive assessment. Usability refers to the ease with which users can interact with XR devices and systems, including the hardware (e.g., headsets, sensors) and software interfaces that deliver the immersive experience [70]. For cognitive assessments, the usability of XR systems is especially important because the tasks typically require active participation and sustained engagement [38,71]. If the XR interface is not intuitive or if the hardware is uncomfortable or difficult to use, it can negatively impact on the participant's cognitive performance, leading to inaccurate assessments [13].

In recent years, significant advancements have been made in improving the usability of XR systems. Modern headsets have become lighter, more ergonomic, and easier to wear for extended periods, while user interfaces have become more intuitive and responsive [25]. For example, the integration of hand tracking, eye tracking, and voice commands has allowed for more natural interactions within virtual environments, reducing the cognitive load associated with learning and using new interfaces [72,73,74]. These improvements are essential in ensuring that users can focus on the cognitive tasks at hand rather than on managing the complexities of the XR system itself [75].

However, despite these improvements, some usability challenges remain. For instance, older adults and individuals with limited experience in using advanced technologies may find XR systems intimidating or challenging to use, which can reduce their willingness to participate in XR-based assessments or training [76,77]. Additionally, for individuals with cognitive impairments, the complexity of XR interfaces may present significant barriers to participation [78]. Addressing these usability concerns is crucial to ensuring that XR systems are accessible to a broad range of populations [79].

3.2. User Acceptability and Experience Across Various Populations

User acceptability refers to how well different populations accept and engage with XR technologies [80]. A key factor influencing acceptability is the user experience (UX)—the overall experience and satisfaction users derive from interacting with XR devices and applications [81]. The success of XR in cognitive assessment and training depends largely on how comfortable and engaged users feel while using the technology [18]. If users find the XR experience disorienting, uncomfortable, or cognitively taxing, their acceptability of the technology will decrease [79].

Different populations—children, middle-aged adults, elderly individuals, and those with cognitive impairments—have unique needs and preferences when interacting with XR systems [71]. For children, XR can provide an engaging and interactive learning environment, but there may be concerns about safety, age-appropriate content, and the impact of extended exposure to immersive technologies [82]. Middle-aged adults may find XR beneficial for cognitive training or rehabilitation; however, balancing XR-based tasks with daily responsibilities, such as work and family commitments, could impact their ability or willingness to engage consistently. High time demands or rigid schedules in XR programs may reduce user acceptability if the training conflicts with other responsibilities [83]. Flexible, time-efficient XR interventions can address this challenge by allowing users to integrate cognitive training more seamlessly into their routines, enhancing overall acceptability and adherence.

The elderly population presents another set of considerations. While XR has demonstrated potential for enhancing cognitive functions among older adults, factors such as limited familiarity with technology, cognitive overload, and physical limitations may affect their ability to engage fully with the system. Insufficient experience with digital interfaces can also contribute to feelings of unease or hesitation, potentially hindering effective use and reducing overall engagement [38]. For instance, the weight and fit of headsets can be a concern for elderly users, and complex interfaces may require additional training or support to ensure that they are comfortable using the technology [77,84]. Furthermore, those with cognitive impairments, such as individuals with dementia or acquired brain injuries, may benefit from XR-based cognitive training, but they may also be more susceptible to disorientation and face physical and cognitive barriers when using immersive technologies [85,86]. Ensuring that XR systems are designed with the specific needs of these populations in mind is critical for promoting user acceptability [80].

3.3. Barriers to XR Adoption: Hardware, Software, and Accessibility Challenges

Despite the growing interest in XR for cognitive assessment and training, several barriers hinder its widespread adoption [23]. One of the most significant barriers is the cost of XR hardware [25]. Although prices have decreased over the years, high-quality XR systems remain expensive, making them inaccessible to many clinical, educational, and research institutions [39]. The cost of the headsets, sensors, and computers capable of running immersive XR environments can be prohibitively high, particularly in low-resource settings [1].

In addition to hardware costs, software development for XR applications can be resource-intensive [44]. Designing, developing, and maintaining high-quality XR applications that are engaging, user-friendly, and compatible with the latest hardware requires specialized expertise, significant investment, and continuous updates as technology evolves [43]. Institutions that do not have access to such resources may struggle to integrate XR into their cognitive assessment and training programs [46].

Another major barrier to adoption is accessibility. XR systems require a certain level of technological literacy to operate, which may not be present in all populations, especially among older adults or individuals with limited experience in using digital technologies [87]. Accessibility also extends to the physical requirements of using XR systems [88]. For example, some users may find it difficult to wear VR headsets for extended periods due to discomfort, while others may have physical limitations that prevent them from fully interacting with XR environments [89]. Additionally, individuals with visual or hearing impairments may face significant challenges in using XR systems if the applications are not designed with their needs in mind [90].

Overcoming these hardware, software, and accessibility challenges is essential for XR to become a widely accepted tool in cognitive science [91]. Efforts to make XR systems more affordable, intuitive, and inclusive will be key to promoting their adoption across a wider range of populations and settings [87,88,90].

3.4. Case Studies on User Experience in Cognitive Training

Numerous case studies have highlighted the impact of XR technologies on user experience in cognitive training [85]. For example, studies involving older adults undergoing cognitive training in VR environments have shown that immersive tasks can improve memory, attention, and executive functions [92,93]. These case studies often report high levels of user engagement, with participants finding the immersive nature of XR to be more motivating and enjoyable than traditional cognitive training methods [84]. However, some studies have also reported challenges related to usability and physical comfort, particularly in populations that are less familiar with digital technologies [81].

In another case study involving children with Autism Spectrum Disorder (ASD), XR was used to simulate real-world environments in which the participants could practice social and cognitive skills [94]. Comparably, using scenario simulating daily life tasks (e.g., going to the cinema, interview, and shopping), there is an effective training of social skills for adults with ASD [95]. Other study focused on social cognition and neurodevelopment disorders found that the children with ADHD were highly engaged with the XR tasks and demonstrated improvements in both cognitive and social functioning [82]. The immersive nature of XR allowed for repeated practice in a safe and controlled environment, offering a unique advantage over traditional training methods [96].

In clinical settings, XR has been used to develop personalized cognitive training programs for individuals with traumatic brain injuries or neurodegenerative conditions [39,97]. These programs have been successful in improving cognitive function, with participants reporting that the immersive, interactive environments helped them stay focused and motivated throughout the training sessions [98,99]. However, these case studies also highlight the importance of designing XR systems that are accessible and easy to use, as the participants often required additional support to navigate the XR environment effectively [39].

4. Multimodal Systems in XR Cognitive Applications

4.1. Overview of Multimodalities: GSR, EEG, Eye Tracking, Hand Tracking, Body Tracking

The integration of multimodal systems in Extended Reality (XR) cognitive applications represents a significant advancement in both cognitive assessment and training [100]. Multimodal systems refer to the use of various sensory and physiological tracking technologies that capture data simultaneously from different dimensions of human experience [101]. This comprehensive integration provides detailed, real-time analysis of cognitive and emotional processes within immersive XR environments [102]. The key modalities commonly employed in XR applications include Galvanic Skin Response (GSR), Electroencephalography (EEG), eye tracking, hand tracking, and body tracking [72,103], see Table 1.

GSR measures changes in skin conductivity, which correlate with physiological arousal, making it a valuable tool for tracking emotional responses such as stress or excitement during virtual tasks [104,105]. EEG is a non-invasive technique that captures electrical activity in the brain, providing insights into neural processes related to attention, cognitive load, and emotional regulation [106]. Eye tracking records eye movements and focus, enabling researchers to examine where users direct their attention and how they process visual information in a dynamic virtual environment [74,75]. Hand tracking captures hand movements and gestures, allowing for natural interactions with virtual objects, while body tracking monitors full-body movements and postures, facilitating the study of motor coordination and spatial navigation [103].

Together, these modalities form a powerful toolkit for understanding the interplay between cognitive, emotional, and motor functions, providing deeper insights into human behavior in real-world-like XR scenarios [107].

4.2. Integration of These Modalities in XR Environments

The integration of these multimodal systems into XR environments significantly enhances the quality and depth of data collected during cognitive assessments and training [100]. Unlike traditional methods, which collect isolated data points at specific intervals, XR environments equipped with multimodal systems offer continuous, real-time data collection across multiple dimensions of human behavior [19]. This synchronized data collection allows for a more comprehensive and dynamic understanding of how cognitive and emotional processes unfold during task performance [108].

For instance, in a cognitive task within a virtual environment, EEG can track cognitive load, GSR can measure emotional arousal, and eye tracking can monitor attentional shifts—all while hand and body tracking provide insights into motor performance [101,103]. The combination of these data streams enables researchers to capture a holistic view of a participant’s cognitive state in real time [109]. This is particularly valuable for understanding complex tasks that require simultaneous engagement of multiple cognitive functions, such as problem-solving, attention management, and decision-making [110].

The integration of these modalities also allows for adaptive feedback mechanisms in XR environments [12]. For example, if EEG data indicates cognitive overload, or GSR shows heightened stress levels, the system can adjust the difficulty of tasks to optimize performance and user experience [16,111]. This adaptive capability makes XR environments more responsive and personalized, contributing to their effectiveness in both research and clinical settings [12].

4.3. Applications of Multimodal Systems in Cognitive Assessment

Multimodal systems in XR environments are applied to a variety of cognitive assessments, offering new possibilities for evaluating cognitive functions [100]. One key application is emotion recognition, where GSR, EEG, and facial expression analysis work together to detect emotional states such as stress, excitement, or frustration [102]. This data can be particularly valuable for mental health assessments, where understanding emotional responses to specific stimuli can inform personalized therapeutic interventions [25].

Another important application is attention monitoring [12]. By combining eye tracking and EEG data, researchers can assess how well participants maintain attention during tasks, as well as identify moments of cognitive overload or attentional lapses [101]. For instance, in a virtual driving simulation, eye tracking can monitor the participant’s gaze to ensure they are focusing on relevant elements of the environment, while EEG can provide information on the mental effort required to sustain attention [109,112]. This dual approach offers more accurate assessments of attentional control compared to traditional methods [12].

Cognitive load assessment is also significantly enhanced by multimodal systems. EEG data, in combination with behavioral metrics from hand and body tracking, can reveal how participants manage cognitive load during complex tasks [107]. This is particularly useful in educational or training contexts, where understanding cognitive load can help optimize task difficulty and improve learning outcomes [113]. Real-time monitoring of cognitive load allows XR systems to adjust the complexity of tasks dynamically, ensuring that participants remain engaged without becoming overwhelmed [114].

Additionally, visuomotor coordination and spatial cognition are areas where multimodal systems offer unique advantages [115]. Hand and body tracking provide detailed data on how participants interact with virtual objects and navigate through virtual spaces, making it possible to assess motor-cognitive integration in ways that traditional testing cannot [72]. This is especially relevant in rehabilitation settings, where improving motor function alongside cognitive abilities is a key goal [116].

4.4. Advantages and Limitations of Multimodal Systems

The primary advantage of multimodal systems in XR environments is the richness and comprehensiveness of the data they provide [100]. By collecting real-time data from multiple modalities, these systems offer a multidimensional view of cognitive and emotional processes, allowing for more accurate assessments and personalized interventions [117] . This holistic approach is particularly valuable in cognitive training and rehabilitation, where understanding the complex interactions between cognitive, emotional, and motor functions is critical for designing effective interventions [118].

Another key advantage is the ability of multimodal systems to provide real-time feedback, which can be used to adjust the difficulty of tasks and optimize user experience [101]. This adaptive feedback is especially useful in educational and training applications, where maintaining an optimal level of challenge is crucial for maximizing learning outcomes [17].

However, there are limitations and challenges associated with the use of multimodal systems in XR environments [119]. One major challenge is the technical complexity involved in integrating and synchronizing data from multiple sensors [120]. This requires sophisticated hardware and software systems capable of processing large amounts of data in real time, which can be a barrier to adoption, particularly for institutions with limited resources or technical expertise [25,121].

Another limitation is the potential for data overload. While multimodal systems provide valuable insights, the sheer volume of data can be difficult to manage and analyze [122]. Advanced data processing techniques and expertise are required to interpret the complex data sets generated by these systems, which may not be readily available in all research or clinical settings [44]. Additionally, the use of multiple sensors and tracking devices can cause discomfort for participants, potentially affecting their performance and the overall user experience [123].

Despite these challenges, the integration of multimodal systems into XR environments represents a significant advancement in cognitive science [119]. By providing a comprehensive, real-time view of cognitive, emotional, and motor processes, multimodal systems have the potential to transform cognitive assessment and training, offering more personalized and effective solutions across various populations and contexts [101,124].

5. XR Applications in Cognitive Assessment

5.1. Review of Current XR-based Cognitive Assessment Tools

XR technologies have revolutionized the approach to cognitive assessment [1,125]. Traditional cognitive assessment methods, such as paper-and-pencil tests or static computerized tasks, provide valuable insights into isolated cognitive functions but often fail to capture the complexity of real-world cognitive demands [47]. XR tools, in contrast, immerse participants in highly interactive, dynamic virtual environments that simulate everyday situations, allowing for more ecologically valid assessments [4], see Table 2.

A range of XR-based cognitive assessment tools have been developed to evaluate various cognitive domains, including memory, attention, executive functions, spatial abilities, and problem-solving [1]. One of the most prominent areas of application is memory assessment, where XR platforms create immersive environments that require participants to remember objects, locations, or sequences within a virtual world [6]. For example, in an XR-based memory task, participants might navigate through a virtual house, remembering where items are located and retrieving them after a delay [4,59]. Such tasks mimic real-world scenarios where memory is employed in navigating spaces and make several actions, offering a more accurate reflection of real-life memory usage compared to traditional tests that may lack contextual richness [5,126].

Similarly, XR-based tools have been developed for assessing executive functions, which include cognitive processes such as planning, decision-making, task-switching, and problem-solving, in virtual environments with complex multi-sept tasks [7]. In one example, participants may be tasked with managing a virtual office, where they must prioritize tasks, respond to unexpected challenges, and make strategic decisions [55]. Other VR task focus on assessing the inhibitory control in a virtual classroom in adults with ASD, demonstrating sensibility to everyday difficulties [127]. This type of tasks mirrors the complexity of executive functions in real-world settings, where individuals are constantly required to plan, adapt, and make decisions in the face of changing conditions [5,37].

Attention assessment is another area where XR tools have been effectively deployed [1,5]. Traditional attention tasks might involve responding to stimuli on a static screen, but XR environments can create more immersive and engaging tasks that better reflect real-world demands [128]. For instance, participants could be placed in a virtual classroom where they must pay attention to a teacher’s instructions while filtering out distractions from other students [129]. This more naturalistic task setup allows researchers to study how attention is maintained in realistic, high-stimulation environments, providing more meaningful insights into attentional control and focus [130].

XR technologies are also used to assess spatial cognition, particularly through tasks that require participants to navigate through complex virtual spaces or manipulate objects in a three-dimensional environment [4,115]. In spatial memory tasks, participants might need to recall the location of specific landmarks in a virtual city or plan a route to a destination, requiring them to use spatial reasoning and memory in ways that closely mirror real-world navigation tasks [5,131,132]. These assessments provide detailed data on how individuals perceive, remember, and interact with spatial environments, which is particularly valuable for assessing populations with spatial deficits, such as individuals with neurodegenerative diseases, brain injuries, or developmental disorders [133].

Another notable example is the Virtual Reality Everyday Assessment Lab (VR-EAL). The VR-EAL is an immersive e neuropsychological battery that measures critical everyday cognitive functions, including executive functions, prospective memory, attention, and episodic memory, within immersive and realistic virtual environments [4,5]. It assesses executive functions by requiring participants to strategically plan their itinerary through a virtual city to efficiently complete a series of tasks, such as shopping, preparing breakfast, and remembering to take medication [4,5]. This simulates real-world demands on planning, decision-making, and problem-solving. Prospective memory is evaluated through time-based and event-based tasks that mimic everyday responsibilities, like taking medication at a specific time or responding to prompts based on environmental cues [56,57]. Episodic memory is measured by recalling details from earlier in the experience, such as remembering items from a shopping list or specific instructions [56,57]. Selective visuospatial and auditory attention tasks involve detecting visual or auditory cues amid distractions, providing a realistic and comprehensive assessment. By embedding these challenges in a dynamic and engaging virtual environment, VR-EAL ensures an ecologically valid evaluation of cognitive functions as they are used in everyday life, capturing both performance and the participant's sense of presence and engagement [4,5].

Overall, the diversity of XR-based cognitive assessment tools highlights their ability to offer more engaging, ecologically valid, and comprehensive assessments across various cognitive domains [18], see Table 2. By replicating real-world tasks in virtual environments, these tools provide deeper insights into cognitive performance and offer new opportunities for personalized cognitive assessments tailored to individual needs [4].

5.2. XR Assessments Compared to Traditional Methods

The key distinction between XR-based cognitive assessments and traditional methods lies in their ability to simulate real-world complexity and provide a more engaging and immersive testing environment [136]. Traditional cognitive assessments, such as computerized tests or structured interviews, often isolate specific cognitive functions—such as memory, attention, or problem-solving—in controlled, low-stakes environments [19]. While these assessments are valuable for standardizing cognitive testing, they often fail to capture the richness of cognitive functioning in everyday life, where cognitive tasks are rarely performed in isolation or under ideal conditions [137].

XR-based assessments offer several distinct advantages over traditional methods, starting with their enhanced ecological validity [50]. XR environments allow for the creation of highly realistic scenarios that more closely mimic the demands of everyday cognitive functioning [136]. For instance, while traditional memory assessments might ask participants to recall a list of words or numbers, XR assessments place participants in virtual environments where they must remember where objects are located or follow complex instructions in real time [6,136]. This more naturalistic approach provides a better understanding of how cognitive processes operate in real-world contexts, making the findings more applicable to everyday situations [53].

Another important advantage of XR assessments is their ability to assess visuomotor coordination and spatial cognition in a way that traditional methods cannot [115]. For example, a traditional test might ask participants to draw a figure or navigate a simple two-dimensional maze [54,138]. In contrast, an XR task could require participants to interact with three-dimensional objects, solve puzzles involving spatial reasoning, or navigate complex environments in a fully immersive virtual world [24]. This immersive nature allows researchers to study how cognitive functions, such as spatial memory or visuomotor coordination, operate under more natural and dynamic conditions [3]. Moreover, XR tasks can incorporate real-time feedback, further enhancing the richness of the assessment by providing insights into how individuals adjust their strategies in response to changing task demands [25].

XR assessments also offer the potential for continuous monitoring of cognitive and motor performance [72]. Traditional assessments typically record performance at specific points in time, such as response times or scores on a standardized test [53]. In contrast, XR systems can collect continuous data throughout the entire task, providing insights into how cognitive processes fluctuate over time [12]. For example, an XR-based attention task could track not only whether participants successfully complete the task but also how their attention shifts over the course of the task, where they direct their gaze, and how their cognitive load changes as the task becomes more complex [16]. This continuous data collection allows for a more detailed and nuanced analysis of cognitive performance than is possible with traditional methods [12].

While XR-based assessments offer numerous advantages, it is important to note that they complement rather than replace traditional methods [1]. Each approach has its own strengths, and the combination of XR assessments with traditional tests can provide a more comprehensive evaluation of cognitive abilities, offering both standardized data and ecologically valid insights into real-world functioning [139].

5.3. XR’s Potential for Real-time Data Collection and Analysis

One of the most significant advantages of XR-based cognitive assessments is their capacity for real-time data collection and analysis [140]. Unlike traditional assessments, which typically focus on isolated performance metrics such as test scores or response times, XR systems can continuously collect a wide range of data streams simultaneously [141]. These data streams can include behavioral data, such as eye movements and body posture, physiological data, such as heart rate and galvanic skin response (GSR), and neural data, such as electroencephalography (EEG) signals [124]. This multimodal data collection provides a much richer and more comprehensive understanding of cognitive and emotional processes, allowing researchers to capture how these processes unfold dynamically during task performance [24,102].

For example, in an XR-based task designed to assess decision-making, the system might track how long it takes participants to make choices, where they direct their attention (using eye tracking), their emotional responses to the task (using GSR), and their cognitive load (using EEG) [101,125]. By combining these different data streams, researchers can gain a holistic understanding of how cognitive, emotional, and motor processes interact during complex tasks [140]. This multimodal approach provides a deeper understanding of the factors that contribute to task performance, as well as the underlying processes that drive decision-making, attention, and memory in real time [142] .

In addition to providing real-time insights into cognitive performance, XR systems offer the potential for adaptive cognitive assessments [12]. In traditional cognitive tests, the level of difficulty is typically predetermined and remains constant throughout the assessment [137]. However, XR systems can dynamically adjust the difficulty of tasks based on real-time feedback from the participant [143]. For instance, if an XR system detects that a participant is experiencing cognitive overload—through EEG signals indicating high levels of cognitive strain or GSR data showing increased stress—the system can automatically reduce the difficulty of the task to prevent frustration and maintain engagement [111,144]. Similarly, if the system detects that a task is too easy, it can increase the complexity of the task to ensure that the participant remains cognitively challenged [119]. This ability to adapt to the participant’s cognitive state in real time makes XR assessments more responsive and personalized, enhancing their effectiveness for a wide range of users [122].

5.4. Challenges in Implementing XR in Large-scale Cognitive Assessments

Despite the numerous benefits of XR-based cognitive assessments, there are several challenges to implementing these tools on a large scale [44]. One of the most significant barriers is the cost of XR hardware and software [1]. High-quality VR and AR systems, which are necessary for creating fully immersive environments, remain relatively expensive compared to traditional assessment tools [25]. These costs can include not only the headsets and sensors but also the powerful computers required to run immersive XR environments at high frame rates [119]. For many schools, clinics, and research institutions with limited budgets, the upfront investment in XR technology may be prohibitive [39].

In addition to the hardware costs, there are also technical requirements for developing and maintaining XR-based cognitive assessments [43]. Creating engaging, scientifically valid XR environments requires expertise in both cognitive science and software development, as well as ongoing maintenance to ensure that the system remains functional and up-to-date with the latest technological advances [44]. The specialized knowledge required to design and implement XR tasks may be a barrier for institutions that do not have access to skilled developers or the financial resources to outsource these services [43].

Accessibility is another challenge in implementing XR technologies at scale [88,90]. While XR systems offer unparalleled immersion, they can also pose accessibility challenges for certain populations, particularly those with physical or cognitive impairments [79,145]. For example, some users may experience discomfort when using VR headsets for extended periods, leading to symptoms such as motion sickness, headaches, or eye strain [31]. These side effects, often referred to as cybersickness, can limit the duration of XR assessments and affect the accuracy of the data collected [146]. Ensuring that XR assessments are designed to minimize discomfort and are accessible to a wide range of users is crucial for their broader adoption [1].

Scalability is another key issue. While XR-based assessments are highly effective in controlled, small-scale research settings, scaling them up for widespread use presents logistical challenges [1]. For example, administering XR assessments to large groups of participants requires a sufficient number of XR systems, as well as the technical support to manage and troubleshoot these devices during testing [44]. Furthermore, the data generated by XR assessments is often more complex and voluminous than that of traditional tests, requiring advanced data processing tools and expertise to analyze effectively [119]. These scalability challenges may limit the widespread adoption of XR technologies for cognitive assessments in resource-constrained settings [43].

Despite these challenges, XR-based cognitive assessments hold significant promise for the future of cognitive science [46]. As the cost of XR hardware continues to decrease and the technology becomes more widely accessible, it is likely that these tools will play an increasingly important role in large-scale cognitive assessments, offering more immersive, adaptive, and ecologically valid methods for evaluating cognitive performance [1].

6. XR Applications in Cognitive Training

6.1. Review of Current XR-Based Cognitive Training Interventions

Extended Reality (XR) technologies have shown immense potential in the field of cognitive training, offering immersive, interactive environments that can be tailored to target specific cognitive functions such as memory, attention, and executive function [25]. These technologies enable the design of dynamic, real-world scenarios that challenge cognitive abilities in ways that traditional training methods often cannot [97], see Table 3. By providing participants with controlled, repeatable tasks in immersive environments, XR-based cognitive training interventions allow for a high degree of engagement and personalization, which has been shown to enhance the effectiveness of training programs [85].

A significant focus of XR-based cognitive training is on memory enhancement [136]. In traditional cognitive training, memory tasks often involve route learning or simple recall exercises, which, while effective to some extent, may not fully engage users or reflect real-world memory challenges [147]. XR environments, on the other hand, immerse participants in virtual scenarios that more closely simulate real-life memory tasks [148]. For instance, memory training in a virtual environment might involve remembering the location of objects in a virtual home or recalling sequences of actions required to complete a task, such as preparing a meal in a virtual kitchen [73]. These scenarios provide a more ecologically valid approach to memory training, as they closely mimic the types of tasks individuals face in their daily lives [116]. Studies have shown that immersive, context-rich tasks in XR environments can lead to greater improvements in memory performance compared to traditional training methods, particularly in populations with memory deficits, such as older adults or individuals with mild cognitive impairment [149,150].

Attention training is another area where XR-based interventions have made significant strides [151]. XR environments can be designed to require sustained attention across complex and dynamic tasks [152]. For example, participants may be asked to focus on specific details within a bustling virtual marketplace or respond to multiple sources of stimuli while filtering out distractions [151]. The immersive nature of these environments forces participants to engage attentional resources in a way that more closely mirrors real-world situations [98]. In traditional attention training, tasks are often limited to simple reaction-time exercises or selective attention tests, lacking transferring results to everyday life [153]. However, XR enables a more nuanced and comprehensive training experience, with real-time feedback and adaptive difficulty levels that can be tailored to the user’s progress [152,154].

XR has also been employed in executive function training, particularly for improving skills such as problem-solving, planning, and task-switching [155]. Executive function deficits are common in individuals with neurological conditions such as traumatic brain injury (TBI) or stroke, as well as in aging populations [156,157]. In XR-based training, participants might engage in tasks that require them to plan and execute multi-step processes, such as navigating a virtual city or managing a series of tasks in a simulated workplace [158]. These tasks engage high-level cognitive functions by requiring participants to adapt to changing scenarios, prioritize tasks, and solve problems in real time [159]. The immersive nature of XR allows for a more interactive and engaging training experience, which can lead to more significant cognitive improvements than traditional methods that often isolate these cognitive processes in static, decontextualized tasks [23].

6.2. Population-Specific XR Training Programs

One of the key strengths of XR-based cognitive training is its adaptability to different populations [116]. Training programs can be specifically tailored to address the unique cognitive needs of various groups, including children, aging adults, and individuals with brain injuries or neurodevelopmental disorders [159]. This adaptability enhances the effectiveness of XR interventions, as it allows training to be personalized based on the cognitive challenges faced by each population [159].

For children, XR-based cognitive training programs can be designed to improve attention, working memory, and problem-solving skills in an engaging and playful manner [135,161]. By using game-like scenarios and interactive tasks, XR environments provide a highly motivating training experience for young users, which can help sustain attention and encourage regular participation [163]. Cognitive training for children might involve tasks such as navigating through a virtual maze, solving puzzles in a fantasy world, or completing missions that require strategic thinking and memory recall [161]. These tasks not only improve cognitive functions but also provide opportunities for children to develop skills like patience, persistence, and adaptability, which are critical for academic success [162,164].

In contrast, aging adults face different cognitive challenges, particularly related to memory decline and decreased executive function [156,165]. XR-based training for older adults often focuses on memory recall, spatial navigation, and everyday problem-solving [166,167]. For example, training programs may simulate real-world tasks such as remembering to the shopping list at the grocery store, making recipes, or navigating through a metro station [168]. These scenarios are designed to be relevant to the daily lives of older adults, making the training both practical and meaningful [166]. Research has shown that XR-based cognitive training can help older adults improve not only memory and attention but also confidence in their cognitive abilities, which can enhance their overall quality of life [169,170]. Furthermore, XR environments can be adjusted to accommodate physical limitations, ensuring that older adults can enhance their physical functioning and participate comfortably and safely [171].

For individuals with brain injuries or neurodevelopmental disorders, XR-based cognitive training can be particularly beneficial in supporting recovery and improving cognitive function [116]. In the case of traumatic brain injury (TBI), for instance, XR environments provide a safe and controlled space where patients can practice cognitive skills such as decision-making, task-switching, and spatial memory [172]. Virtual environments can simulate complex, real-world scenarios—such as driving or returning to work—that challenge cognitive abilities in ways that traditional therapy cannot [160]. Moreover, the flexibility of XR allows for gradual increases in task difficulty, which can be critical for patients recovering from neurological trauma [173]. Similarly, individuals with neurodevelopmental disorders, such as autism spectrum disorder (ASD), can benefit from XR-based training that focuses on social cognition, executive function, and adaptive skills [18,94,95].

For example, XR environments have been employed effectively to assist children with ASD in developing essential social and cognitive skills [95]. In a case study, XR was used to replicate real-world environments, providing a safe and engaging platform for children to practice tasks like making eye contact, understanding social cues, and engaging in conversations [94]. These virtual scenarios are not only interactive but can be tailored to each child’s needs, allowing for personalized and repeated practice in a way that traditional training methods struggle to achieve [46].

Extending this approach to adults with ASD, another study demonstrated the efficacy of XR-based training in enhancing social skills by simulating daily life situations, such as attending a job interview, going shopping or to the gym, or visiting the cinema [95]. The immersive nature of XR helps participants feel more comfortable and prepared for these activities in real life, offering an environment where they can make mistakes and learn from them without fear of judgment or real-world consequences [46]. These immersive simulations have been found to foster improvements in both social interactions and adaptive behaviors [95].

Furthermore, XR has shown promise for children with Attention Deficit Hyperactivity Disorder (ADHD). Wong et al. (2023) [82] found that XR tasks designed to train social cognition and executive function were highly engaging for children with ADHD. The virtual environments kept the participants focused and motivated, leading to observable improvements in attention, task management, and social functioning [82]. The structured yet adaptable nature of XR allows for the creation of scenarios that can challenge and gradually enhance the cognitive abilities of children with ADHD, providing a highly effective, user-centered approach to therapy [82].

Overall, these studies underscore the potential of XR technologies to transform cognitive and social skills training for individuals with neurodevelopmental disorders [95]. By creating engaging, realistic, and adaptive learning environments, XR enables both children and adults to practice and refine skills in ways that are highly transferable to real-world situations [46]. The flexibility and immersive quality of XR not only improves engagement but also offers opportunities for more personalized and effective interventions tailored to the unique needs of each individual [152,154]. By providing immersive, controlled environments, XR training allows individuals to practice these skills in a way that feels real but without the unpredictability and stress of real-world situations [116].

6.3. Long-term Effects and Retention of Cognitive Skills in XR Training

The long-term effects and retention of cognitive improvements following XR-based cognitive training are critical areas of ongoing research [174]. One of the key questions is whether the cognitive skills developed in virtual environments transfer to real-world tasks, and whether these skills are retained over time without continued training [116]. Initial studies suggest that XR-based cognitive training can lead to durable improvements in cognitive function, particularly in areas such as attention, memory, and executive function [23].

One factor contributing to the long-term retention of cognitive skills is the ecological validity of XR environments [166]. Because XR-based tasks closely resemble real-world challenges, the skills developed in these environments are more likely to transfer to everyday life [166]. For example, memory training in an XR environment that simulates a real-world task, such as remembering a shopping list while navigating a virtual store, is more likely to improve real-world memory performance than training tasks that are abstract or disconnected from daily experiences [96].

Another factor influencing retention is the level of engagement that XR environments provide [170]. Research has shown that the more engaging and immersive a training task is, the more likely participants are to stay motivated and complete the training program [23]. This engagement also appears to enhance long-term emotional arousal, as participants are more likely to maintain their cognitive performance and successfully implement cognitive strategies [175]. Additionally, the ability of XR systems to provide real-time feedback and adapt to the user’s progress ensures that participants are continuously challenged, which promotes deeper learning and better retention of cognitive skills [174].

However, some challenges remain in ensuring the long-term retention of skills gained through XR training [174]. For instance, while XR environments are highly immersive, they are still distinct from real-world settings in certain ways, and it is possible that some participants may have difficulty transferring their skills to non-virtual environments [166]. Moreover, the long-term impact of XR training on brain plasticity and cognitive reserve—the brain’s ability to adapt to cognitive challenges over time—remains an area of active research [116]. Continued studies are needed to determine the optimal frequency and duration of XR-based training to ensure long-lasting cognitive benefits [174].

6.4. Future Directions in XR-based Cognitive Training

Looking forward, the field of XR-based cognitive training is poised for significant growth as new technologies and research findings continue to emerge [24]. One of the most promising future directions is the integration of multimodal systems—such as combining XR with neuroimaging techniques like EEG and fMRI—to provide real-time feedback on brain activity during VR task [176]. This would allow for even more personalized and adaptive training programs, where tasks can be adjusted in real time based on the user’s cognitive load, attention, or emotional state [122].

Another important future direction is the development of socially interactive XR environments, where users can engage with virtual avatars while performing tasks or even training in adaptive challenge environment [18,177]. These social interactions would provide an additional layer of cognitive challenge, particularly in areas such as executive function, problem-solving, and emotional regulation [178]. For example, users could collaborate on solving complex tasks or engage in role-playing scenarios that require social cognition, empathy, and perspective-taking [179].

Artificial Intelligence (AI) will also play a crucial role in the future of XR-based cognitive training [122]. AI-driven algorithms can analyze user performance data and adapt the difficulty of tasks automatically, ensuring that participants are always working at an optimal level of challenge [122]. AI could also be used to develop more sophisticated, naturalistic virtual environments that respond dynamically to the user’s actions, providing a more seamless and realistic training experience [176].

Finally, as the cost of XR hardware continues to decrease and the technology becomes more accessible, there is potential for scaling XR-based cognitive training to reach larger populations [25]. This could include the development of affordable XR headsets and applications that can be used in schools, clinics, and homes around the world, democratizing access to high-quality cognitive training [86]. In clinical settings, XR-based training could become a standard part of rehabilitation programs for individuals recovering from brain injuries, strokes, or other neurological conditions [149,173].

7. Clinical Utility of XR in Cognitive Assessment and Training

7.1. Benefits of XR-Based Cognitive Tools in Clinical Settings

Extended Reality (XR) technologies, including virtual, augmented, and mixed reality, offer unique advantages in clinical settings, especially for cognitive assessment and training [25]. XR-based tools provide an immersive, interactive, and flexible platform for assessing and improving cognitive functions in a wide range of populations [97,136]. One of the primary benefits of XR in clinical contexts is its ability to simulate real-world scenarios that allow clinicians to observe cognitive functioning in environments that closely mirror everyday life [74]. This ecological validity is particularly important in clinical assessments, as it helps bridge the gap between laboratory-based testing and real-world cognitive performance [1].

In clinical settings, XR-based tools have proven beneficial for assessing and training cognitive functions such as memory, attention, executive function, and spatial reasoning [7,148]. For example, in neuropsychological testing, XR allows clinicians to place patients in controlled yet realistic environments where they must perform tasks such as navigating through a virtual space or completing memory-based tasks [71,134]. These tasks not only assess cognitive abilities but also provide clinicians with valuable insights into how patients interact with complex environments, including how they manage distractions, solve problems, and make decisions [180]. For patients with cognitive impairments, such as those with dementia or traumatic brain injury (TBI), XR-based assessments provide a more comprehensive view of cognitive functioning compared to traditional methods, which often focus on isolated cognitive domains [71,181].

Another significant benefit of XR-based cognitive tools is their potential for customization and adaptation. XR systems can be tailored to the individual needs of patients, allowing clinicians to adjust the difficulty level of tasks in real time or to focus on specific cognitive functions that require attention [12]. This flexibility is particularly valuable in rehabilitation settings, where patients may have different levels of cognitive impairment or varying needs for support [175]. Moreover, the interactive nature of XR environments increases patient engagement, which can lead to improved outcomes in cognitive training programs. This is especially important in populations such as children or older adults, where maintaining motivation and adherence to training programs can be challenging [79,163].

7.2. Comparative Analysis of Traditional vs. XR-Based Cognitive Assessments and Training

When comparing traditional cognitive assessments and training methods to XR-based approaches, several key differences emerge [1,116]. Traditional cognitive assessments, such as paper-and-pencil tests or computer-based tasks, typically isolate specific cognitive functions (e.g., memory, attention, or problem-solving) and are conducted in static, controlled environments [49]. While these methods provide valuable standardized data, they often lack ecological validity—the extent to which the testing environment reflects real-world conditions [137]. In contrast, XR-based assessments allow participants to engage in dynamic, immersive environments that more closely resemble real-life situations [136]. This enables a more comprehensive evaluation of cognitive abilities, including how multiple cognitive processes interact in complex, real-world tasks [1].

For example, a traditional memory test might involve recalling a list of words, while an XR-based memory assessment could place participants in a virtual grocery store where they must remember and retrieve specific items from a shopping list [182]. The latter approach not only tests memory but also integrates attention, spatial navigation, and executive functions, offering a more complete picture of cognitive performance [4]. Additionally, XR systems can collect multimodal data, such as eye movements, hand tracking, and physiological responses, which provide deeper insights into cognitive and emotional states during task performance [124]. This real-time data collection and analysis is often missing from traditional assessments [137].

In terms of cognitive training, traditional methods usually rely on repetitive exercises designed to strengthen specific cognitive functions [147]. While effective, these methods can become monotonous, leading to decreased motivation and engagement over time [173]. XR-based cognitive training offers a more engaging alternative by immersing participants in interactive, game-like environments that keep them motivated to complete the training [159]. For instance, XR platforms can simulate scenarios such as driving, workplace tasks, or social interactions, allowing participants to practice cognitive skills in meaningful contexts [94,160]. This not only improves engagement but also enhances skill transfer to real-world situations [166].

7.3. Case Examples of Clinical Applications

Several clinical applications of XR-based cognitive assessment and training tools have already demonstrated their effectiveness in real-world settings [1,172]. One example is the use of XR in neuropsychological testing for patients with mild cognitive impairment (MCI) or dementia [78]. In these cases, XR environments are used to assess spatial memory, attention, and executive function by placing patients in virtual environments that simulate daily activities, such as navigating through a virtual city or managing household tasks [133]. These assessments provide a more accurate reflection of how patients function in their everyday lives, which can aid in the early diagnosis of cognitive decline and the development of personalized treatment plans [183].

XR is also being used in rehabilitation programs for individuals recovering from stroke or traumatic brain injury (TBI) [155]. In these programs, patients engage in virtual tasks that challenge their cognitive and motor skills, such as reaching for virtual objects, navigating through virtual spaces, or completing multitasking exercises [168]. These virtual environments allow clinicians to gradually increase the difficulty of tasks based on the patient’s progress, providing a personalized and adaptive rehabilitation experience [141]. Additionally, XR-based training has been used to help individuals with autism spectrum disorder (ASD) practice social cognition and communication skills by engaging with virtual avatars in controlled social scenarios [94].

Another promising application is in dementia screening and cognitive training for older adults [71]. XR tools allow older adults to practice tasks that are relevant to their daily lives, such as managing finances or remembering appointments, in a safe and controlled environment [184]. This training can help improve cognitive function and delay the progression of cognitive decline, offering a non-invasive and engaging approach to dementia care [158].

7.4. Challenges for Clinical Adoption

Despite the numerous benefits of XR-based cognitive tools, there are several challenges that need to be addressed before they can be widely adopted in clinical settings [44]. One of the primary barriers is the cost of XR hardware and software [43]. High-quality VR or AR headsets, motion tracking systems, and the powerful computers required to run these systems are expensive, making them inaccessible to many clinics and healthcare providers [1,44]. While the cost of XR technology is gradually decreasing, it remains a significant obstacle for widespread clinical adoption [43].

Another challenge is the technical complexity involved in implementing XR systems [122]. Developing, maintaining, and troubleshooting XR software requires specialized knowledge in both cognitive science and software engineering, which may not be available in many clinical settings [44]. Additionally, clinicians must be trained to use these systems effectively, which can require a significant investment of time and resources [39]. The integration of XR systems into existing clinical workflows also poses challenges, as these tools must be compatible with other diagnostic and therapeutic systems [44].

Regulatory issues are another hurdle for the clinical adoption of XR technologies [185]. The use of XR for cognitive assessment and training involves the collection of sensitive patient data, including behavioral and physiological data, which raises concerns about data privacy and security [14,139]. Regulatory frameworks must be developed to ensure that patient data is protected and that XR tools meet the necessary clinical standards for efficacy and safety [186].

7.5. Current Issues with Using XR for Cognitive Assessment and Training

7.5.1. Overview of Key Challenges Beyond Usability, UX, and Acceptability

While XR technologies offer significant potential for cognitive assessment and training, there are several challenges that go beyond typical concerns of usability, user experience (UX), and acceptability [25]. These challenges stem from the nature of immersive technologies and their interaction with the human body and brain [91]. One of the most prominent challenges is the issue of cybersickness, a phenomenon similar to motion sickness, which can occur when there is a disconnect between visual input and the body’s sense of motion [146]. Symptoms of cybersickness, such as nausea, dizziness, and eye strain, can limit the duration for which users can engage with XR environments and may negatively impact cognitive performance during assessments [30,31].

Visual fatigue is another issue that arises with prolonged immersion on XR tasks [16]. While immersion is one of the key benefits of XR, allowing users to feel fully engaged in virtual environments, extended exposure to immersive tasks can lead to mental and physical fatigue [16,31]. This can reduce the length of cognitive training programs, as users may become less motivated or less able to focus on the tasks at hand [119]. Managing the duration and intensity of XR sessions is crucial for maintaining user engagement and ensuring positive outcomes [31].

7.5.2. Underutilization of XR Technologies

Despite the advancements in XR technology, many current applications in cognitive assessment and training fail to fully exploit XR’s potential [25]. One of the most significant areas of underutilization is the integration of multiple modalities, such as combining visual, auditory, and haptic feedback with biometric data like EEG or GSR [8,122]. These modalities can enhance both the immersion and presence of the user, making the virtual experience more life-like and engaging [174]. For instance, the inclusion of haptic feedback (i.e., tactile sensations) can make interactions with virtual objects feel more realistic, while biometric data can provide real-time insights into cognitive and emotional states, allowing for more adaptive and personalized assessments [26,187].

The integration of multimodal systems not only enhances user experience but also provides valuable data that can improve the accuracy and effectiveness of cognitive assessments [101]. However, many current XR applications rely primarily on visual and auditory inputs, neglecting the full range of sensory and physiological data that could be captured [187]. This underutilization limits the potential of XR technologies to provide more comprehensive and personalized cognitive assessments and training programs [122].

7.5.3. Impact on Different Populations

The impact of XR technologies varies across different populations, and certain groups may experience unique challenges when using XR for cognitive assessment and training [14,181]. Children, for example, may be more susceptible to cybersickness due to their developing vestibular systems, and they may also require more engaging and interactive content to maintain attention during training sessions [163]. Older adults may face challenges related to physical discomfort when using headsets, as well as difficulties adapting to new technologies [145]. Additionally, individuals with cognitive impairments, such as those with dementia or brain injuries, may struggle with the complexity of XR interfaces or the intensity of immersive environments [80,86].

It is essential to consider these population-specific challenges when designing and implementing XR-based cognitive tools [25]. By tailoring the content, interface design, and hardware to the needs of different user groups, developers can improve the accessibility and effectiveness of XR technologies for cognitive assessment and training [87].

7.5.4. Hardware Limitations and Their Effect on the Immersive Experience

The immersive nature of XR technologies relies heavily on the quality of the hardware and software used, including VR headsets, motion tracking systems, and input devices [31]. However, current hardware limitations can detract from the overall user experience and reduce the effectiveness of cognitive assessments and training [44]. For instance, resolution and field-of-view limitations in VR headsets can lead to visual discomfort and reduce the realism of the virtual environment [188]. Inaccurate or lagging motion tracking or interaction can disrupt their performance, making it difficult for users to interact smoothly with virtual objects or environments [107].

Weight and ergonomics of XR headsets also play a critical role in user comfort [89]. Heavy or poorly balanced headsets can cause physical discomfort, particularly during extended sessions, which may limit the duration for which users can engage with XR tasks [79]. Addressing these hardware limitations is essential for improving user experience and ensuring that XR-based cognitive tools are accessible and comfortable for a wide range of users [25,88,90].

7.5.5. Strategies to Mitigate Issues and Improve the XR Experience

Several strategies can be employed to mitigate the challenges associated with using XR technologies for cognitive assessment and training [13]. To address cybersickness, developers can implement techniques such as dynamic field-of-view modification or restriction, which reduces the user’s field of view during rapid motion to minimize the sensory disconnect between visual input and physical movement [189,190]. Adjusting the frame rate of the virtual environment and optimizing the responsiveness of motion tracking systems can also help reduce motion sickness and improve user comfort [43,191] .

To prevent immersion fatigue, it is important to design XR tasks with appropriate session lengths and breaks to ensure that users do not become overwhelmed or fatigued by prolonged exposure to virtual environments [31]. Incorporating natural transitions or periods of lower cognitive demand within training sessions can help maintain user engagement without causing cognitive overload [141].

Finally, addressing hardware limitations through the development of lighter, more ergonomic headsets and improving the accuracy and responsiveness of motion tracking systems will be critical for enhancing the user experience [43]. As XR technology continues to evolve, these improvements will contribute to more comfortable and immersive cognitive assessment and training tools, making them accessible to broader populations [25].

8. General Discussion

This review highlights the transformative potential of Extended Reality (XR) technologies in cognitive assessment and training. One of the standout features of XR is its ability to provide high ecological validity by immersing participants in realistic, interactive environments that closely mimic real-world tasks [1,136,166]. This is a significant advantage over traditional assessment methods, which often isolate specific cognitive functions in artificial, controlled settings [54,137]. Through immersive simulations, XR environments enable a more accurate evaluation of cognitive abilities as they are applied in everyday life, from memory and attention to decision-making and problem-solving [6,37].

Another key finding is the growing importance of multimodal integration in XR applications [25]. Multimodal systems, which combine inputs from sources such as galvanic skin response (GSR), electroencephalography (EEG), eye tracking, hand tracking, and body tracking, offer a comprehensive view of the user’s cognitive and emotional state [101,192]. These systems not only provide richer datasets but also enhance user immersion by making the virtual environment more responsive and personalized [119]. For example, integrating haptic feedback into cognitive tasks can enhance realism, while physiological data can enable adaptive tasks that adjust in real time based on user stress levels or cognitive load [17,193]. However, current XR applications are not yet fully utilizing the potential of multimodal systems, which represents a critical area for future development [25].

In terms of user experience, XR has shown significant advantages, particularly in its ability to engage users through immersive, interactive tasks [79]. This is especially relevant for cognitive training programs, where motivation and adherence are key to success [175]. XR environments provide a more engaging and dynamic experience than traditional methods, which often become repetitive and disengaging over time [163,173]. However, challenges such as cybersickness, immersion fatigue, and hardware limitations continue to impact user experience, especially during prolonged sessions [23]. Addressing these issues is essential to improving the efficacy and accessibility of XR tools [44].

8.1. Key Limitation: Underutilization of Available Technology

Despite the many advantages of XR in cognitive assessment and training, a major limitation is the underutilization of available technology. Many current applications fail to fully exploit the potential of XR, particularly in the integration of multiple modalities [25]. While XR systems are capable of capturing a wide range of biometric data, including eye movements, EEG signals, and physiological responses [74,125], many applications still rely on visual and auditory inputs alone. This underutilization limits the potential of XR environments to provide a truly immersive and responsive experience [25].

Integrating multimodal data streams can significantly enhance the quality of both cognitive assessments and training programs [12,152]. For instance, real-time monitoring of physiological responses such as heart rate or skin conductance could be used to adjust task difficulty, ensuring that participants remain challenged without becoming overwhelmed [111]. Similarly, integrating haptic feedback can make interactions with virtual objects more lifelike, further improving immersion and the ecological validity of tasks [193]. The full potential of these technologies remains largely untapped, representing a critical area for future research and development [26].

8.2. Implications for Future XR-based Cognitive Tools

The findings of this review have several important implications for future XR-based cognitive tools. First, there is a clear need to prioritize the integration of multimodal systems in cognitive assessments and training [25]. By combining multiple data streams, such as EEG, GSR, and eye tracking, future XR applications can provide a more holistic view of cognitive processes and emotional states, leading to more accurate assessments and more effective training programs [101,124]. Additionally, the real-time adaptation of tasks based on these data streams can enhance user engagement and improve learning outcomes [12,117,152].

Second, future XR-based tools should focus on increasing personalization and adaptability. By leveraging real-time data collection and analysis, XR systems can adapt to the cognitive and emotional states of individual users, providing a more tailored experience [94,144]. This adaptability is particularly important for clinical populations, such as individuals with cognitive impairments or neurological conditions, who may require customized training programs to meet their specific needs [51].

Finally, the ongoing improvements in XR hardware—such as lighter, more ergonomic headsets and more responsive motion tracking systems—will play a crucial role in making XR tools more accessible and effective for a broader range of users [1,145]. As these technological advancements continue, it will become increasingly feasible to implement XR-based cognitive tools in clinical, educational, and even home settings [32,185].

8.3. Potential for Enhancing Both Cognitive Assessment and Rehabilitation

XR has the potential to significantly enhance both cognitive assessment and rehabilitation. In terms of assessment, XR environments allow clinicians to evaluate cognitive functions in more realistic and complex scenarios, providing a deeper understanding of how individuals perform in everyday tasks [4,103]. For example, XR can simulate driving, navigating unfamiliar environments, or managing household tasks, all of which provide more meaningful insights into cognitive abilities than traditional tests [73,160].

In rehabilitation, XR’s adaptability and immersion make it an ideal tool for improving cognitive functions [97]. Rehabilitation programs can be tailored to the specific needs of patients recovering from brain injuries, strokes, or other neurological conditions, allowing them to practice real-world tasks in a controlled and safe environment [51]. Additionally, the immersive nature of XR can increase patient engagement, leading to more consistent participation and better outcomes [13].

8.4. Unresolved Issues: Usability in Specific Populations, Regulatory Hurdles

Despite the potential of XR in cognitive assessment and training, several unresolved issues remain. One of the most pressing challenges is the usability of XR technologies in specific populations, such as older adults or individuals with cognitive impairments [80]. These groups may experience difficulties with XR interfaces, which can be physically uncomfortable or cognitively overwhelming [79]. Addressing these usability concerns is critical for ensuring that XR tools are accessible to a wide range of users [14].

Additionally, regulatory hurdles pose a significant challenge to the widespread adoption of XR in clinical settings [194]. As XR technologies become more sophisticated and start collecting more sensitive biometric data, there is an increasing need for clear regulatory frameworks to ensure patient privacy and data security [14,186]. Furthermore, XR tools must meet rigorous clinical standards for efficacy and safety, which may slow down their integration into mainstream healthcare practices [44,145].

9. Conclusion

Extended Reality (XR) technologies have shown immense promise in transforming the landscape of cognitive assessment and training. By providing immersive, interactive environments that simulate real-world tasks, XR tools offer a level of ecological validity and engagement that traditional methods cannot match. Through the integration of multimodal systems, real-time data collection, and adaptive task design, XR tools have the potential to offer more personalized, effective, and engaging cognitive assessments and training programs.

However, the full potential of XR has yet to be realized. Future research should focus on addressing the usability challenges that certain populations face when using XR technologies. It is essential to develop interfaces that are more accessible and comfortable for older adults, individuals with disabilities, and others who may struggle with current XR systems. Additionally, there is a critical need for more research on the integration of multimodal systems. Combining data streams from multiple sources, such as EEG, GSR, and eye tracking, can enhance the accuracy of cognitive assessments and make training programs more responsive to the needs of individual users.

As XR technologies continue to evolve, they hold great potential for both clinical and educational applications. In clinical settings, XR can revolutionize neuropsychological testing, cognitive rehabilitation, and therapeutic interventions by providing more ecologically valid, adaptive, and engaging tools. In education, XR can offer personalized cognitive training programs that enhance learning outcomes and provide real-time feedback to students and educators. As these technologies become more accessible and cost-effective, it is likely that XR will become an integral part of cognitive assessment and training across a wide range of settings.

In conclusion, the findings of this review emphasize the necessity of systematically examining and analyzing the validity and quality of available multimodal XR environments in cognitive assessment and training. Conducting a systematic review in this area will enable a more detailed and robust analysis of these environments, with a focus on accessibility for clinical populations, and on multimodal data processing techniques in cognitive evaluation and rehabilitation. Ultimately, this review will propose a framework for developing guidelines for researchers and developers, ensuring that XR environments are accessible to health professionals and clinical populations.