Creating an Immersive Virtual-Reality Space for Multiuser Synchronous Co-Located Collaboration: Design Considerations and Influencing Factors

The last decade has witnessed rapid development of immersive virtual reality (IVR) and its application in various contexts. However, its application in supporting real-time virtual collaboration has been quite rare due to technical barriers and the lack of validated design principles. To address this research gap, this study designed and developed an IVR space to enable multi-user synchronous co-located collaboration. An evaluation study (N = 95) was conducted to explore its useful design considerations and the influencing factors for collaboration experience. The IVR space was enabled by the simultaneous localization and mapping (SLAM)-based inside-out tracking technique and was informed by four essential design considerations for promoting effective collaboration in IVR, namely, the role script, learning task, collaboration mechanism, and communication design. The study results revealed that students in general were satisfied with their collaboration experience in IVR, with social presence and collaboration competency as significant predictors of collective efficacy and social experience. Based on both quantitative and qualitative results, this study proposes four validated principles for designing effective IVR spaces to support synchronous co-located collaboration.

Keywords:

Subject: Computer Science and Mathematics - Computer Science

1. Introduction

Immersive virtual reality (IVR) is considered a promising space for delivering authentic and vivid experiences [1,2]. IVR enables students to gain a presence in a highly immersive virtual environment with its capacity to afford high-fidelity simulations, rich sensory stimuli, and multimodal interaction [3]. The sense of presence describes the illusive perception of being present in an artificial or remote environment [4], and immersion is often referred to as a feature of VR that provides a convincing illusion [5]. The past decade has witnessed IVR being applied in various domains, such as entertainment [6], medicine [7], engineering [8], consumption [9], and education [10]. However, previous studies on IVR have mainly focused on individual experience, and little is known regarding users’ collaboration experience in IVR [11].

The implementation of collaboration in IVR is hampered by several challenges such as geographical distance, temporal distance, and perceived distance [12]. Collaborative work can be classified according to the modalities of the space and time taxonomy as follows: synchronous co-located, asynchronous co-located, synchronous distributed, and asynchronous distributed collaboration [13,14]. Both verbal and nonverbal interactions, which are pivotal for effective group work, are diminished by the asynchrony of time and space during collaboration [15]. This reduction in interaction negatively affects trust between team members and increases the perceived distance between individuals, ultimately undermining the collaborative experience and effectiveness [16,17]. For instance, a distributed virtual collaboration has been explored by several studies [18,19], where communication primarily takes place through formal texts. However, this mode of communication still poses challenges to establishing confidence and trust within the virtual space.

Synchronous co-located virtual collaboration is an innovative mode characterized by users entering the virtual space while being physically present in the same location, which enables direct communication between users without any mediation [14]. The social and emotional experiences between collaborators are enhanced through unmediated interaction, which are strongly associated with learners' engagement and performance [20]. For example, Liang et al. [21] designed a virtual mall where two players stayed in the same physical space and wore VR head-mounted displays (HMDs), which helped them communicate naturally to find goods collaboratively or competitively. Positive affective outcomes, such as increased enjoyment, intrinsic motivation, and social presence, could be facilitated by immersive and interactive experiences [22], which can benefit learning performance by focusing learners’ attention on the task and enabling generative cognitive processing through more frequent and natural social dialogue [23].

However, synchronous co-located virtual collaboration has rarely been explored owing to its additional technological requirements and a lack of design principles. First, the commonly used outside-in tracking technology that enables multiuser copresence in IVR has proved to be problematic [24]. It is susceptible to glitches caused by occlusion and lighting issues, and it also requires many cameras and sensors to be mounted in advance, which increases the total cost and preparation time. Second, a list of validated design assumptions for promoting effective collaboration in IVR is still lacking owing to insufficient design cases and intervention studies in the literature. It remains unknown whether effective design principles for collaboration in brick-and-mortar settings still apply to IVR spaces. It is imperative to overcome these two challenges to enable collaboration in IVR so that students can harness its pedagogical benefits, including social dialogue [25], shared inquiry [26], and participatory equity [27].

To address this research need, we conducted a design case that used a simultaneous localization and mapping (SLAM)-based inside-out tracking technique to design and develop an IVR collaboration space. By searching the literature on collaboration design, we identified the following four design considerations for the IVR space: role script, task, collaboration mechanism, and communication design. Based on both quantitative and qualitative evidence, these considerations were empirically validated. Additionally, a series of psychological factors were examined as possible predictors of collaboration experience in IVR, enriching the understanding of instructional design in such a context. According to Boling [28], a good design case should offer in-depth explanations of design rationales, multidimensional descriptions of experiences, and documentation of factors that influence the design and implementation process. Consequently, we sought to answer the following three research questions:

What are the overall collaboration experiences in the IVR space?
What are the effective design features of the IVR space that support collaboration experiences?
What are the factors that influence collaboration experiences in the IVR space?

2. Literature review

2.1. Key constructs of collaborative experience

Most scholars are concerned with the effects of collaboration on learning outcomes, such as cognitive improvement, behavioral skill development, and work completion, but less research has been done on collective efficacy and social experience, which are strongly related to collaborative effectiveness [29,30]. For example, Elms et al. [31] investigated collective efficacy as a key predictor of team effectiveness, which was consistent with the argument of Goncalo et al. [29] that collective efficacy contributes to team performance owing to its ability to motivate members and provide direction for cohesive effort. Additionally, learners’ cognitive processes and learning outcomes tend to be profoundly affected by social experience, like social-emotional experience [32].

Collective efficacy is defined as “a group’s shared belief in its conjoint capabilities to organize and execute the course of action required to productive given levels of attainment” [34]. In other words, collective efficacy reflects a team’s shared confidence in its ability to perform an assigned task well to meet common goals [30]. Collective efficacy has a strong and positive association with team effectiveness (e.g., cognitive skills in computer-mediated communication) [34], performance in programming [35], and sports team performance [36]. Social experience is about socializing, relationship formation and teamwork, and a feeling of real or virtual connectedness [37], which has significant effects on group collaboration effectiveness.

Although collaboration pedagogy is widely used in multiple contexts owing to its effectiveness, the collective efficacy and social experience of collaboration are influenced by many factors, including the individual’s social ability, extroversion, social presence, and past team performance [38]. Social ability usually refers to a person’s capacity to interact, communicate, and use shared resources with peers to accomplish complex tasks [39]; it consists of three key prominent constructs: social presence, social navigation, and social connectedness [40]. Social presence refers to the degree to which one perceives the sense of being in a community and belonging to a group [39,41], and it can predict students’ collaboration experience and satisfaction [42]. Additionally, individuals high in extroversion are likely to be gregarious, assertive, and sociable, as well as to enjoy participating in groups and to believe that group work is a productive use of class time [38].

2.2. Collaboration in IVR

A highly immersive, interactive, and authentic virtual space is created by IVR technology, which completely separates users from the physical world using devices such as HMDs or mobile viewers (e.g., Google Cardboard) [43]. IVR may benefit collaboration by enabling higher levels of representational fidelity and embodiment, (non-)verbal communication, and avatar realism [44]. Several studies have explored the collaboration effectiveness in an IVR environment; the results indicate that IVR can support communication, understanding, and teamwork for medical training and education [45] and that it can be used to effectively assess social communication skills [46]. Other benefits include enhanced creativity compared with conventional tools and systems [47], empowered empathetic behavior [48], and improved biological acquisition [49].

According to the computer-supported cooperative work time-space matrix, most existing collaboration types in IVR environments are categorized as asynchronous co-located, synchronous distributed, and asynchronous distributed collaboration; there has been relatively less research conducted on synchronous co-located collaboration [13]. In the first three collaboration types, individuals typically use different devices and engage with the virtual content at different times [11,50]. These patterns of IVR collaboration present unequal perspectives that can lead to communication difficulties and poor collective efficacy, especially when different devices are used. Contrarily, synchronous co-located collaboration enables the use of the same devices and equal interaction among peers, which is essential for enhancing presence and facilitating seamless communication [11,51]. For example, Ghoshal et al. [52] developed a co-located multiplayer game and found that co-located collaboration in VR could evoke a strong sense of presence. Drey et al. [11] obtained similar results, finding that symmetric and co-located collaboration led to higher presence and immersion than other collaboration types and also positively influenced the cognitive load.

However, limited research has been done on synchronous co-located IVR collaboration, particularly involving more than three users. Possible reasons for this include the requirement for additional technology and space, as well as the absence of design principles for symmetric IVR collaboration [12,21].

2.3. Design considerations for collaboration in IVR

By searching the existing literature on collaboration design, we identified four key design considerations for IVR spaces supporting effective collaboration. The first is role script design. Considering the imaginative features of VR, designers can create vivid and diverse function avatars and establish rules that encourage multiple learners to collaborate toward a shared goal [49]. For example, Thompson et al. [50] developed a Cellverse virtual space, established roles (explorers and navigators), and distributed resources between players. The explorers are responsible for completing tasks that involve identifying protein structures and tracking processes within the cell, and the navigators guide the explorers and provide helpful clues to working with other team members. This sharing of responsibilities in design builds positive interdependence and individual accountability between team members.

The second is task design. As emphasized by Won et al. [53] and Ferguson et al. [1] regarding the characteristics of the IVR story structure, multiuser collaborative tasks should be narrative-driven, attractive, and challenging to engage users emotionally and intellectually in common objectives. For instance, Drey et al. [11] constructed a VR forest environment where students could roam freely, see three forest animals, and learn their typical characteristics. This IVR task demonstrated an authentic and interesting virtual scenario to attract users to find animals located in different areas. However, the approach of introducing the characteristics of animals mainly focused on content in a direct presentation, lacking challenge and exploration.

The third is collaboration mechanism design. This principle refers to design content and instructions that are equally accessible to all users [49]. This can be achieved through task design, role design, or by allowing students to collaboratively discuss strategies. Several studies have been designed to allow collaborators to access the VR world through different devices [14,49]—for example, two collaborators who physically stayed in the same space, one of them wearing HMD and the other using a tablet to experience VR separately. However, there existed unequal involvement among students because these IVR collaboration rules allowed one to experience a highly immersive virtual world and the others merely to access information through a 2D or 3D flat screen, which potentially influences effective and worthwhile collaboration.

The fourth is feedback and communication mechanism design. This involves integrating virtual-space feedback design with real-time communication design while aligning real and virtual locomotion. This aligns with the "actional" and "social" features of IVR-based learning environments proposed by Dede et al. [54]. Eiris et al. [18] established a 360° panoramic virtual learning space for masonry and architectural knowledge learning, where students can use their keyboard and mouse interfaces to direct a color-coded virtual laser pointer. This IVR space enabled students to point to 360° panoramic content while describing their thoughts using a third-party voice system (Zoom). Students were more engaged in collaborative tasks when interactions were varied and feedback was immediate.

3. Design and development of an IVR collaboration space

The IVR collaboration space developed by the research team was a multiplayer collaborative fantasy game called The League of Castle Defenders. The game provides opportunities for five players to collaboratively complete interactive missions. It features three roles with different functions that help players build shared responsibilities and positive interdependence between team members. According to the collaboration mechanism and task design considerations, the game scenarios are inspired by ancient Chinese culture and present heroes fighting against enemies to defend their homeland, which requires players to defend a castle with their unique role functions. Additionally, this game was designed specifically so that players situated in co-located physical space can communicate strategies and provide timely feedback when players’ conditions have changed. Figure 1 shows a prototype of the game, which includes three game roles: enemies, a castle, and a warship, among other elements. The focus of the game is that the three roles collaboratively resist the attacking enemies.

3.1. Mission and role design

The mission of defending a castle is accomplished after the players slay five enemies before the castle gates fall and while at least one player remains alive. During the game, five enemies fire random artillery at 5-second intervals from the cannons, which cause 100 damage points to either the castle or the players. The castle has 1,000 health points and can withstand up to 10 artillery attacks. Each enemy is set to 1,000 health points, while the defender is initially set to 500 health points, and the other two roles are set to 50. As shown in Table 1, the defender can hold a shield against the enemies’ artillery fire to protect teammates or the castle from attack; the attacker can shoot arrows at the enemy. Each attacker initially has 10 arrows, and one arrow can generate 10 points of damage; the collector is responsible for collecting supplies by opening boxes, including an ammunition supply box to replenish arrows for the attackers and a medical supply box to restore a defender’s health by 100 points. Players are eliminated from the game when their health points fall below zero.

The players need to collaboratively complete the missions of the game to achieve victory. Figure 2 shows the prototype design of the three roles. While executing missions, the defender needs to stand at the front of the team to block the artillery attack and, at the same time, needs to anticipate the flight path of the artillery and pay attention to the changes in health points. The attacker needs to pay attention to the line of the artillery while ensuring accurate shooting. The collector needs to move quickly in a large area at the back of the team near the walls to find supply boxes and prioritize the correct supply boxes for the team’s specific situation.

3.2. Human-computer interaction design

The researchers designed interactions between the player and the virtual space or virtual objects to enhance realism and users’ immersion. The researchers classified the interactions into visual and action interactions to highlight the large spatial multiplayer collaboration in VR. For visual interaction, players can check their health points or arrows by observing the status bar. Players who select the defender role can observe their health points through the status bar, while players who selected the attacker role can observe their health points and remaining arrows, as shown in Figure 3. Additionally, people outside the game field can see the status of each player in real time by looking at the right side of the computer interface. For example, when a player has zero health points, the status bar appears grayed out.

In terms of action interactions, players can run and move fully in space to complete missions. The players who choose the attacker role pick up the in-game bow and shoot arrows in the virtual space by raising their arms and pressing the button on the VR controller. The players who choose the defender role also lift their shields by raising their controllers. The players who choose the collector role roam in the virtual space to collect and open the supply boxes by wielding their controllers.

3.3. Collaborative mechanism design

This game requires three roles to use their character features and equipment, thus interdependently collaborating to achieve game success. The defender, standing at the front of the team, needs to try to block each launch of the enemy’s artillery fire, thus ensuring the safety of other teammates. The attacker, hiding behind the defender, needs to accurately attack the enemy as fast as possible, which is the key to achieving victory. The collector, at the end of the team, needs to communicate with the other players to obtain supplies to ensure that the defender has enough health points to withstand enemy fire and that the attacker has enough arrows to inflict damage on the enemies. Overall, without the defender on the team, the other two roles would soon be injured by artillery fire; without the attacker, the enemy would continually attack the castle, causing its gates to collapse. Furthermore, without the collector, the attacker would not be able to obtain weapons, and the defender would have difficulty blocking artillery fire owing to insufficient health points. Therefore, to achieve victory and complete the game together, the three roles need to have positive interdependence, prompt communication, and effective collaboration.

3.4. Technical features

3.4.1. Hardware

The device used in this study was the Oculus Quest 2, an all-in-one VR device developed by Meta. It offers high resolution and strong refresh-rate capabilities, resulting in immersive and realistic visual effects. The Oculus Quest 2 simplifies the integration of the VR headset and Leap Motion device by providing 360° head-movement tracking and gesture tracking without the need for controllers. (We still used controllers in our IVR game for steadier performance and prolonged battery usage.) Consequently, users can engage in VR experiences without being limited by physical space, making the device well suited for large-scale multiplayer VR activities.

3.4.2. Software

The collaboration space for this study was created using Unreal Engine 4 (UE4), version 4.26.2. UE4 offers a comprehensive visual scripting system that enables researchers to build diverse virtual environments and programmable visual representations. The underlying layer of UE4 is implemented in the C++ programming language, which effectively improves the programming quality of large-scale programs. At the same time, UE4 provides a complete visual scripting system: the Blueprint tool. To make development more efficient, the Blueprint tool and C++ tools in UE4 can be used in tandem. The game was also developed using Microsoft Visual Studio, which is a largely complete set of development tools. UE4 works perfectly with Visual Studio, enabling users to write project code quickly and easily.

3.4.3. Algorithm

To facilitate real-time online interactions among multiple participants, this virtual collaboration game relies on a spatial-positioning technology called “inside-out tracking.” Inside-out tracking uses the cameras on the HMD to detect changes in the external environment. It also leverages SLAM algorithms and depth cameras to calculate the precise position of the HMD in space. This SLAM-based inside-out tracking technology enables multiuser VR spatial localization by sharing SLAM data and ensuring real-time synchronization among users. Combining the rendering mechanism of virtual scenes allows for the copresence of multiple users in virtual collaboration spaces, leading to a heightened level of social immersion.

Consequently, The League of Castle Defenders is a multiplayer VR game that was designed to provide visual, auditory, and physical interactive experiences. Figure 4 depicts the complete process, starting from the beginning of the game and leading to victory. Although the process is not overly complex, successful completion requires effective communication and collaboration between all players involved.

4. Evaluation study

4.1. Participants

The participants were 95 undergraduate students (30 men and 65 women) recruited from a Chinese university via advertisements posted on a social platform. The minimum sample size calculated by the software G*Power 3.1.9.4 was 92 (f² = 0.15, α err prob = 0.05, power = 0.8, number of predictors = 5), which was adequate to conduct the linear multiple regression in this study according to the approach of Faul et al. [55]. The participants' ages ranged from 18 to 21 years (Mean = 21.68, Standard Deviation [SD] = 1.93). Based on the information that they provided in a formal questionnaire before the experiment, participants came from a wide range of academic programs (e.g., history, journalism, education, computer science, and chemistry) and reported being unfamiliar with playing VR games. All the participants had normal vision or corrected-to-normal vision and received a small reward to compensate them for participating. Informed consent was obtained from all the participants before conducting the study. The study was approved by the Institutional Review Board of Central China Normal University (IRB No. CCNU_IRB_202103_019).

4.2. Procedure

Before the experiment, participants were required to complete a questionnaire on a scale that included basic information, personality, collaboration ability, and VR experience. The participants were then randomly divided into 19 groups of five participants and informed of the accurate time and location of the formal experiment by message. The experimental site chosen was in the school’s indoor gymnasium, which provided a spacious, safe environment.

The formal experiment was divided into three phases. The first phase was for observing and generating a collaboration strategy. Before a group entered the IVR collaboration space to complete the collaborative task, the researcher introduced the task scenario, game mechanism, role functions, and individual responsibilities. All members of the group observed the performance of the group before them that was playing on the laptop and then chose an assignment strategy for the three roles through face-to-face negotiation. The second phase was the formal IVR experience phase. Five research assistants assisted participants in wearing the HMD to ensure that they could clearly see the objects in the IVR environment, guided them on how to use the VR equipment, and explained the rules of the game. Then, under the guidance of the research assistants, the participants chose their roles according to what they had discussed, and then they were moved to the specific location to realize an exact match between the physical and virtual locations. During the experience phase, participants could communicate and exchange information by shouting aloud. Finally, participants played the VR games under the supervision of three research assistants, who ensured their safety in the physical tracking space. Figure 5 illustrates the game screenshot and the corresponding physical space as the students collaborated to complete the game task.

After the VR game, each participant was required to complete a questionnaire on a scale related to their VR experience. Additionally, we randomly selected seven groups and conducted semi-structured interviews to gain an in-depth understanding of the collaboration experience through the following questions: Can you briefly describe your gaming experience this time? What designs and experiences impressed you? Did anything interesting or difficult happen during the game? The interview duration for each group ranged from 13 to 25 minutes, and the recorded interviews were later partially transcribed, generating a text content of 11,244 Chinese words for qualitative analysis.

4.3. Data collection and analysis

This study included quantitative and qualitative data. The quantitative data were collected through a five-point Likert questionnaire after the IVR game experience. The questionnaire comprised three parts. Part 1 of the questionnaire featured basic information about the participants, including their personality, collaborative competence, and VR readiness (see Appendix A). Collaborative competence measured participants’ communication skills, teamwork ability, leadership, and collaboration experiences; these items were modified from the scales designed by Radziej et al. [56] and Duong [57]. Moreover, Cronbach’s alpha coefficients for personality, collaborative competence, and VR readiness were 0.728, 0.944, and 0.809, respectively, which are considered acceptable in nonclinical applications owing to the small number of scale items.

Parts 2 and 3 of the questionnaire assessed the IVR collaboration experience (collective efficacy and social experience) and the IVR presence (spatial presence and social presence) (see Appendix A). The items related to IVR collaboration experience were based on the scale for digital collaborative games proposed by Vidergor [37], and the items related to IVR presence were based on the scale for collaborative augmented-reality games proposed by von der Pütten et al. [58]. Cronbach’s alpha coefficients for the subscales all ranged from 0.708 to 0.942, which are considered acceptable when the number of items is less than 10, according to Bland and Altman [59]. The Kaiser-Meyer-Olkin measure of sampling adequacy for collective efficacy, social experience, spatial presence, and social presence was 0.812, 0.880, 0.728, and 0.737, respectively, and Bartlett’s test of sphericity value was significant (p = 0.000). Further, we calculated the indices for determining convergent validity. According to Campbell and Fiske [60], a composite reliability (CR) greater than 0.6 and an average variance extracted (AVE) values greater than 0.5 are indicators of good convergent validity. Our results (collective efficacy: AVE = 0.7483, CR = 0.922; social experience: AVE = 0.641, CR = 0.9142; spatial presence: AVE = 0.632, CR = 0.8313; and social presence: AVE = 0.408, CR = 0.7702) meet the criteria in general, indicating an acceptable convergent validity.

A descriptive statistical analysis and regression analysis were applied to the quantitative data. The former was used to understand the participants’ experience based on IVR experience and basic information. The latter was used to explore the factors that influenced students’ collaboration experience in IVR. Consequently, collective efficacy and social experiences were considered outcome variables, and all other variables (e.g., personality, collective competency, social presence, and spatial presence) were entered into regressions as predictors. In terms of interview data, we used various coding techniques (e.g., Structural, In Vivo, and Evaluation coding) described by Saldana [61] to qualitatively analyze the interview transcripts, aiming to assist in the meaningful interpretation of participants’ collaboration. The coding was performed mainly by the first, fourth, fifth, and sixth authors, and any disputable issues that had arisen during the coding process were resolved through weekly meetings attended by all authors. Finally, 181 codes were identified from the transcribed interviews. Then, we sorted the codes for further analysis and theme generation, as suggested by Bazeley [62]. As shown in Table 2, we derived 37 nodes and created nine categories. Appendix B lists the complete qualitative coding results.

4.4. Quantitative findings

4.4.1. Overall experience of collaboration in IVR

To understand participants’ experience during IVR-based collaboration, we conducted a descriptive analysis. As shown in Table 3, participants were satisfied with the IVR collaboration experience, which can be seen from the high evaluation of all dimensions (all Mean > 3), especially in spatial presence (Mean = 4.22, SD = 0.66) and social experience (Mean = 4.11, SD = 0.66).

4.4.2. Factors influencing the collaboration experience in IVR

Next, to determine the relationships among all of the variables, a correlation analysis was performed. Table 4 shows the preliminary results for the descriptive statistics and Pearson’s correlation coefficients. First, an extroverted personality and collaboration competence were weakly positively correlated with spatial presence (r = 0.226 and 0.375, respectively) and social presence (r = 0.227 and 0.302, respectively). In addition, spatial presence and social presence had a medium correlation (r = 0.706). Additionally, VR readiness was not correlated with collective efficacy, while an extroverted personality (r = 0.303) and collaboration competence (r = 0.297) were weakly correlated with collective efficacy, and spatial presence (r = 0.459) and social presence (r = 0.521) were moderately positively correlated with collective efficacy. Moreover, an extroverted personality and collaboration competence were weakly correlated with social experience, while spatial presence and social presence were moderately correlated with social experience.

To find the most parsimonious model for predicting collective efficacy and social experience, we used the backward-elimination multiple linear regression analysis approach. Using collective efficacy as the dependent variable, the final model included two independent variables (IVs): extroverted personality and social presence. The regression analysis suggested a significant final model (F _{(2, 92)} = 19.577, p < 0.001) in which the IVs explained 28.3% of the variance in collective efficacy (R² = 0.283), and only social presence (β = 0.473, p < 0.001) was a significant predictor of the collective efficacy. In terms of social experience, the results showed that in the final significant model (F _{(2, 92)} = 44.161, p < 0.001), the IVs explained 47.9% of the variance (R² = 0.479), and collaboration competence (β = 0.176, p < 0.05) and social presence (β = 0.626, p < 0.001) were significant predictors. The collinearity analysis further reported an acceptable multicollinearity level among the independent variables [63]. Table 5 shows the regression model summary.

4.5. Qualitative findings

We identified four main themes through the qualitative analysis of interviews. The first three described the specific aspects of participants’ experiences with the VR game and revealed the possible influencing factors such as technical affordances and individual attributes. The themed findings provide data triangulation and an explanation of the quantitative results, and they are detailed in this section. The fourth theme identified several issues regarding the current game design and technologies, which improved our understanding of participants’ collaboration experiences in VR and provided a basis for future VR game refinement. Table 2 lists the themes, categories, and nodes.

4.5.1. Themed finding 1: Spatial presence and natural feedback afford a sense of immersion in the IVR collaboration space

Although the students admitted that they had limited past VR experience, their unease and self-doubt rapidly disappeared after entering virtual world, and they engaged with the created virtual environment immediately. A possible explanation is that the virtual world was highly authentic, with the same spatial and temporal rules as the physical world, thus greatly reducing the time needed for adaptation and adjustment. As one student explained, “As the artillery fire flew toward me, I felt the danger and needed something to block.” Additionally, the fluency and details of the virtual scenarios could enhance the immersion level of students. Especially when seeing smoke appearing after the explosion of artillery or the enemy fall, they found it extremely “playful,” “exciting,” and even “intimidating.”

The immersion could be further enhanced when various learner actions were triggered in the virtual space, with effects similar to those in the real world; for example, the speed of running in reality matched the speed of moving in the virtual space. Additionally, the gesture interaction was an important factor in enhancing immersion; as explained by a student playing the role of the attacker, “I can press the button on the handle to shoot arrows to attack the enemy, which is the same as pulling the trigger to actually shoot an arrow.” When interactions in VR were designed to mimic interactions with everyday objects, the students could quickly accept the game setting, which enhanced learner engagement.

4.5.2. Themed finding 2: Co-presence in IVR is a key feature that improves social presence and collaboration experience

Compared with computer-mediated communication in VR worlds, the synchronous co-located collaboration space enabled a high level of co-presence through real-time negotiation, problem-solving, and reflection among the students. According to Strojny et al. [64], the perceived presence of group members affects one’s behavior and attitude directly or vicariously, thus facilitating the social dynamics and decisions within the group. This phenomenon is also known as social facilitation. Consequently, we should design co-presence into the IVR collaboration space to promote social facilitation. The responsibility and interdependence designed into the roles can promote communication between members and enhance co-presence. One student who played the role of attacker stated that “the Collectors were able to give ample resources,” and with the “Defender in front against the artillery fire,” he could go forward without fear.

Additionally, the co-presence and social presence in the VR collaboration were further enhanced through real-time mapping of the physical and virtual space. When activities in the virtual space are comparable to those in the physical world, students will carefully consider individual skills that are suitable to each role and make their role assignments and game plan accordingly. The sense of co-presence is further enhanced through the overlapping of virtual and physical tasks, which leads to greater social presence and collective efficacy. For example, those who run faster in the real world than other participants are better suited for the role of collector, while those with better communication skills may be better suited for the role of attacker, who needs to constantly command the team during gameplay.

4.5.3. Themed finding 3: Individual competence rather than personality influences the collaboration experience in IVR

Although an extroverted personality is a vital factor that affects collaboration in traditional face-to-face environments, its influence was diminished in VR environments. When wearing the HMD and entering the VR environment, students create an avatar to collaborate with others, instead of acting as themselves, which makes it more comfortable for introverted students to become involved in collective tasks. For example, a self-reported introverted student immersed in IVR would keep trying to talk to his teammates to ask for more supplies when he realized he was low in health points. However, the other two extroverted students did not collaborate as well in the IVR collaboration space, noting that

“…We discussed the division of labor before game started, but there were still some unexpected issues when artillery fire appeared. Sometimes two of us bumped into each other trying to block the same artillery fire, and sometimes no one blocked a particular fire…”

Moreover, compared with the individual personality, students’ collaborative competencies, such as communication, negotiation, and decision-making, were crucial factors that influenced the social experience of VR environments. The groups that completed the game tasks preferred to negotiate and develop an effective role-assignment strategy, and they were also skilled in finding ways to contribute to teamwork based on their abilities. For example, an introverted student who had achieved victory chose the role of collector because she was used to roaming in a mobile game. Similarly, as an extroverted student from the winning group explained,

“I’ve always been a novice at playing games, so I thought I might not be able to shoot in this game, but the Defender only needs to run back and forth, and I thought I’d be better able to do it, so I chose the Defender……”

5. Discussion

This paper presented a design case of a highly immersive, naturalistic, and multiuser IVR collaboration space enabled by SLAM-based inside-out tracking technology. To explore the students’ experience in this virtual space and the factors that affected collective efficacy and social experience, an evaluation study was also conducted. Consequently, this design case provided important empirical evidence that instantiates and validates the four design considerations for creating effective synchronous, co-located collaboration in IVR. In this section, we provide an in-depth discussion of the four design considerations, with elaborations on their theoretical underpinnings and pragmatic implications.

5.1. Role and collaboration mechanism design

The fact that social presence can significantly predict students’ collective efficacy and social experience in IVR highlights the importance of designing a proper role script and collaborative mechanism for the IVR learning space because these two design features are known to cultivate a strong social presence during group collaboration [50,65]. Our interview and observation results showed that the role script (featuring three distinctive functions) and collaborative mechanism were necessary and useful in structuring collaboration and delivering co-presence in the novel context of IVR. Such role design allowed the students to engage in meaningful negotiation and joint actions, which enabled the whole collaborative system to accomplish the common goal in a more effective and standardized way, as argued by Strijbos and Weinberger [66]. The reciprocal relationship between the role design and collaboration mechanism was also emphasized by Laal [67], who argued that well-designed role scripts can promote group awareness and participation and thus benefit the collaborative process with increased interdependence. Additionally, IVR can enable role scripts and collaboration mechanisms that are more engaging, creative, and complex through virtual avatars and fantasy worldviews, leading to enhanced social experiences.

5.2. Task design

According to the qualitative findings of this study, the collaboration task of the IVR game The League of Castle Defenders was well received by the students owing to three design features. First, the collaboration task allowed the students to freely explore an authentic virtual world characterized by an authentic environment, action, and narratives [68]. IVR allowed students to experience imaginary environments as active explorers, rather than as passive observers. According to Thompson et al. [50], such affordance of IVR is ideal for the construction of inquiry-based collaboration tasks. Second, the collaboration task engaged the students to actively participate in the process of discussion, decision-making, and problem-solving to create positive interdependence. As Rodrı́guez Illera [41] concluded, tasks that create genuine interdependence have three features: “(1) to share the resources; (2) the division of work between the members of the team; and (3) to share the cognition through a joint activity” (p. 485). In this study, the castle defense task was divided into collecting resources, attacking enemies, and defending against artillery fire such that the dependence between the subtasks and the main task was clearly and intuitively established. Third, the collaboration task allowed for real-time mapping of the virtual and physical space, which made the students’ cognition and decisions in IVR more intuitive and reflective of their skills and preferences in the physical world. The researchers favor such a design as it maximizes individual strengths and can potentially enhance students’ independent thinking, communication, and problem-solving skills [69].

5.3. Feedback and communication design

A feedback design for The League of Castle Defenders game in this study was generated through two types of interaction between the learner and the IVR environment: visual interaction and action interaction. The interview results revealed that the natural feedback of the VR game scenes (e.g., smoke dispersal) and the same spatial and temporal rules as the physical world generated a high level of spatial presence and immersion. The game was designed to provide a high degree of interaction between students and the virtual objects included in the IVR environments, as well as ongoing feedback to the players, which has been linked to deeper learning and assists in the learning process [70]. Additionally, the communication mechanism in this study was designed based on the mapping of virtual and real space, which allowed students to synchronously communicate and perceive the presence of others more intensely. We believe that this mechanism can generate greater social presence and thus lead to enhanced social experience and collective efficacy, as indicated by the regression model. Such a finding supports previous research findings that link social presence with enhanced collaboration experience and performance in online learning contexts [42].

However, there is still room for improvement in the feedback and communication design. In our study, the students regretted not being able to keep track of their teammates’ status in the virtual game because the physical game field was extremely large, which was one of the key factors affecting the sense of co-presence. As Bulu [71] argued, co-presence consists of two dimensions: “having a sense of feeling of other individuals and having a sense of feeling that others were actively perceiving us and being part of a group” (p. 155). This collaborative space lacked dynamic perception among peers owing to poor communication. Thus, future research can consider elaborating on the internal communication channel to facilitate verbal interaction.

5.4. Pre-training design

Despite students’ general positive ratings of their collaboration experience in IVR, few groups completed the task in this study. We believe that the lack of individual collaboration competency for certain students is a possible reason. The regression results suggest that one’s social experience in IVR collaboration can be negatively affected by low collaboration competency, which, in turn, can lead to decreased cognitive engagement and performance [32]. Therefore, it is necessary to conduct pre-training to develop students’ collaboration skills prior to their IVR experience for enhanced social experience and group performance. The importance of adding a pre-training session in IVR interventions is also supported by previous studies, reporting benefits such as enhanced self-efficacy [72] and reduced cognitive load [73].

5.5. Limitations and future research

This study has several limitations. First, the IVR collaboration space developed in this study lacks stability owing to technical glitches such as the loss of the player’s position in virtual space, which had negative effects on the collaboration experience. Future research needs to improve the stability of the IVR space to boost the relevance of study results on collaboration. Second, because the empirical investigation was conducted only once in this study, students’ experiences in the IVR collaboration space may have been influenced by the novelty effect. Future researchers are recommended to conduct longitudinal studies to enhance the credibility of the empirical findings. Third, this study measured collaboration experience by collecting data only through a self-reported questionnaire, which tends to be subjective and less accurate. Subsequent studies could use more diverse data-collection instruments, such as observation protocols, location heatmaps, or the Kinect sensor to collect multimodal data for a more accurate and comprehensive analysis of students’ collaboration behaviors and experiences. Last, note that the overall prediction capacity of the regression models was low, suggesting the absence of key influencing factors. Future studies should extend the scope of the investigation to include a more comprehensive list of predictors for collaboration experience in IVR.

6. Conclusions

This study presented a design case of a synchronous co-located IVR space that supports multiuser real-time collaboration. We explicated and empirically validated four design considerations for IVR collaboration spaces by documenting the design, development, and evaluation of a collaborative IVR game. Moreover, the study results revealed that the students gained an immersive social experience with enhanced collective efficacy and that social presence and collaboration competence positively predicted social experience and collective efficacy. We further identified spatial presence and natural feedback as two important factors that affect the immersion of IVR space through qualitative data analysis, finding that co-presence was essential for enabling social presence in IVR collaboration activity. Additionally, synchronous co-located virtual collaboration was found to place high demands on certain individual competencies, such as motor ability.

We believe this study makes several contributions to the existing literature. The first contribution is that we innovatively created a synchronous co-located IVR space using a SLAM-based inside-out tracking technology that support multi-player presence, and thus the research findings extend our contextual understanding of computer mediated collaboration. The second contribution is that we demonstrated a complete design case of an IVR collaboration space, which serves as an exemplar showing how different design considerations can be integrated to enable group collaboration in the IVR environment. The third contribution is that we utilized both quantitative and qualitative data to empirically explore the effects of the IVR design on promoting positive collaboration experiences as well as the potential influencing factors. The empirical findings provide valuable implications for designing and implementing synchronous co-located collaboration in IVR.

Author Contributions

Conceptualization, H.L.; methodology, Q.F.; software, Z.L.; validation, Q. F., G. L. and H. L.; formal analysis, Q. F., J. L., Y. Y. and Z. L.; investigation, Q. F.; resources, Z. L.; data curation, G. L. and Y. Y.; writing—original draft preparation, Q. F. and J. L.; writing—review and editing, H. L.; visualization, J. L.; supervision, H. L. and Z.L.; project administration, H. L.; funding acquisition, H. L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Nature Science Foundations of China, grant number 62177021.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are openly available in Mendeley Data at https://www.doi.org/10.17632/78py4t2bc5.2.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A: IVR collaboration questionnaire

Part 1: Basic information

Personality

I enjoy meeting new people

I can always find something to talk about when I talk to people.

I can ask for what I want without worrying too much.

I have many hobbies and interests.

I know a lot of interesting places

Collaboration competency

Communication skills

I am able to express myself in clear and precise terms.

I can use appropriate non-verbal behaviors (body language, facial expressions) to express my ideas.

I pay attention to the intervals of my speech when expressing my ideas.

I always give other members the opportunity to express their ideas.

I am responsible for teamwork.

Teamwork ability

I often actively express my opinion in group discussions.

I listen carefully and give my advice when others present their views.

I can empathize with the feelings of others.

I can harmonize myself and my team members well in a team.

Leadership

I am usually confident.

I am more concerned about the responsibilities of all members.

I share the same goals as other team members.

I get along well with my team members.

I can work with team members toward a common goal.

I could accept others’ ideas.

Collaboration experience

I have regularly participated in group tasks.

I actively participated in group discussions in group tasks.

I was able to listen to others' ideas in group tasks.

I feel that I have performed satisfactorily in group work in the past.

VR readiness

I used to watch VR movies.

I used to play VR games.

I used to participate in VR multiplayer games.

I am familiar with what equipment is needed for VR.

I am familiar with the areas in which VR is used.

I play 3D games a lot.

I play 3D games very well.

Part 2: IVR collaboration experience

Collective Efficacy

My group could pull itself out of difficult situations.

My group members worked harder than expected.

My group members worked hard to complete the group’s task.

My group was effective in finishing the task.

My group did a good job in getting things done.

My group was effective in meeting the task requirements.

My group accomplished its goals successfully.

My group completed its task successfully.

Social experience

I felt I was not alone.

I felt the group members supported me.

I felt I had someone to work with in my group.

I felt like a member of my group.

I felt connected to others in my group.

I felt I could discuss with members of my group.

Part 3: IVR presence

Social presence

I felt the presence of virtual objects in the game scene.

I felt that virtual things were real.

I often forgot that it was a computer-generated virtual space while playing the game.

I was aware of the presence of my teammates while playing the game.

I was able to easily communicate with my teammates while playing the game.

Spatial presence

At the start of the game, I felt as if I were entering a new world.

While playing the game, I felt as if I were part of a game that was not part of reality.

All of my senses were stimulated by the (game) experience.

Appendix B: Qualitative Coding Results

Appendix B. can be accessed at https://www.doi.org/10.17632/78py4t2bc5.2.

References

Ferguson, C.; van den Broek, E.L.; van Oostendorp, H. On the role of interaction mode and story structure in virtual reality serious games. Comput. Educ. 2020, 143, 103671. [Google Scholar] [CrossRef]
Luo, H.; Li, G.; Feng, Q.; Yang, Y.; Zuo, M. Virtual reality in K-12 and higher education: A systematic review of the literature from 2000 to 2019. J. Comput. Assist. Learn. 2021, 37, 887–901. [Google Scholar] [CrossRef]
Ke, F.; Xu, X. Virtual reality simulation-based learning of teaching with alternative perspectives taking. Br. J. Educ. Technol. 2020, 51, 2544–2557. [Google Scholar] [CrossRef]
Ristor, R.; Morélot, S.; Garrigou, A.; N’ Kaoua, B. Virtual reality for fire safety training: study of factors involved in immersive learning. Virtual Real. 2023, 27, 2237–2254. [Google Scholar] [CrossRef]
Slater, M.; Wilbur, S. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence (Camb.) 1997, 6, 603–616. [Google Scholar] [CrossRef]
Lu, A.S.; Pelarski, V.; Alon, D.; Baran, A.; McGarrity, E.; Swaminathan, N.; Sousa, C.V. The effect of narrative element incorporation on physical activity and game experience in active and sedentary virtual reality games. Virtual Real. 2023, 27, 1607–1622. [Google Scholar] [CrossRef] [PubMed]
Tursø-Finnich, T.; Jensen, R.O.; Jensen, L.X.; Konge, L.; Thinggaard, E. Virtual reality head-mounted displays in medical education. Simul. Healthc. 2023, 18, 42–50. [Google Scholar] [CrossRef] [PubMed]
Wu, T.-H.; Wu, F.; Liang, C.-J.; Li, Y.-F.; Tseng, C.-M.; Kang, S.-C. A virtual reality tool for training in global engineering collaboration. Univers. Access Inf. Soc. 2019, 18, 243–255. [Google Scholar] [CrossRef]
Laukkanen, T.; Xi, N.; Hallikainen, H.; Ruusunen, N.; Hamari, J. Virtual technologies in supporting sustainable consumption: From a single-sensory stimulus to a multi-sensory experience. Int. J. Inf. Manage. 2022, 63, 102455. [Google Scholar] [CrossRef]
Shi, A.; Wang, Y.; Ding, N. The effect of game–based immersive virtual reality learning environment on learning outcomes: designing an intrinsic integrated educational game for pre–class learning. Interact. Learn. Environ. 2022, 30, 721–734. [Google Scholar] [CrossRef]
Drey, T.; Albus, P.; der Kinderen, S.; Milo, M.; Segschneider, T.; Chanzab, L.; Rietzler, M.; Seufert, T.; Rukzio, E. Towards collaborative learning in virtual reality: A comparison of co-located symmetric and asymmetric pair-learning. In CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022–5 May 2022. [CrossRef]
Morrison-Smith, S.; Ruiz, J. Challenges and barriers in virtual teams: a literature review. SN Appl. Sci. 2020, 2, 1096. [Google Scholar] [CrossRef]
Hube, N.; Müller, M.; Lapczyna, E.; Wojdziak, J. Mixed reality based collaboration for design processes. i-com 2020, 19, 123–137. [Google Scholar] [CrossRef]
Billinghurst, M.; Cordeil, M.; Bezerianos, A.; Margolis, T. Collaborative immersive analytics. In Immersive Analytics; Marriott, K., Schreiber, F., Dwyer, T., Klein, K., Riche, N.H., Itoh, T., et al., Eds.; Springer: Cham, Switzerland, 2018; pp. 221–257. [Google Scholar] [CrossRef]
Lehmann-Willenbrock, N.; Hung, H.; Keyton, J. New frontiers in analyzing dynamic group interactions: Bridging social and computer science. Small Group Res. 2017, 48, 519–531. [Google Scholar] [CrossRef] [PubMed]
Chow, K.; Coyiuto, C.; Nguyen, C.; Yoon, D. Challenges and design considerations for multimodal asynchronous collaboration in VR. Proc. ACM Hum. Comput. Interact. 2019, 3, 1–24. [Google Scholar] [CrossRef]
Saunders, C.S.; Ahuja, M.K. Are all distributed teams the same? Differentiating between temporary and ongoing distributed teams. Small Group Res. 2006, 37, 662–700. [Google Scholar] [CrossRef]
Eiris, R.; Wen, J.; Gheisari, M. iVisit-Collaborate: Collaborative problem-solving in multiuser 360-degree panoramic site visits. Comput. Educ. 2022, 177, 104365. [Google Scholar] [CrossRef]
Eriksson, T. Failure and success in using mozilla hubs for online teaching in a movie production course. In 2021 7th International Conference of the Immersive Learning Research Network (iLRN), Eureka, CA, USA, 17 May 2021–10 June 2021. [CrossRef]
Papamitsiou, Z.; Pappas, I.O.; Sharma, K.; Giannakos, M.N. Utilizing multimodal data through fsQCA to explain engagement in adaptive learning. IEEE Trans. Learn. Technol. 2020, 13, 689–703. [Google Scholar] [CrossRef]
Liang, H.-N.; Lu, F.; Shi, Y.; Nanjappan, V.; Papangelis, K. Evaluating the effects of collaboration and competition in navigation tasks and spatial knowledge acquisition within virtual reality environments. Future Gener. Comput. Syst. 2019, 95, 855–866. [Google Scholar] [CrossRef]
Makransky, G. The Immersion Principle in Multimedia Learning. In R. Mayer & L. Fiorella (Eds.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 296-303). Cambridge: Cambridge University Press. 2021. [CrossRef]
Makransky, G.; Borre-Gude, S.; Mayer, R.E. Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments. J. Comput. Assist. Learn. 2019, 35, 691–707. [Google Scholar] [CrossRef]
Jiang, X.; Zhu, L.; Liu, J.; Song, A. A SLAM-based 6DoF controller with smooth auto-calibration for virtual reality. Visual Comput. 2023, 39, 3873–3886. [Google Scholar] [CrossRef]
Maqtary, N.; Mohsen, A.; Bechkoum, K. Group formation techniques in computer-supported collaborative learning: A systematic literature review. Technology, Knowledge and Learning 2019, 24, 169–190. [Google Scholar] [CrossRef]
Han, J.; Kim, K.H.; Rhee, W.; Cho, Y.H. Learning analytics dashboards for adaptive support in face-to-face collaborative argumentation. Comput. Educ. 2021, 163, 104041. [Google Scholar] [CrossRef]
Shah, N.; Lewis, C.M. Amplifying and attenuating inequity in collaborative learning: Toward an analytical framework. Cogn. Instr. 2019, 37, 423–452. [Google Scholar] [CrossRef]
Boling, E. The need for design cases: Disseminating design knowledge. Int. J. Des. Learn. 2010, 1. [Google Scholar] [CrossRef]
Goncalo, J.A.; Polman, E.; Maslach, C. Can confidence come too soon? Collective efficacy, conflict and group performance over time. Organ. Behav. Hum. Decis. Process. 2010, 113, 13–24. [Google Scholar] [CrossRef]
Stajkovic, A.D.; Lee, D.; Nyberg, A.J. Collective efficacy, group potency, and group performance: Meta-analyses of their relationships, and test of a mediation model. J. Appl. Psychol. 2009, 94, 814–828. [Google Scholar] [CrossRef] [PubMed]
Elms, A.K.; Gill, H.; Gonzalez-Morales, M.G. Confidence is key: Collective efficacy, team processes, and team effectiveness. Small Group Res. 2023, 54, 191–218. [Google Scholar] [CrossRef]
Huang, X.; Lajoie, S.P. Social emotional interaction in collaborative learning: Why it matters and how can we measure it? Soc. Sci. Humanit. Open 2023, 7, 100447. [Google Scholar] [CrossRef]
Bandura, A.; Freeman, W.H.; Lightsey, R. Self-efficacy: The exercise of control. J. Cogn. Psychother. 1999, 13, 158–166. [Google Scholar] [CrossRef]
Wang, S.-L.; Lin, S.S.J. The effects of group composition of self-efficacy and collective efficacy on computer-supported collaborative learning. Comput. Human Behav. 2007, 23, 2256–2268. [Google Scholar] [CrossRef]
Wang, S.-L.; Hwang, G.-H.; Chu, J.-C.; Tsai, P.-S. The role of collective efficacy and collaborative learning behavior in learning computer science through CSCL. In 14th annual ACM SIGCSE conference on Innovation and technology in computer science education, Paris, France, July 2009. [CrossRef]
Hong, E.; Jeong, Y. Coach leadership style and Korean professional soccer team performance: Collective efficacy as a mediator. Soc. Behav. Pers. Int. J. 2020, 48, 1–14. [Google Scholar] [CrossRef]
Vidergor, H.E. Effects of digital escape room on gameful experience, collaboration, and motivation of elementary school students. Comput. Educ. 2021, 166, 104156. [Google Scholar] [CrossRef]
Forrester, W.R.; Tashchian, A. Effects of personality on attitudes toward academic group work. Am. J. Bus. Educ. (AJBE) 2010, 3, 39–46. [Google Scholar] [CrossRef]
Türel, Y.K. Relationships between students' perceived team learning experiences, team performances, and social abilities in a blended course setting. Internet High. Educ. 2016, 31, 79–86. [Google Scholar] [CrossRef]
Laffey, J.; Lin, G.Y.; Lin, Y. Assessing social ability in online learning environments. J. Interact. Learn. Res. 2006, 17, 163–177. [Google Scholar]
Rodrı́guez Illera, J.L. Collaborative environments and task design in the university. Comput. Human Behav. 2001, 17, 481–493. [Google Scholar] [CrossRef]
Minor, M.; Swanson, A. Instructor social presence within the community of inquiry framework and its impact on classroom community and the learning environment. Online J. Distance Learn. Adm. 2014, 17. [Google Scholar]
Moser, I.; Chiquet, S.; Strahm, S.K.; Mast, F.W.; Bergamin, P. Group decision-making in multi-user immersive virtual reality. Cyberpsychol. Behav. Soc. Netw. 2020, 23, 846–853. [Google Scholar] [CrossRef] [PubMed]
Dalgarno, B.; Lee, M.J.W. What are the learning affordances of 3-XD virtual environments? Br. J. Educ. Technol. 2010, 41, 10–32. [Google Scholar] [CrossRef]
Chheang, V.; Fischer, V.; Buggenhagen, H.; Huber, T.; Huettl, F.; Kneist, W.; Preim, B.; Saalfeld, P.; Hansen, C. Toward interprofessional team training for surgeons and anesthesiologists using virtual reality. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 2109–2118. [Google Scholar] [CrossRef]
Zhang, L.; Weitlauf, A.S.; Amat, A.Z.; Swanson, A.; Warren, Z.E.; Sarkar, N. Assessing social communication and collaboration in autism spectrum disorder using intelligent collaborative virtual environments. J. Autism Dev. Disord. 2020, 50, 199–211. [Google Scholar] [CrossRef]
Agnès, A.; Sylvain, F.; Vanukuru, R.; Richir, S. Studying the effect of symmetry in team structures on collaborative tasks in virtual reality. Behav. Inf. Technol. 2022, 42, 2467–2475. [Google Scholar] [CrossRef]
Stavroulia, K.E.; Lanitis, A. The role of perspective-taking on empowering the empathetic behavior of educators in VR-based training sessions: An experimental evaluation. Comput. Educ. 2023, 197, 104739. [Google Scholar] [CrossRef]
De Back, T.T.; Tinga, A.M.; Louwerse, M.M. Learning in immersed collaborative virtual environments: design and implementation. Interact. Learn. Environ. 2021, 31, 5364–5382. [Google Scholar] [CrossRef]
Thompson, M.M.; Wang, A.; Roy, D.; Klopfer, E. Authenticity, interactivity, and collaboration in VR learning games. Front. Robot. AI 2018, 5, 133. [Google Scholar] [CrossRef]
Xia, H.; Herscher, S.; Perlin, K.; Wigdor, D. Spacetime. In 31st Annual ACM Symposium on User Interface Software and Technology, Berlin, Germany, October 2018. [CrossRef]
Ghoshal, M.; Ong, J.; Won, H.; Koutsonikolas, D.; Yildirim, C. Co-located immersive gaming: A comparison between augmented and virtual reality. In 2022 IEEE Conference on Games (CoG), Beijing, China, 21–24 August 2022. [CrossRef]
Won, M.; Ungu, D.A.K.; Matovu, H.; Treagust, D.F.; Tsai, C.-C.; Park, J.; Mocerino, M.; Tasker, R. Diverse approaches to learning with immersive Virtual Reality identified from a systematic review. Comput. Educ. 2023, 195, 104701. [Google Scholar] [CrossRef]
Dede, C.J.; Jacobson, J.; Richards, J. Introduction: Virtual, augmented, and mixed realities in education. In Virtual, Augmented, and Mixed Realities in Education; Liu, D., Dede, C., Huang, R., Richards, J., Eds.; Springer: Singapore, 2017; pp. 1–16. [Google Scholar] [CrossRef]
Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef] [PubMed]
Radziej, K.; Loechner, J.; Engerer, C.; Niglio de Figueiredo, M.; Freund, J.; Sattel, H.; Bachmann, C.; Berberat, P.O.; Dinkel, A.; Wuensch, A. How to assess communication skills? Development of the rating scale ComOn Check. Med. Educ. Online 2017, 22, 1392823. [Google Scholar] [CrossRef] [PubMed]
Duong, M.-Q. Perceptions of faculty members on the leadership capacity of their students at Vietnam national university Ho Chi Minh City. Int. J. Educ. Pract. 2020, 8, 356–364. [Google Scholar] [CrossRef]
von der Pütten, A.M.; Klatt, J.; Ten Broeke, S.; McCall, R.; Krämer, N.C.; Wetzel, R.; Blum, L.; Oppermann, L.; Klatt, J. Subjective and behavioral presence measurement and interactivity in the collaborative augmented reality game TimeWarp. Interact. Comput. 2012, 24, 317–325. [Google Scholar] [CrossRef]
Bland, J.M.; Altman, D.G. Cronbach's alpha. BMJ 1997, 314, 572. [Google Scholar] [CrossRef] [PubMed]
Campbell, D.T.; Fiske, D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 1959, 56, 81–105. [Google Scholar] [CrossRef] [PubMed]
Saldana, J. The coding manual for qualitative researchers; Sage: Los Angeles (CA), 2013.
Bazeley, P. Qualitative data analysis: Practical strategies. Qual. Data Anal. 2020, 1–584. [Google Scholar]
O’brien, R.M. A caution regarding rules of thumb for variance inflation factors. Qual. Quant. 2007, 41, 673–690. [Google Scholar] [CrossRef]
Strojny, P.M.; Dużmańska-Misiarczyk, N.; Lipp, N.; Strojny, A. Moderators of social facilitation effect in virtual reality: Co-presence and realism of virtual agents. Front. Psychol. 2020, 11, 1252. [Google Scholar] [CrossRef] [PubMed]
Lui, M.; Slotta, J.D. Immersive simulations for smart classrooms: exploring evolutionary concepts in secondary science. Technol. Pedagog. Educ. 2014, 23, 57–80. [Google Scholar] [CrossRef]
Strijbos, J.-W.; Weinberger, A. Emerging and scripted roles in computer-supported collaborative learning. Comput. Human Behav. 2010, 26, 491–494. [Google Scholar] [CrossRef]
Laal, M. Positive interdependence in collaborative learning. Procedia Soc. Behav. Sci. 2013, 93, 1433–1437. [Google Scholar] [CrossRef]
Wang, A.; Thompson, M.; Uz-Bilgin, C.; Klopfer, E. Authenticity, interactivity, and collaboration in virtual reality games: Best practices and lessons learned. Front. Virtual Real. 2021, 2. [Google Scholar] [CrossRef]
Care, E.; Scoular, C.; Griffin, P. Assessment of collaborative problem solving in education environments. Appl. Meas. Educ. 2016, 29, 250–264. [Google Scholar] [CrossRef]
Hong, S.W.; Jeong, Y.; Kalay, Y.E.; Jung, S.; Lee, J. Enablers and barriers of the multi-user virtual environment for exploratory creativity in architectural design collaboration. CoDesign 2016, 12, 151–170. [Google Scholar] [CrossRef]
Bulu, S.T. Place presence, social presence, co-presence, and satisfaction in virtual worlds. Comput. Educ. 2012, 58, 154–161. [Google Scholar] [CrossRef]
Meyer, O.A.; Omdahl, M.K.; Makransky, G. Investigating the effect of pre-training when learning through immersive virtual reality and video: A media and methods experiment. Comput. Educ. 2019, 140, 103603. [Google Scholar] [CrossRef]
Mayer, R.E.; Pilegard, C. Principles for managing essential processing in multimedia learning: Segmenting, pre-training, and modality principles. In: The Cambridge Handbook of Multimedia Learning; Mayer, R.E. Eds.; Cambridge University Press: Cambridge, England, 2014, pp. 316–344. [CrossRef]

Figure 1. Prototype illustration of the IVR game design.

Figure 2. Illustration of the three roles: the (a) defender, (b) attacker, and (c) collector.

Figure 3. Interaction design of the game: (a) game map; (b) defender’s shield raising action; (c) attacker picking up a crossbow; and (d) collector opening a supply crate.

Figure 4. Flowchart of the game algorithm.

Figure 5. Real-time mapping between virtual and physical world: (a) a collector observing their teammates during the game; (b) five students facing the enemy ship.

Table 1. Functions and missions of the three different roles.

Role type	Role functions	Role missions
Defender	• Block artillery fires • Initially 500 health points	Anticipate the flight path of the artillery and move quickly to block it.
Attacker	• Attack the enemy • Initially 50 health points • Initially own 10 arrows (an arrow = 100 damage)	Slay all the enemies by shooting arrows while avoiding the enemies’ artillery.
Collector	• Initially 50 health points • Open medical supply boxes that restore 100 health points to defenders and ammunition supply boxes that replenish two arrows to attackers	Find and open the supply box at the back of the group to provide the defender with blood or the attacker with arrows.

Table 2. Results of themes, categories, and nodes for VR experiences.

Themes	Categories (frequency count)	Nodes (frequency count)
Theme 1	Game experience (25)	Novelty and interesting (9)
		Challenging (8)
		Intense (3)
		Enjoy (3)
		Competing and exploring (2)
	Immersion (24)	Spatial presence (5)
		Exquisitely crafted visuals (5)
		Realistic special effects (5)
		Natural feedback (4)
		Authentic story (5)
Theme 2	Co-presence (14)	Observing teammates’ positions (2)
		Seeing avatars' actions (3)
		Interacting with teammates (6)
		Moving realistically (1)
		Face-to-face negotiation (2)
	Co-located (11)	Safety issues (6)
		Real and warm (2)
		Form strategies quickly (3)
	Pretraining (22)	Lack of understanding of game rules (9)
		Unfamiliarity with VR devices (1)
		Effective observation and learning (11)
		Prior experience (1)
Theme 4	Technological issues (7)	Unstable real-mapping technology (5)
		Unstable game functionality (2)
	Communication issues (14)	Channels of communication (5)
		Timeliness of communication (5)
		Flexibility in communication (4)
	Game issues (19)	Lack of dynamic information cues (10)
		Lack of level challenges (3)
		Lack of audio cues (2)
		Lack of key content prompts (3)
		Lack of personalized choices (1)

Table 3. Descriptive analysis of the collaboration experience in IVR.

Item statement^a	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Social presence (Mean = 3.84, SD = 0.64)
Presence of virtual objects in the game scene	1 (1.1%)	0	9 (9.5%)	55 (57.9%)	30 (31.6%)
Realistic feeling of virtual things	0	5 (5.3%)	27 (28.4%)	42 (44.2%)	21 (22.1%)
Forgot that it was computer-generated	0	13 (13.7%)	22 (23.2%)	36 (37.9%)	24 (25.3%)
Awareness of teammates presence	0	6 (6.3%)	10 (10.5%)	50 (52.6%)	29 (30.5%)
Easy communication with teammates	3 (3.2%)	21 (22.1%)	25 (26.3%)	31 (32.6%)	15 (15.8%)
Spatial presence (Mean = 4.21, SD = 0.66)
A feeling of entering a new world	0	2 (2.1%)	4 (4.2%)	48 (50.5%)	41 (43.2%)
A feeling of being part of the game	0	3 (3.2%)	13 (13.7%)	41 (43.2%)	38 (40.0%)
All my senses were stimulated in the game	0	4 (4.2%)	15 (15.8%)	43 (45.3%)	33 (34.7%)
Collective efficacy (Mean = 3.98, SD = 0.72)
My group could solve difficult situations.	0	2 (2.1%)	16 (16.8%)	52 (54.7%)	25 (26.3%)
Group members worked harder than expected.	0	4 (4.2%)	11 (11.6%)	50 (52.6%)	30 (31.6%)
Group members worked hard to complete task.	0	1 (1.1%)	5 (5.3%)	51 (53.7%)	38 (40.0%)
My group was effective in finishing the task.	1 (1.1%)	4 (4.2%)	26 (27.4%)	38 (40.0%)	26 (27.4%)
My group did a good job in getting things done.	0	3 (3.2%)	23 (24.2%)	36 (37.9%)	33 (34.7%)
My group effectively fulfilled task requirements.	0	3 (3.2%)	23 (24.2%)	40 (42.1%)	29 (30.5%)
My group accomplished its goals successfully.	0	9 (9.55)	28 (29.5%)	35 (36.8%)	23 (24.2%)
My group completed its task successfully.	2 (2.1%)	14 (14.7%)	22 (23.2%)	32 (33.7%)	25 (26.3%)
Social experience (Mean = 4.11, SD = 0.66)
I felt I was not alone.	1 (1.1%)	4 (4.2%)	13 (13.75%)	46 (48.4%)	31 (32.6%)
I felt the group members supported me.	0	2 (2.1%)	9 (9.5%)	57 (60.0%)	27 (28.4%)
I felt I had someone to work with in my group.	0	4 (4.2%)	6 (6.3%)	52 (54.7%)	33 (34.7%)
I felt like a member of my group.	0	1 (1.1%)	6 (6.3%)	50 (52.6%)	38 (40.0%)
I felt connected to others in my group.	0	4 (4.2%)	12 (12.6%)	52 (54.7%)	27 (28.4%)
I felt I could discuss with members of my group.	2 (2.1%)	13 (13.7%)	11 (11.6%)	40 (42.1%)	29 (30.5%)

^a. Certain item statements are condensed to fit in the table column. See Appendix A for complete statements.

Table 4. Correlation analysis between predictors (Pearson’s correlation coefficient).

	EP	RD	CC	SpaP	SocP	CE	SE
Extroverted personality (EP)	1
VR readiness (RD)	0.476^**	1
Collaboration competence (CC)	0.612^**	0.373^**	1
Spatial presence (SpaP)	0.226^*	0.153	0.375^**	1
Social presence (SocP)	0.277^**	0.154	0.302^**	0.706^**	1
Collective efficacy (CE)	0.303^**	0.174	0.297^**	0.459^**	0.521^**	1
Social experience (SE)	0.275^**	0.119	0.365^**	0.577^**	0.679^**	0.627^**	1

Note: ** p < 0.01 and *p < 0.05.

Table 5. Key statistics of the backward-elimination multiple linear regression analysis (N = 95).

Dependent variable	Predictors	B	SE	β	Collinearity		Adjusted R²
Dependent variable	Predictors	B	SE	β	Tolerance	VIF	Adjusted R²
Collective- efficacy	Extraversion	0.197	0.104	0.172	0.923	1.083	0.283
Collective- efficacy	Social presence	0.529	0.102	0.473***	0.923	1.083
Social experience	Collaboration competence	0.214	0.095	0.176*	0.909	1.100	0.479
Social experience	Social presence	0.642	0.080	0.626***	0.909	1.100

Note: SE is an abbreviation of Standard Error; VIF is an abbreviation of Variance inflation factor; *** p < 0.001 and *p < 0.05.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer