4. Research Methodology
4.1. Research Aim
In this study, we leverage Kolb’s experiential learning framework to assess the instructional effectiveness of VR software applications within the field of underwater engineering. Specifically, we employ Kolb’s model to examine how diverse learning styles interact with immersive environments, influencing knowledge acquisition and skill development. Drawing on the framework’s evolution, including its refined nine-style granularity (Experiencing, Imagining, Reflecting, Analysing, Thinking, Deciding, Acting, Initiating, and Balancing), our investigation evaluates these styles in relation to flow states, satisfaction in VR use and learning outcomes among engineering students.
The Submarine Simulator VR software was custom-developed for this study to provide a dedicated pedagogical platform that integrates three key elements: a realistic underwater physics engine for authentic hydrodynamic feedback, an in-VR 3D modeling toolkit that allows for direct hands-on design and iteration, and a gamified competitive framework featuring head-to-head racing challenges.
4.2. Hypothesis
We tested the following hypothesis:
H1. The perceived quality of the VR learning environment, in conjunction with students’ learning styles, significantly predict the level of immersive engagement and the overall satisfaction of the learning experience.
H2. Learning styles significantly influence students’ performance in the VR-based Submarine Simulator.
H3. Immersive engagement and the flow state in VR mediates the relationship between learning styles and learning outcomes.
H4. Learning styles significantly influence students perceived immersion and flow state in the VR-based Submarine Simulator.
H5. Students’ performance in VR conditions are influenced by the perceived quality of VR software.
4.3. Target Group
The research protocol was tested on a group of 26 students, in their 4th year at MINES Paris - PSL, during the underwater engineering course. The course was conducted on a full-time basis over a 10-week on-campus period, during which students were not enrolled in any other courses. Each student completed all three phases described in the research protocol (subchapter 4.5). Following each phase, the facilitator conducted a brief 10-minute interview to ensure the experience was well-received, confirming that students enjoyed the session and felt comfortable, without experiencing nausea or discomfort.
All students signed an agreement to participate in research protocol and were informed that during the research and their dissemination the anonymity of the participants will be respected. For the reference purposes, during the phase of data processing stage, only randomly assigned numerical codes have been used. The characteristics of the group, from the point of view of the learning styles, flow, immersion, satisfaction and performance. are presented in Appendix 2.
4.4. Methods
To highlight the research variables, the following instruments and tests were used:
- 1.
-
The Kolb Experiential Learning Profile (KELP) is a practical self-assessment instrument that can help us assess our unique learning styles and has the advantage of only taking 15-25 minutes to complete [
32]. Based on the results of the test, students have received a scored on the following four quadrants, describing how they process and transform experiences into knowledge:
- (1)
Concrete Experience (CE) - Learners immerse themselves fully, openly, and without preconceived biases in novel experiences, emphasizing direct involvement and sensory engagement.
- (2)
Reflective Observation (RO) - They contemplate and examine these experiences from diverse viewpoints, fostering introspection and nuanced understanding.
- (3)
Abstract Conceptualisation (AC) - They synthesize observations into coherent concepts, forming logically robust theories that explain patterns and relationships.
- (4)
Active Experimentation (AE) - They apply these theories practically to inform decision-making and address real-world problems, testing ideas through action.
Drawing from these quadrants, Kolb’s refined framework identifies nine distinct learning styles, each offering greater granularity and reflecting unique preferences in how individuals navigate the learning cycle: experiencing; imagining; reflecting; analyzing; thinking; deciding; acting; initiating; balancing (Appendix 1). The analysis of the questionnaire involved characterizing the group in terms of learning dimensions, learning styles, and flex learning styles.
Learning styles are determined by an individual’s preferences along two key bipolar dimensions, which are calculated as difference scores from questionnaire responses. These dimensions capture how people balance opposing approaches to perceiving (grasping information) and processing (transforming information): ACCE (Abstract Conceptualization minus Concrete Experience) and AERO (Active Experimentation minus Reflective Observation) (Appendix 1).
In Kolb’s experiential learning theory, a student’s primary learning style reflects their preferred approach to the learning cycle. This primary style is a preference, not a rigid limitation, indicating where a learner naturally feels most comfortable. Alongside this, students exhibit flex styles where they also perform effectively, showcasing their adaptability. Non-flex styles, or developing styles, are areas where learners are less proficient but can improve through practice and self-awareness, leading to a more complete and balanced learning profile.
- 2.
-
The Flow State Scale (FSS), developed by Jackson and Marsh (1996), is a 36-item psychometric tool designed to measure the nine dimensions of flow as outlined by Csikszentmihalyi, capturing the optimal psychological state of complete immersion and engagement in an activity [
33]. These dimensions include:
Challenge-Skill Balance: Perceiving that personal skills match the task’s demands.
Action-Awareness Merging: Experiencing seamless integration of actions and awareness.
Clear Goals: Having a clear understanding of objectives.
Unambiguous Feedback: Receiving immediate, clear feedback on performance.
Concentration on the Task: Maintaining deep focus without distractions.
Sense of Control: Feeling in command of the activity.
Loss of Self-Consciousness: Becoming less aware of self and external judgments.
Transformation of Time: Perceiving time as altered, either speeding up or slowing down.
Autotelic Experience: Finding the activity intrinsically rewarding.
Each dimension is assessed through four items, rated on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree), enabling precise measurement of flow intensity. The FSS’s use in this study was approved by Mind Garden, Inc., on March 6, 2024, ensuring ethical compliance.
For each dimension, the four item scores are summed and divided by four to calculate an average score. This method quantifies flow experiences, facilitating analysis of their relationship with learning styles and performance outcomes in immersive VR settings like the Submarine Simulator, where engagement is key to learning.
- 3.
Immersive Tendencies Questionnaire (ITQ) was developed by Bob G. Witmer and Michael J. Singer (1998) and it was designed to assess an individual’s inherent propensity or tendency to become immersed in everyday activities, media, and environmental situations, particularly as a predictor of how readily they might experience presence in virtual environments (VEs) [
34]. Presence, in this context, refers to the psychological state of feeling "there" in a mediated or simulated environment, and the ITQ aims to capture individual differences that could influence immersion levels. It is typically administered prior to VE exposure to stratify participants, predict performance, or identify factors contributing to immersion, such as involvement in activities like reading, watching movies, or playing games. The questionnaire is supported by correlations with related measures like the Tellegen Absorption Scale (r ≈ .40-.60). Factor analysis in follow-up research confirms loadings on immersion-related constructs. Higher ITQ scores have been linked to better VE performance and higher presence ratings in some studies. Each item is rated on a 7-point Likert-type, higher scores indicate stronger immersive tendencies.
- 4.
The Basic Needs in Games (BANG) scale is an open-access, free-to-use questionnaire developed to evaluate the satisfaction and frustration of basic psychological needs—autonomy, competence, and relatedness—experienced by players during video game play, rooted in Self-Determination Theory (SDT) [
35]. The BANG scale includes six subscales—three assessing satisfaction (autonomy, competence, relatedness) and three measuring frustration—allowing researchers to compute mean scores for each need separately, offering detailed and interpretable insights into how well a game supports these psychological needs.
The "Satisfaction" dimension of the BANG results measures the fulfilment of three core needs—autonomy, competence, and relatedness—which significantly enhance player enjoyment, engagement, and psychological well-being. Autonomy satisfaction arises from players’ control over meaningful choices, competence satisfaction from mastering challenges and achieving goals, and relatedness satisfaction from forming social connections with others, including players and non-player characters. Fulfilling these needs fosters motivation, immersion, and sustained engagement, contributing to a rewarding gaming experience.
The scale has been statistically validated through rigorous psychometric testing, with initial studies demonstrating good internal consistency (Cronbach’s alpha values typically exceeding 0.80 for the satisfaction subscales) and construct validity, confirmed through factor analysis that aligns with SDT constructs. Furthermore, its reliability and applicability have been supported across diverse gaming contexts and populations, with ongoing research continuing to refine its sensitivity and predictive power.
This validation ensures that the BANG scale provides a robust tool for researchers and designers to assess and enhance the psychological impact of games, making the "Satisfaction" dimension a critical metric for optimizing player-centered design in virtual environments.
- 5.
-
Scoring on the Performance - to examine the interplay between learning styles, immersion and flow states, and learning outcomes in the Submarine Simulator VR environment, a tailored scoring system was established. This system draws on the distinct tasks across the three phases of software interaction, assigning scores that capture objective performance in each phase:
Phase 1 (Basic Submarine Construction and Testing, scored on a 0–20 Scale): This structured phase emphasizes foundational building and initial testing.
Phase 2 (Designing Tight and Loose Spiral Models, scored on a 0–2 Scale): Focused on strategic planning and iterative refinement, the binary scoring reflects a pass/fail mechanism for the two required models: 2 points for successfully completing both, 1 point for one, and 0 points for none.
Phase 3 (Competitive Racing on Tracks, scored on a 0–20 Scale): This dynamic phase demands real-time adaptation and quick decision-making. The scoring system was designed to be simple yet engaging. Points were awarded in accordance with finishing the race on each track, with 1 point for Track 1, 2 points for Track 2, and 3 points for the more complex Track 3. Additional points would be awarded after winning the race against an opponent.
Statistical methods. A Partial Least Squares Structural Equation Model (PLS-SEM) [
36] was employed to examine whether virtual reality (VR) experience—including Flow, Immersion, Satisfaction and Mindset—mediates the relationship between learning styles and learning performance across three instructional phases (Points1, Points2, Points3). As a valuable for understanding causal processes and testing mediation effects, SEM analysis combines aspects of factor analysis – learning styles components, cognitive, affective and mindset aspects, and path analysis to test and to assess the relationships among variables.
4.5. Research Protocol
To investigate the interplay between learning styles, flow state, and learning outcomes within the immersive VR environment of the
Submarine Simulator software application—a custom tool designed for underwater engineering education—we developed a structured research protocol. This protocol comprises sequential phases meticulously aligned with Kolb’s experiential learning cycle (Concrete Experience, Reflective Observation, Abstract Conceptualization, and Active Experimentation), ensuring a holistic approach that accommodates diverse learner preferences and promotes deeper engagement. (
Figure 1)
The VR simulation replicates authentic hydrodynamics and intricate underwater physics, posing challenges that are hard to envision or grasp without immersive tools. This cutting-edge and novel VR environment presents a unique challenge for participants, most of whom have little to no prior experience with VR technology or advanced underwater simulations. It demands a seamless blend of creative problem-solving and precise interaction, pushing the boundaries of realistic 3D modelling and navigation tasks. By mirroring Kolb’s framework, the protocol not only facilitates hands-on interaction but also encourages reflection, theoretical integration, and practical application, ultimately aiming to quantify how these elements influence knowledge acquisition, skill retention, and motivational flow in a simulated submarine modelling and testing scenario. The protocol unfolds in four primary phases, as detailed in
Table 1.
The learning outcomes are assessed through repeated VR sessions, enabling the evaluation of improvements in engineering competencies, such as problem-solving and technical precision. This dynamic process, rooted in Kolb’s learning cycle, fosters adaptive learning in the novel VR context, where participants can test hypotheses and refine skills in a realistic, immersive setting, ultimately strengthening their ability to tackle complex engineering challenges.
4.6. Research Design
This study adopted a multi-phase experimental design, carefully structured to align with Kolb’s experiential learning cycle, ensuring a scaffolded progression that fosters both practical engagement and theoretical understanding. Each phase was tailored to build upon the previous one, creating a cohesive framework that supports iterative learning, accommodates diverse learning styles, and captures nuanced data on participant performance and psychological states. This methodical design enhances the study’s ability to generate reliable, actionable insights into the efficacy of VR-based instruction in STEM education. All figures presented in this section are screenshots captured directly from the original Submarine Simulator VR software.
In the initial phase of the protocol, participants were gradually introduced to the virtual reality (VR) platform, with the main goal of helping them feel comfortable and confident in using it. This introductory session focused on familiarizing them with the system’s interactive features, including user-friendly navigation controls and tools for building models.
Through guided, hands-on tutorials led by a facilitator, learners actively engaged in practical exercises, learning to create submarine prototypes (
Figure 2) and conducting initial tests of navigation dynamics in the virtual underwater environment (
Figure 3).
The primary focus was on cultivating a core understanding of hydrodynamic principles, buoyancy, and propulsion mechanisms relevant to submarine design. This study does not evaluate performance metrics during this phase; instead, it functions as a preparatory stage to ensure all participants achieve a consistent baseline proficiency. Phase 1 allocated a strict 45-minute duration for each participant to complete, ensuring uniform timing across all sessions.
During phase 1, the following steps describe the instructions received (
Table 2).
Building upon the foundational skills developed in Phase 1, Phase 2 prioritizes independent design and iterative refinement within the Submarine Simulator VR environment. Participants commenced with a low-tech activity, sketching preliminary submarine designs on paper to promote conceptual planning and creative visualization unconstrained by digital tools. This approach cultivates deliberate ideation, enabling learners to freely explore concepts prior to immersing themselves in the VR platform. The underlying intention of this protocol was to examine whether VR tends to suppress creativity.
In the VR environment, the primary task during the phase involved designing submarines capable of navigating two distinct spiral trajectories in the simulated underwater setting (
Figure 4):
Tight spirals: Requiring high precision and small radius, challenging participants to optimize control and maneuverability.
Loose spirals: Emphasizing stability over larger radius, testing the model’s structural integrity under varying conditions.
Real-time feedback during testing provides data on trajectory accuracy, speed, and stability, enabling participants to refine their designs iteratively. This phase aimed to enhance problem-solving, adaptive design thinking, and practical application of hydrodynamic principles. Each participants had 35 minutes in VR during this phase.
Following Phase 2, a brief 10-minute interview was conducted to assess the students’ planning approaches for the submarine design task. Based on these interviews, participants were divided into three categories: (1) those who started with an initial plan but adapted it iteratively, based on intuition and observations (75%); (2) those who proceeded without a predefined plan, adapting dynamically along the way (20%); and (3) those who entered VR with a well-defined plan and followed it consistently throughout the phase (5%).
In the final phase of the study, a competitive twist was added by pairing participants into 13 teams, encouraging teamwork within each pair and friendly rivalry between groups. Each team collaborated to brainstorm ideas, then worked individually to design submarine models optimized for three increasingly challenging virtual underwater race tracks. These tracks were designed to reflect real-world engineering challenges, testing navigation, performance, and design skills in a dynamic, engaging and gamified way.
Track 1 - Straight-Line Navigation: Teams engineered submarine prototypes optimized for seamless straight-line propulsion, emphasizing robust propulsion mechanisms and consistent directional stability to prevent any unintended deviations from the intended path (
Figure 5).
Track 2 - Horizontal Maneuverability: This race track challenged teams to design submarine models capable of executing precise left and right maneuvers, integrating effective buoyancy adjustments, depth regulation, and thrust management to navigate the course successfully (
Figure 6).
Track 3 - Multi-Directional Agility: The most challenging track demanded complete maneuverability, requiring submarine models to execute fluid up, down, left, and right movements while maintaining precise control within a highly complex virtual underwater environment (
Figure 7).
In the concluding phase, participants put their submarine designs to the test through timed virtual simulations, where evaluations centered on key factors like whether the model fully completed the course.
Scoring was straightforward and motivating: for each track, points were allocated based on achievement—Track 1, which involved straightforward linear paths, offered 1 point if completed; Track 2 granted 2 points; and the more intricate Track 3 provided 3 points.
To heighten the excitement and competitive spirit, an additional layer was added: individuals could gain 1 bonus point by outperforming a rival in a direct race, with the opponent’s performance visualized as a "shadow" submarine (
Figure 8).
For clarity, this shadow wasn’t a live competitor, but a digital replay drawn from a prior participant’s successful run on the same track, enabling the current user to race alongside this ghostly echo in real-time within the immersive VR world, fostering a sense of head-to-head rivalry while building on collective progress.
If the current student finished the race ahead of this shadow opponent based on time, they would secure the additional point, adding a layer of competitive incentive to the exercise.
This phase emphasized interpersonal dynamics, such as competition and contentiousness, within a paired setting. Phase 3 spanned about 45 minutes, including team brainstorming, design collaboration, individual building, and racing. It is important to note that students could attempt the same track until successful or could change the track.
To conclude, the research design included three evaluation phases, each with its own set of instructions and a specific system for assessing the level of learning. (
Figure 9)