Submitted:
22 August 2024
Posted:
30 August 2024
You are already at the latest version
Abstract

Keywords:
1. Introduction
2. Materials and Methods
2.1. Study design
2.2. Creation and Validation of the Training Models
2.3. Ethical Considerations and Recruitment
2.4. Inclusion and Exclusion Criteria
2.5. Randomization and Participant Grouping
2.6. Measurement of Stress Response
2.7. Measurement of Self-Perceived Difficulty
2.8. Data Collection Phases
2.9. Statistical Analysis
3. Results
3.1. Analysis of Learning Outcomes
3.1.1. Normality Test
3.1.2. Wilcoxon Signed-Rank Test (Table 1)
3.2. Analysis of Stress Using Biometric Measurements
3.2.1. Normality Test
3.2.2. Friedman Test (Table 2 and Table 3)
- LF and HF: There were no significant differences in LF and HF values across the three scenarios (Chi-square = 2.643, df = 2, p = 0.267; and Chi-square = 2.951, df = 2, p = 0.229, respectively). LF/HF: there were no significant differences in LF/HF ratio and rMSSD values across the three conditions as well (Chi-square = 4.133, df = 2, p = 0.127; Chi-square = 3.211, df = 2, p = 0.201, respectively).
- PNN20: There were significant differences in pNN20 values across the three conditions (Chi-square = 7.393, df = 2, p = 0.025), with higher values observed in the basal state compared to VR and mannequin scenarios. Similar results were found with PNN50 (Chi-square = 9.435, df = 2, p = 0.009), with higher values observed in the basal condition compared to mannequin and VR.
- SD1/SD2: There were significant differences in SD1/SD2 ratio values across the three conditions (Chi-square = 14.157, df = 2, p = 0.001), with higher values in the mannequin group compared to basal and VR.
3.3. Self-Perceived Difficulty Analysis
3.3.1. Normality Test
3.3.2. Wilcoxon Signed-Rank (Table 4 and Table 5)
| Variable | N | Mean | SD | Min | Max | 25th Perc | Median | 75h Perc |
|---|---|---|---|---|---|---|---|---|
| MENTAL_DEM_MANI | 36 | 11.97 | 4.116 | 2 | 19 | 9.00 | 13.00 | 14.75 |
| PHISYCAL_DEM_MANI | 36 | 7.72 | 4.425 | 1 | 18 | 4.00 | 7.50 | 10.00 |
| TEMPO_DEM_MANI | 36 | 10.03 | 4.359 | 1 | 19 | 7.00 | 10.00 | 13.00 |
| PERFORMANCE_MANI | 36 | 8.67 | 4.414 | 1 | 18 | 5.00 | 9.00 | 11.75 |
| EFFORT_MANI | 36 | 11.61 | 3.774 | 4 | 18 | 8.25 | 12.00 | 14.00 |
| FRUSTRATION_MANI | 36 | 7.89 | 4.961 | 1 | 17 | 3.00 | 7.50 | 12.00 |
| MENTAL_DEM_VR | 62 | 10.13 | 4.611 | 1 | 20 | 5.75 | 10.50 | 14.00 |
| PHISYCAL_DEM_VR | 62 | 6.18 | 4.615 | 1 | 18 | 3.00 | 4.00 | 8.25 |
| TEMPO_DEM_VR | 62 | 9.82 | 4.091 | 1 | 18 | 7.00 | 10.00 | 12.25 |
| PERFORMANCE_VR | 62 | 7.50 | 4.218 | 3 | 17 | 4.00 | 6.00 | 11.00 |
| EFFORT_VR | 62 | 9.63 | 4.553 | 1 | 18 | 5.75 | 9.00 | 13.00 |
| FRUSTRATION_VR | 62 | 6.74 | 4.637 | 1 | 18 | 2.75 | 5.00 | 11.00 |
| Item | N Neg Rank | Mean Rank | Sum of Ranks | N Post Ranks | Mean Rank | Sum of Ranks | Ties | Z | p |
|---|---|---|---|---|---|---|---|---|---|
| MENTAL_DEM_MANI | 21 | 19.07 | 400.50 | 12 | 13.38 | 160.50 | 3 | -2.147 | 0.032 |
| PHISYCAL_DEM_MANI | 22 | 17.82 | 392.00 | 13 | 18.31 | 238.00 | 1 | -1.263 | 0.207 |
| TEMPO_DEM_MANI | 20 | 17.20 | 344.00 | 15 | 19.07 | 286.00 | 1 | -0.476 | 0.634 |
| PERFORMANCE_MANI | 17 | 19.97 | 339.50 | 16 | 13.84 | 221.50 | 3 | -1.057 | 0.290 |
| EFFORT_MANI | 23 | 18.91 | 435.00 | 11 | 14.55 | 160.00 | 2 | -2.356 | 0.018 |
| FRUSTRATION_MANI | 19 | 15.29 | 290.50 | 12 | 17.13 | 205.50 | 5 | -0.834 | 0.404 |
| MENTAL_DEM_VR | 21 | 19.07 | 400.50 | 12 | 13.38 | 160.50 | 3 | -2.147 | 0.032 |
| PHISYCAL_DEM_VR | 22 | 17.82 | 392.00 | 13 | 18.31 | 238.00 | 1 | -1.263 | 0.207 |
| TEMPO_DEM_VR | 20 | 17.20 | 344.00 | 15 | 19.07 | 286.00 | 1 | -0.476 | 0.634 |
| PERFORMANCE_VR | 17 | 19.97 | 339.50 | 16 | 13.84 | 221.50 | 3 | -1.057 | 0.290 |
| EFFORT_VR | 23 | 18.91 | 435.00 | 11 | 14.55 | 160.00 | 2 | -2.356 | 0.018 |
| FRUSTRATION_VR | 19 | 15.29 | 290.50 | 12 | 17.13 | 205.50 | 5 | -0.834 | 0.404 |
4. Discussion
4.1. Interpretation of Results
4.1.1. Learning Outcomes
4.1.2. Biometric Stress Analysis
4.1.3. Self-Perceived Difficulty Interpretation
4.2. Implications for Training
4.3. Limitations
4.4. Future Directions
5. Conclusions
Appendix A
Appendix B


References
- Barrett, H.; Bion, J. F. COBATRICE CORE COMPETENCIES AND SYLLABUS DEVELOPMENT. Critical care medicine 2005, 33, A14. [Google Scholar] [CrossRef]
- Sinnaeve, P. R.; Adriaenssens, T. A contemporary look at pericardiocentesis. Trends in cardiovascular medicine 2019, 29(7), 375–383. [Google Scholar] [CrossRef] [PubMed]
- Kalivoda, E. J.; Sullivan, A.; Bunting, L. A Cost-Effective, Rapidly Constructed Simulation Model for Ultrasound-Guided Pericardiocentesis Procedural Training. The Journal of emergency medicine 2019, 56(1), 74–79. [Google Scholar] [CrossRef] [PubMed]
- Kolb, D. Experiential Learning: Experience As The Source Of Learning And Development; 1984; Vol. 1.
- Weinger, M. B.; Banerjee, A.; Burden, A. R.; McIvor, W. R.; Boulet, J.; Cooper, J. B.; et al. Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists. Anesthesiology 2017, 127(3), 475–489. [Google Scholar] [CrossRef] [PubMed]
- Dey, A.; Billinghurst, M.; Lindeman, R. W.; Swan, J. E. A systematic review of 10 years of Augmented Reality usability studies: 2005 to 2014. Frontiers in Robotics and AI 2018, 5. [Google Scholar] [CrossRef]
- Wanderling, C.; Saxton, A.; Phan, D.; Sheppard, L.; Schuler, N.; Ghazi, A. Recent Advances in Surgical Simulation For Resident Education. Current Urology Reports 2023, 24(11), 491–502. [Google Scholar] [CrossRef]
- Taba, J. V.; Cortez, V. S.; Moraes, W. A.; Iuamoto, L. R.; Hsing, W. T.; Suzuki, M. O.; et al. The development of laparoscopic skills using virtual reality simulations: A systematic review. PloS one 2021, 16(6), e0252609. [CrossRef]
- Liaw, S. Y.; Wu, L. T.; Soh, S. L. H.; Ringsted, C.; Lau, T. C.; Lim, W. S. Virtual Reality Simulation in Interprofessional Round Training for Health Care Students: A Qualitative Evaluation Study. Clinical simulation in nursing 2020, 45, 42–46. [Google Scholar] [CrossRef]
- Otero-Varela, L.; Cintora, A. M.; Espinosa, S.; Redondo, M.; Uzuriaga, M.; González, M.; et al. Extended reality as a training method for medical first responders in mass casualty incidents: A protocol for a systematic review. PLOS ONE, 2023, 18. [CrossRef]
- Al-Hashimi, K.; Said, U. N.; Khan, T. N. Formative Objective Structured Clinical Examinations (OSCEs) as an Assessment Tool in UK Undergraduate Medical Education: A Review of Its Utility. Cureus 2023, 15(5), e38519. [CrossRef]
- Vázquez-Minero, J. C.; Olmos-Zúñiga, J. R.; Alba, E. G. de; Iñiguez-García, M. A.; Santillán-Doherty, P.; Chávez-Tinoco, A.; et al. La simulación con modelo biológico, como herramienta en el proceso de enseñanza-aprendizaje de la residencia de Cirugía Torácica en México. Neumologia y cirugía de tórax 2019, 78(1), 20–24. [Google Scholar] [CrossRef]
- Said, S.; Gozdzik, M.; Roche, T. R.; Braun, J.; Rössler, J.; Kaserer, A.; et al. Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) Questionnaire to Assess Perceived Workload in Patient Monitoring Tasks: Pooled Analysis Study Using Mixed Models. Journal of Medical Internet Research 2020, 22(9), e19472. [CrossRef]
- Thielmann, B.; Pohl, R.; Böckelmann, I. Heart rate variability as a strain indicator for psychological stress for emergency physicians during work and alert intervention: a systematic review. Journal of occupational medicine and toxicology (London, England) 2021, 16(1), 24. [CrossRef]
- Bruno, R. R.; Wolff, G.; Wernly, B.; Masyuk, M.; Piayda, K.; Leaver, S.; et al. Virtual and augmented reality in critical care medicine: the patient’s, clinician’s, and researcher’s perspective. Critical Care (London, England) 2022, 26(1), 326. [CrossRef]
- Gerrah, R.; Haller, S. J. Utilizing the Fourth Dimension for Patient Education in Cardiovascular Surgery. The Annals of Thoracic Surgery, 2022, 113, e59. [Google Scholar] [CrossRef] [PubMed]
- Cross, D. E.; Bauer, T. M.; Tchantchaleishvili, V. 3D organ modeling with open-source software. Artificial Organs, 2018, 43, 596. [Google Scholar] [CrossRef] [PubMed]
- Lerner, D.; Mohr, S.; Schild, J.; Göring, M.; Luiz, T. An Immersive Multi-User Virtual Reality for Emergency Simulation Training: Usability Study. JMIR serious games 2020, 8(3), e18822. [CrossRef]
- Kyaw, B. M.; Saxena, N.; Posadzki, P.; Vseteckova, J.; Nikolaou, C. K.; George, P. P.; et al. Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. Journal of Medical Internet Research, 2019, 21. [CrossRef]
- Watson, M. A.; Howle, A. M.; Lenz, O.; Scalese, R. J.; King, J. D.; Bolanos, J. A.; et al. Adapting an Objective Structured Clinical Examination (OSCE) on Conducting Difficult Conversations Between Nephrology Fellows and Patients to a Virtual Platform: A Research Letter. Canadian journal of kidney health and disease 2023, 10, 20543581221150553. [Google Scholar] [CrossRef] [PubMed]
- Hernández-Mustieles, M. A.; Lima-Carmona, Y. E.; Pacheco-Ramírez, M. A.; Mendoza-Armenta, A. A.; Romero-Gómez, J. E.; Cruz-Gómez, C. F.; et al. Wearable Biosensor Technology in Education: A Systematic Review. Sensors (Basel, Switzerland) 2024, 24(8), 2437. [CrossRef]
- Aganov, S.; Nayshtetik, E.; Nagibin, V.; Lebed, Y. Pure purr virtual reality technology: measuring heart rate variability and anxiety levels in healthy volunteers affected by moderate stress. Archives of medical science : AMS 2020, 18(2), 336–343. [Google Scholar] [CrossRef] [PubMed]
- Rizzo, A.; Parsons, T. D.; Lange, B.; Kenny, P.; Buckwalter, J. G.; Rothbaum, B.; et al. Virtual reality goes to war: a brief review of the future of military behavioral healthcare. Journal of Clinical Psychology in Medical Settings 2011, 18(2), 176–187. [Google Scholar] [CrossRef] [PubMed]
- Nayar, S. K.; Musto, L.; Fernandes, R.; Bharathan, R. Validation of a virtual reality laparoscopic appendicectomy simulator: a novel process using cognitive task analysis. Irish Journal of Medical Science 2019, 188(3), 963–971. [Google Scholar] [CrossRef] [PubMed]
- Bing, E. G.; Parham, G. P.; Cuevas, A.; Fisher, B.; Skinner, J.; Mwanahamuntu, M.; et al. Using Low-Cost Virtual Reality Simulation to Build Surgical Capacity for Cervical Cancer Treatment. Journal of Global Oncology 2019, 5, 1–7. [Google Scholar] [CrossRef] [PubMed]





| Learning Item | Z | p | r |
|---|---|---|---|
| Speed_VR - Speed_mani | -0.25 | 0.806 | -0.04 |
| Pulsi_VR - Pulsi_mani | -3.00 | 0.003 | -0.50 |
| Satu_VR - Satu_mani | -1.00 | 0.316 | -0.17 |
| BP_VR - BP_mani | -1.11 | 0.268 | -0.18 |
| Electrod_VR - Electrod_mani | -1.54 | 0.124 | -0.26 |
| Handling_VR - Handling_mani | -2.56 | 0.011 | -0.43 |
| Gloves_VR - Gloves_mani | -0.34 | 0.731 | -0.06 |
| Asepsis_VR - Asepsis_mani | -2.31 | 0.021 | -0.38 |
| Drape_VR - Drape_mani | -1.14 | 0.256 | -0.19 |
| Locate_VR - Locate_mani | -0.26 | 0.796 | -0.04 |
| Angle_VR - Angle_mani | -1.27 | 0.203 | -0.21 |
| Gguide_VR - Guide_mani | -1.74 | 0.083 | -0.29 |
| Drainage_VR - Drainage_mani | -2.34 | 0.019 | -0.39 |
| Total_VR -Total_mani | -0.58 | 0.561 | -0.10 |
| Condition | N | Mean | SD | Min | Max | 25th Perc | Median | 75h Perc |
|---|---|---|---|---|---|---|---|---|
| rMSSD_BASAL | 36 | 88.92 | 72.575 | 12 | 349 | 41.25 | 61.00 | 100.75 |
| rMSSD_VR | 36 | 91.81 | 85.381 | 18 | 381 | 33.25 | 58.00 | 126.25 |
| rMSSD_MANI | 36 | 155.42 | 375.796 | 15 | 2311 | 44.75 | 71.50 | 145.75 |
| LF_BASAL | 36 | 4112.00 | 7.773.149 | 104 | 38646 | 902.50 | 1440.50 | 3535.00 |
| LF_VR | 36 | 5097.28 | 5.720.810 | 261 | 24931 | 1160.75 | 2285.50 | 9398.00 |
| LF_MANI | 36 | 3345.83 | 3.769.927 | 477 | 17888 | 1145.75 | 2378.00 | 3647.50 |
| HF_BASAL | 36 | 5746.58 | 15.880.231 | 49 | 93054 | 525.25 | 982.00 | 4492.75 |
| HF_VR | 36 | 3568.11 | 4.966.091 | 148 | 18728 | 279.00 | 1224.00 | 4048.00 |
| HF_MANI | 36 | 2930.17 | 3.385.364 | 78 | 15175 | 548.75 | 1926.00 | 4686.25 |
| LF_HF_BASAL | 36 | 3.036 | 9.148 | 0.40 | 56.00 | 0.800 | 1.050 | 2.175 |
| LF_HF_VR | 36 | 3.465 | 5.717 | 0.50 | 35.00 | 0.925 | 2.200 | 4.150 |
| LF_HF_MANI | 36 | 2.734 | 2.670 | 0.40 | 10.40 | 0.700 | 1.050 | 4.675 |
| pNN20_BASAL | 36 | 54.89 | 19.371 | 10 | 92 | 44.25 | 54.50 | 67.50 |
| pNN20_VR | 36 | 47.22 | 18.891 | 10 | 92 | 33.00 | 46.00 | 57.75 |
| pNN20_MANI | 36 | 45.78 | 20.044 | 16 | 92 | 32.25 | 42.50 | 61.00 |
| pNN50_BASAL | 36 | 23.75 | 20.050 | 0 | 80 | 10.00 | 18.50 | 28.75 |
| pNN50_MANI | 36 | 21.19 | 21.794 | 0 | 89 | 6.00 | 13.50 | 29.50 |
| pNN50_VR | 36 | 18.03 | 18.894 | 1 | 82 | 5.25 | 12.00 | 23.75 |
| SD1_SD2_BASAL | 36 | 0.407 | 0.160 | 0.18 | 0.84 | 0.260 | 0.405 | 0.538 |
| SD1_SD2_VR | 36 | 0.465 | 0.201 | 0.20 | 0.87 | 0.285 | 0.415 | 0.590 |
| SD1_SD2_MANI | 36 | 0.519 | 0.197 | 0.20 | 0.88 | 0.330 | 0.530 | 0.675 |
| rMSSD_BASAL | 36 | 88.92 | 72.575 | 12 | 349 | 41.25 | 61.00 | 100.75 |
| rMSSD_VR | 36 | 91.81 | 85.381 | 18 | 381 | 33.25 | 58.00 | 126.25 |
| rMSSD_MANI | 36 | 155.42 | 375.796 | 15 | 2311 | 44.75 | 71.50 | 145.75 |
| Condition | Mean Rank |
|---|---|
| rMSSD_BASAL | 2.03 |
| rMSSD_VR | 1.78 |
| rMSSD_MANI | 2.19 |
| LF_BASAL | 1.83 |
| LF_VR | 2.21 |
| LF_MANI | 1.96 |
| HF_BASAL | 2.19 |
| HF_VR | 2.01 |
| HF_MANI | 1.79 |
| LF_HF_BASAL | 1.79 |
| LF_HF_VR | 2.25 |
| LF_HF_MANI | 1.96 |
| pNN20_BASAL | 2.35 |
| pNN20_VR | 1.90 |
| pNN20_MANI | 1.75 |
| pNN50_BASAL | 2.36 |
| pNN50_MANI | 1.99 |
| pNN50_VR | 1.65 |
| SD1_SD2_BASAL | 1.64 |
| SD1_SD2_VR | 1.88 |
| SD1_SD2_MANI | 2.49 |
| rMSSD_BASAL | 2.03 |
| rMSSD_VR | 1.78 |
| rMSSD_MANI | 2.19 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).