Immersive VR (Virtual Reality) Simulator for Vein Blood Sam-Pling

1. Introduction

Clinical practice is one of the most essential parts of medical education. Clinical practice plays a very important role in nurturing high-quality medical personnel by learning through practice how the theoretical contents of the medical education course are used in the actual clinical field [1,2]. For this reason, equipment for clinical practice is being developed and advanced. However, although various equipment and materials are used for clinical practice, there is a problem of using only a limited environment and model for clinical practice due to many limitations of the practice room and laboratory. Accordingly, many medical educational institutions perform only fragmentary clinical practice, and experience to understand the actual clinical field is not sufficiently provided. This situation in which the provision of experience is not sufficient makes it difficult for trainees to understand the anatomical features and may cause problems of clinical practice [1,3,4]. To solve the problem of clinical practice, medical educational institutions or trainees use various methods. Typically, mutual practice is performed between the patient and the trainee or between the trainee and the trainee, or a high-quality anatomical model is separately prepared and used for practice. However, the method of conducting mutual practice between the patient and the trainee or between the trainee and the trainee has disadvantages such as ethical or stability problems and difficulty in securing patients suitable for clinical practice, and the method of performing practice through a high-quality anatomical model requires a lot of time and money to search and purchase a high-quality model suitable for clinical practice, and it is difficult to respond to unexpected situations that occur in actual clinical practice. To solve these shortcomings, simulators using virtual reality (VR) or augmented reality (AR) technology are being used for clinical practice in many medical education fields [5,6]. In Northampton university, in order to improve nursing students' ability to respond to clinical situations, a virtual reality simulation product that integrates four HMD (Head mounted display) devices and a large screen was developed. Through this, nursing students can experience various clinical scenarios in a virtual environment, learn and verify clinical practice and communication skills [7]. Ox-ford University developed OMS (Oxford Medical Simulation) to improve the clinical practice of nurses, doctor and medical students working at John Radcliffe Hospital. through this simulation, clinical practice in nursing education, medical emergencies, pediatrics, internal medicine, mental health care, and physical examination can be learned and verified [7,8]. Falah et al. [9] developed a virtual reality-based cardiac anatomy education system using stereo 3D glasses to solve the problems of existing visualization-based anatomy education methods, and confirmed its usefulness through a group of medical experts. Schild et al. [10] developed an emergency response training simulation for anaphylactic shock, and improved training efficiency by adding a module so that two trainees and one educator can access the simulation at the same time. Pulijala et al. [11] developed a virtual reality education tool for loft type 1 fracture using HMD and Leap Motion(Ultraleap, Silicon Valley, US) [12], and verified the usefulness of the virtual reality education tool through a surgeon to confirm the potential of VR surgery for orthodontic surgery training. Ropelato et al. [13] produced an augmented reality-based microsurgery simulator that works with AR glasses for micromanipulation training in ophthalmic surgery, and verified the effectiveness of the simulator with 50 trainees. As a result of the verification, it was confirmed that the training method through the simulator improved the fine manipulation ability compared to the existing training method. Rhienmora et al. [14] developed a dental practice simulator using a haptic device and an augmented reality algorithm AR Toolkit [15], and several educators and experts identified the advantages and disadvantages of the technical practice simulator to confirm the potential of augmented reality technology in the field of dental practice. Si et al. [16] enhanced CT data-based brain 3D models to 3D printed skull models, developed an educational simulator that can train brain tumor resection through Hololens [17] and two haptic devices, and confirmed the usefulness and potential of augmented reality technology in neurosurgery training.

Many medical educational institutions and hospitals are developing and utilizing virtual reality or augmented reality-based clinical practice tools for training in various medical fields. However, studies on the development of simulators based on virtual reality or augmented reality related to vein blood sampling practice are still incomplete. Vein blood sampling is the most commonly used method of blood sampling, and as one of the methods of mass blood sampling, it is the act of extracting blood from veins. It is used to distinguish blood types, check various physiological indicators, and diagnose diseases. Then, a vein blood sampling interaction plan is implemented to perform vein blood sampling through 3D model manipulation using interactive equipment. Finally, the effectiveness of the simulator was evaluated by performing a usability test on the vein blood sampling simulator implemented for trainees in medical educational institutions. Therefore, this paper proposes a plan to develop a vein blood sampling simulator using virtual reality and interaction equipment. To this end, an environmental 3D model, an interactive 3D model and a blood vessel 3D model related to vein blood sampling tools are created to construct a virtual space similar to the actual space for vein blood sampling, and a virtual space is created using a virtual environment production tool. Then, a vein blood sampling interaction plan is implemented for performing vein blood sampling through 3D model manipulation using interactive equipment. Finally, we would like to evaluate the utility of the simulator by performing a usefulness test on the vein blood sampling simulator implemented for educators in medical education institutions.

2. Materials and Methods

2.1. Production of virtual space for vein blood sampling

In order to create a simulator for learning vein blood sampling, information on the actual environment where vein blood sampling is performed was collected, and based on this, a 3D model of the space where vein blood sampling was performed, related tools, and a patient model was directly created using 3D CAD software. 3D models include geometry, color, and material information, and were saved in a format (DXF, OBK, FBX, etc.) usable in Unity (Unity, San Francisco, US), a virtual environment production software. Figure 1 shows the configured virtual space for vein blood sampling learning.

The 3D models constituting the virtual space are classified into three types: an environment model expressing the space where vein blood sampling is performed, a 3D model that interacts with the user's (educator's) HMD Controller or Haptic device manipulation, and a 3D model of a vacuum tube containing blood for vein sampling. Table 1 shows the classification of 3D models according to the utilization plan.

In the case of hospital rooms and medical beds among the models belonging to the environmental model classification, they were produced very similar to the actual environment information referenced to improve the user's immersion, and rigid body properties were given to prevent overlapping or passing through other object 3D models. In the case of the patient 3D model, the left arm was exposed so that vein blood sampling could be performed based on the left arm, and a clothing 3D model was created and attached to the patient 3D model to reduce rendering resources for body parts models other than the left arm. Figure 2 indicates 3D models included in the environment model.

In addition, an anatomical 3D model was produced for the left arm of the patient 3D model to increase the sense of reality in performing vein blood sampling. To this end, bone, muscle, artery, and vein structures for areas belonging to the left arm (upper arm, lower arm, wrist, hand) were investigated, a 3D volume model was created and classified by structure, and 3D geometric information for each structure was collected. Then, based on the collected information, we fabricated a 3D model of each structure for the area belonging to the left arm, and added collision information to each structure 3D model for interaction through the Haptic device. Figure 3 shows the left arm anatomy 3D model.

Among the interaction model classification, the hand 3D model is linked with the user's controller or haptic device movement, the left hand is used for UI and 3D patient model's left arm operation and vacuum sampling tube selection, and the right hand is used for operating the Tourniquet and alcohol swab. It is used to operate the syringe for blood sampling. In addition, collision information was added to the 3D hand model, Tourniquet, alcohol swab, and syringe model for smooth selection and operation. The details of the interaction method using the 3D model included in the interaction model will be discussed in detail in 2.2. Figure 4 indicates 3D models included in the interaction model.

And in the case of hand 3D models, hand motion animation was produced to express natural hand shapes when interacting with Tourniquets, alcohol swab, Syringes, etc. To express hand movements, Leap Motion, a device that can recognize user's hand movements through depth cameras, and software that can produce and edit human motion animations were used. Major keyframe data for hand motion was acquired with hand skeleton data obtained through Leap Motion, and then hand motion animation was generated by applying keyframe interpolation. Figure 5 shows object utilization animation using Leap Motion.

In the case of the vacuum tube model among the blood sampling model, six vacuum tube (Solium, SST, Plain, Heparin, PST, EDTA) corresponding to each blood sampling or-der were manufactured to the appropriate dose (2ml, 3ml, 4.5ml, 6ml, 10ml), adjusted, and placed on the tray 3D model. To prevent overlapping or passing between the vacuum tube 3D model and the tray 3D model, rigid properties were added to each model, and collision information was added to the vacuum sampling 3D model for vacuum tube selection through the left-handed 3D model. Figure 6 shows the 3D models included in the blood sampling model.

2.2. Implementation of vein blood sampling interaction

HMD controllers (Oculus Quest 2, Facebook, Inc, US) and Haptic device (Touch, 3D systems, California, US) are used as a way to interact between users, vein blood samplings, and 3D models in virtual space in a vein simulator. In the case of HMD Controllers, inter-actions with virtual objects are performed using ray-cast techniques. When the object is in contact with the virtual object for more than a certain period of time by projecting a ray in the direction indicated by the stop of the hand 3D model linked to the HMD controller, the object is determined to be selected. Based on this, UI selection, manipulation of the left arm of the 3D model of the patient, selection of the vacuum tube, and selection and manipulation of the Tourniquet or alcohol swab are performed. Figure 7 shows the interaction method used in the HMD controller.

In the case of a Haptic device, interactions are performed using physical collisions. When the movement of the Haptic device is linked to the hand 3D model, and the hand 3D model directly collides with the virtual object through the Haptic device, the object is determined to be selected, and compression bands or alcohol swab selection and operation, Syringe selection and operation are performed. In addition, when a selected object is contacted with a patient 3D model through a Haptic device, the physical reaction is fed back to the user with a touch to improve the sense of immersion in the interaction, and processes such as tying a Tourniquet and applying alcohol swab are performed. Also, the angle between the blood sampling area of the patient 3D model and the syringe 3D model is calculated using a vector, and when the syringe insertion angle range (about 15-25 degrees) is reached during actual vein blood sampling, the syringe 3D model is inserted into the patient 3D model's blood sampling area to perform Vein blood sampling. Figure 8 shows the interaction method using the Haptic device, and Figure 9 shows the method of calculating the angle between the blood sampling area and the Syringe 3D model.

2.3. Evaluation of vein blood sampling simulator

This study’s protocol was approved by the institutional ethics committee (CUDHIRB 2202005). In order to verify the effectiveness of the implemented vein blood sampling simulator, a usability evaluation of the virtual environment-based vein blood sampling simulator was conducted on 40 dental college students who had no experience in performing virtual environment-based content. For the objectivity of the usability evaluation results, students who had no experience in actual vein blood sampling were selected, and education on vein blood for actual patients was conducted through textbooks and practice books. In addition, the learning method for the vein blood sampling process was regenerated in the vein blood sampling simulator, and subjects performed vein blood sampling simulations according to the learning method. Figure 10 shows the performance of the virtual environment-based vein blood sampling simulator.

After performing the vein blood sampling simulator, a questionnaire consisting of 5 items was provided, and each item was evaluated on a 5-point scale (very good(5), good(4), normal(3), bad(2), very bad(1)). The five items are simulator resolution, simulator accuracy, simulator immersion, simulator usability, simulator educational usefulness, and each item consists of 4 or 5 questions.

3. Results

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

3.1. Implementation of vein blood sampling simulator

Based on the proposed method, a vein blood sampling simulator was implemented. Unity was used to implement the virtual environment of the vein simulator, and virtual objects for the environmental model, interaction model, and sampling model that make up the virtual environment were created using 3D CAD software 3Ds Max [18] and Cine-ma4D [19]. In addition, HMD used HMD and Controller of Oculus Quest 2 for a realistic experience of the virtual environment of the vein simulator, Android SDK, Android NDK, and Oculus SDK were used for smooth interworking of Oculus Quest 2 in Unity, and Haptic devices used to perform vein simulators were used to smoothly. In addition, the module for interaction between the controller of Oculus Quest 2 and the virtual object through the Haptic device was self-produced based on the C# script. In addition, a simple GUI allows users to select necessary functions in real time in a virtual environment, train vein blood sampling, and check the current status. Guide to performing vein blood sampling, skin transparency and strength of patient 3D models, and recording vein blood sampling can be used through GUI buttons, and injector angles for vein blood sampling, and current vein names are displayed through GUI labels. Figure 11 shows the execution scene of the vein blood sampling simulator implemented in the Unity environment.

3.2. Application and Efficacy evaluation of vein blood sampling simulator

Table 2 shows the usability evaluation items and result for vein blood sampling simulation. Figure 12 shows a graph of average results of usability evaluation.

As a result of the evaluation, the average score of all items was 3.807 or higher. Analyzing the results by item, in the case of resolution, an average of 4.06 points and a standard deviation of 0.847 points for 4 questions were obtained, and in the case of accuracy, an average of 3.59 points and a standard deviation of 1.017 points for 5 questions were obtained. In the case of immersion, an average of 3.475 points and a standard deviation of 1.079 points were shown for 4 questions. In the case of usefulness, an average of 4.0125 points and a standard deviation of 0.941 points were shown for the four questions. The reason for the low accuracy and immersion scores compared to other items is that there is no experience with virtual environment contents, due to inexperience in manipulation (difficulty in navigating the linked hand model, difficulty in manipulating virtual objects, etc.), and this is expected to be improved by repeatedly experiencing various virtual environment-based contents.

4. Discussion

In this paper, a method for developing a vein blood sampling simulator based on a virtual environment was proposed, and the effectiveness of the developed vein blood sampling simulator was confirmed. The environmental model, interaction model, and blood sampling models used in the vein blood sampling simulator were created using 3D CAD software, and a patient 3D model including an anatomical 3D model was created and used to improve the realism of vein blood sampling, and information such as rigid body and collision was created for smooth interaction of each model using Unity. And, for interaction with virtual objects in the Unity environment, an interaction method using HMD controller-based ray-cast and an interaction method using haptic device-based collision were implemented and applied to the simulator. Then, in order to verify the effectiveness of the vein blood sampling simulator, usability evaluation was conducted on 40 dental students, and as a result, the validity of the vein blood sampling simulator was verified. As a result, it was confirmed that the practice of vein blood sampling is possible through the proposed vein blood sampling simulator.

However, it was confirmed that the proposed vein blood sampling simulator method was not suitable for application to the evaluation of vein blood sampling skills because it was difficult to evaluate the user's vein blood sampling practice process in real time and objectively. In the future, we will study the method for applying the proposed vein blood sampling simulator to the evaluation of vein blood sampling skills. To this end, we plan to develop a multi-access module that allows a large number of people to access the simulator so that the vein blood sampling practice process can be checked in real time, and we develop artificial intelligence model for evaluation of vein blood sampling techniques to objectively evaluate the user's vein blood sampling technique process.