Could You Evaluate Sounds in a Virtual Environment?: Evaluation Components of Auditory Experience in a Metaverse Environment

1. Introduction

Metaverse technology is growing rapidly as it is used in various industries such as manufacturing, construction, education, healthcare, and social networking to enhance the participation of society members by recreating sensory experiences [1,2]. Especially in metaverse fields such as education, gaming, and healthcare, research on presence and immersion has been actively conducted to provide more realistic experiences, and such presence is an essential element of a metaverse environment [3,4,5,6,7,8]. There is an ongoing discussion on how to integrate sensory information such as visual and auditory information of users in realizing presence, and up to now, auditory factors have been used mainly as a supplement to visual information, but recently, realization of enhanced presence has become an important task, and realization of a lifelike auditory environment is becoming important [9]. In general, questionnaire evaluation methods such as presence questionnaire (PQ) and slater-usoh-steed presence questionnaire (SUS-PQ) are used to identify presence, and recently, auditory factors have been treated as a major factor to improve presence, and the need to evaluate them has emerged [10,11]. However, the existing questionnaires for evaluating presence have the limitation that it is difficult to evaluate auditory presence because there are fewer auditory examples compared to visual studies [12,13], and presence is evaluated by various factors such as emotion, ease, and familiarity that users experience in interacting with avatars and systems [14], but the existing questionnaires for evaluating auditory presence do not fully reflect these aspects. In particular, considering that the sense of presence is evaluated by factors such as the illusion created by the user’s sensory information, the cumulative user experience (UX) of using the system, and the experience formed after using the system, it seems that it can be used if evaluation factors related to the sense of presence are added to the existing UX evaluation factors [8,15,16], but the existing UX evaluation contents have a limitation that it is difficult to evaluate the auditory part due to its general characteristics [17].

Against this background, since UX evaluation factors affect the understanding of presence, and presence evaluation factors act to understand UX [15,16], extracting common components of AUX evaluation factors and auditory presence evaluation factors may be able to identify auditory experience evaluation factors considering presence enhancement in metaverse environments. Therefore, this study aims to develop an auditory experience evaluation questionnaire considering presence by conducting a literature review and primary and secondary expert interviews to improve the presence of the metaverse environment, and to derive the evaluation components considering auditory presence and AUX through questionnaires, PCA, and RA.

2. Literature Review

2.1. Presence and Auditory Characteristics of a Metaverse Environment

An aspect that should be considered to provide a more realistic experience in the field of metaverse is the sense of presence, which is considered an essential prerequisite for visual, tactile, as well as auditory factors [10]. The sense of presence considered in metaverse environments is related to sensations such as perceiving oneself as existing in a virtual space [18] or being transported to a virtual space [19], and the feeling of ‘being there,’ i.e., the subjective experience of the receiver focusing on and immersing in the virtual world [20]. In metaverse environments, presence is also related to the perception of the user and others with whom they interact, including the experience of perceiving virtual objects as if they were real [21], the degree to which the user identifies with the avatar's body [22], the degree to which they feel they are interacting with others [23,24], and the degree to which the recipient attributes personality to virtual humans, making them perceive them as social actors [25,26]. Therefore, the sense of presence can be categorized as the feeling of being transferred and immersed in a virtual world [18,19,20], the feeling of similarity to a real person [21,22], and the feeling of similarity in social behavior [23,24,25,26]. Table 1 shows the definitions and classification of presence by researchers.

In a metaverse environment, presence is interrelated to spatial, temporal, and acoustic properties related to hearing. Spatial characteristics describe where the sound is coming from, temporal characteristics describe when the sound is heard, and acoustic characteristics describe the pitch or intensity of the sound. These are the auditory characteristics of a metaverse environment because they help users recognize their location or orientation in the virtual space, determine their distance or relationship to objects or people, and feel emotions or moods [27]. The auditory evaluation factors of a metaverse environment can be derived by identifying the auditory factors that make the metaverse environment feel real [19]. In addition, since the equipment elements of the metaverse environment, subjective factors considering the user's personality characteristics and immersion tendencies, social factors arising from interactions with users, emotional factors, movement in the virtual environment, and familiarity can affect the sense of presence of the metaverse environment [14,28], the UX factors of existing accumulated digital devices, including the newly emerging metaverse environment, can be considered together [16]. Therefore, it is desirable to consider the auditory factors of the metaverse environment together with the AUX evaluation factors of digital devices that are familiar to the user and the auditory presence evaluation factors of the newly emerging metaverse environment.

2.2. Auditory Evaluation Factors in the Presence Evaluation Questionnaire for a Metaverse Environment

Since most of the evaluations of auditory presence in metaverse environments are technical evaluations of engineering sensors, and it is difficult to find data in the form of questionnaires for auditory presence evaluation, we first conducted a literature review of auditory evaluation factors in general presence evaluation questionnaires. In order of development date, the reviewed questionnaires include presence questionnaire (PQ), slater-usoh-steed presence questionnaire (SUS-PQ), reality judgment presence questionnaire (RJPQ), igroup presence questionnaires (IPQ), independent television commission-sense of presence inventory (ITC-SOPI), engagement, enjoyment and immersion questionnaire (E2IQ), and measurement, effects, conditions spatial presence questionnaire (MEC-SPQ).

PQ categorizes presence into three characteristics: involvement, immersion, and presence, and emphasizes sensory stimulation, participation, immersion, and interaction. The auditory presence evaluation factors include QN 6 (How much did the auditory aspects of the environment involve you?), QN 15 (How well could you identify sounds?), and QN 16 (How well could you localize sounds?), which are all about presence [20]. SUS-PQ assesses the sense of presence, which is the degree to which a person feels that he or she exists in a virtual environment rather than in an actual physical space. In particular, the degree to which a person is convinced that fictional information is real is viewed as presence, and it measures the degree of presence caused by the relevance of the user and the avatar. SUS-PQ consists of five evaluation factors: the experience and degree of 'being there', comparison of virtual and real images, association of the visual structure of the virtual environment, and the frequency of detecting virtual reality [28,29].

RJPQ categorizes presence into nine dimensions: reality judgment, presence, involvement, interaction, control, attention, realism, perceptual congruence, and expectations in virtual environment, and emphasizes natural interaction. The auditory presence dimension consists of three questions: QN 4 (How clear were the sounds in the virtual world?) on realism, QN 73 (To what extent did what you heard and the quality of the sound in the virtual world influence how real the experience seemed to you?) on reality judgment, and QN 76 (To what extent did the sounds influence how deep into the virtual world you went?) on presence [30]. IPQ categorizes presence into three factors: spatial presence, involvement, and realism, and covers a wide range of subjective experiences, immersion, and interaction, with no questions related to hearing [23]. ITC-SOPI categorized the sense of presence into four factors: sense of physical space, engagement, ecological validity, and negative effects, and found that not only the implementation of the physical environment, but also the overall usability, such as the attractiveness, naturalness, and trustworthiness of the content, could affect the sense of presence, and there were no questions related to hearing [31].

E2IQ categorizes the sense of presence into six factors: sensory factor, distraction factor, realism factor, control factor, pleasure factor, and satisfaction factor, and considers various UX aspects such as the attractiveness of the visual scene, the degree of distraction caused by noise outside the device, the realism and consistency of the sensation of virtual movement, and the immersion of the task. The auditory dimension is covered by QN 2 (To what extent did events such as noise occurring outside Crayolaland distract your attention from Crayolaland?), which is about distraction [32]. MEC-SPQ approaches the evaluation of presence from a UX-centered perspective rather than a technology-centered perspective, and its contents are important for redefining spatial presence, which has not been systematically addressed in existing studies. In the MEC-SPQ, there are questions that consider the user's state, behavior, and sense of presence, and in particular, the questionnaire on the sense of presence consists of questions about the sense of location of the user and the objects surrounding the user, and questions about whether the impression (feeling) received in the virtual environment allows the user to control the user's behavior [33]. As shown above, the evaluation factors of auditory presence in the questionnaire for evaluating presence in metaverse environments were examined and are shown in Table 2.

As shown in Table 2, the characteristics of the sense of presence questionnaire include sensory stimulation, engagement, immersion, interaction, attractiveness, distraction, and coherence, confirming that the multifaceted experience of a virtual world device can affect the sense of presence. In addition, because of examining the auditory presence evaluation factors in existing presence evaluations, most of the evaluation items related to hearing are very small, so it is very difficult to identify the auditory evaluation factors only from the evaluation questionnaire. Therefore, the auditory evaluation factors of the metaverse environment can be derived by identifying the auditory factors that make the metaverse environment feel real [19]. Based on the literature study, we reviewed the auditory presence factors of the metaverse environment as follows.

2.3. Prior Research on Auditory Presence in a Metaverse Environment

In addition to the implementation of acoustic technologies such as soundscapes that embody real-world sounds [34,35,36,37,38,39], studies on the quality of sound sources [34,39], studies on the stereophonics of sounds [34,39], studies on the spatial characteristics of sounds [38,39,39,40], studies on auditory cues that help with navigation tasks [39], and studies on auditory factors that match visuals [35,37], factors such as coherence between sounds [12] have been identified as influencing the sense of presence. Table 3 shows research examples of auditory presence evaluation factors in metaverse environments identified by researchers.

As shown in Table 3, we analyzed the research cases of auditory presence evaluation factors in metaverse environments and found that even the same kind of auditory presence evaluation factors are classified as different presence factors by individual researchers. In the case of 'spatial acoustics', some studies classified presence as presence [34,38,39], while others classified presence as immersion [37], presence [39], and social presence [40]. Studies on 'soundscape' classify presence as presence [35] and social presence [36], and studies on 'auditory factors related to visual experience' classify presence [35] and presence and immersion [37]. In other words, we found that auditory presence factors are not yet systematically organized as they are classified differently by different researchers. This result is consistent with a literature review that found it difficult to generalize about auditory presence [10]. Against this background, this study focuses on auditory presence, which is the most basic and essential element of virtual worlds, i.e., the feeling of 'being there', and the auditory sensation such as the transfer of sounds from the physical world to the virtual space, among various concepts of presence.

2.4. Reflections on the Evaluation Factors of AUX for Products and Services

UX is a branch of interaction research in human computer interaction (HCI) that refers to the total experience, including usability and emotions, experienced in interactions with products, systems, and services [41,42]. AUX factors are specifically about the auditory aspects of UX and include musical elements (melody, rhythm, harmony, timbre, etc.), auditory user interface (AUI) such as notifications that occur while using products and services; AUI such as notifications that occur while using products and services, auditory backgrounds such as ambient sounds and noises, and voice user interfaces (VUI) that assist in using the product [43]. AUX is also considered very important in the sense that it provides differentiated emotions and usability through hearing and imprints a company's identity and is often described as the counterpart of graphic user interface (GUI) [44].

Since the evaluation of UX is characterized by avoiding formalized formats and reconfiguring them to fit the unique features of the target audience [45], AUX factors can be described as evaluating the usability and emotions that users experience in auditory interactions with products, systems, and services [46]. However, research on auditory UX factors does not have the same standard as the existing visual UX evaluation, 'heuristic evaluation of user interfaces' [46], so we conducted a literature review to identify auditory UX factors. The literature review includes a study on deriving a meaningful auditory attractiveness measure for UX [48], a study on proposing an auditory interface UX scale [17], a study on evaluating acoustic measurements and noise abatement in general [49], a study on delivering consistent and differentiated experience and value in products and services [50], a study on AUX in UX evaluation of glasses-type hearable products [51], and an expert interview on designing AUX in a web or app environment [52]. From these results, we found that the existing studies on AUX evaluation factors are mainly distributed in products and services, and we derived AUX evaluation factors according to researchers as shown in Table 4.

As shown in Table 4, a literature review of AUX evaluation factors shows that they deal with the accessibility and intuitiveness of the UX, such as 'ease of access to sound control' [51] and ‘how easy it is to understand what the sound represents' [17], while the characteristics of auditory presence evaluation factors differ in that they aim to realize the sound of reality, such as ' spatial properties of sound' [12,38,39,40] and ‘similarity to real-world sounds’ [30], ‘implementing real-world sounds' [34].

3. Methods

In this study, we conducted three steps in addition to the literature review to derive auditory experience evaluation components in a metaverse environment. In Step 1, we grouped auditory factors through primary expert interviews; in Step 2, we created an auditory experience evaluation questionnaire that considers presence so that it is easy for the general public to understand through secondary expert interviews; and in Step 3, we conducted surveys and statistical analysis. The overall research process is shown in Figure 1.

3.1. Step 1: Primary Expert Interviews for Grouping Auditory Eveluation Factors

The primary expert interviews were conducted to group the auditory evaluation factors derived from the literature review. Participants in the primary expert interviews were selected as experts with a level of understanding of the existing literature review, and with at least three years of experience in the field of sound design (sound design, planning, composition, etc.) in a metaverse environment. The primary expert interview method was conducted in the form of listening to the opinions of experts using an Excel sheet, and the participant information is shown in Table 5.

3.2. Step 2: Secondary Expert Interviews to Create a Questionnaire to evaluate auditory experiences for Presence

The purpose of the secondary expert interviews was to define auditory presence in a way that the public can understand, and to create an evaluation questionnaire that the public can easily understand. The criteria for recruiting participants for the secondary expert interviews and the interview method were the same as the primary expert interviews, and additional experts in the fields of music education and music therapy were selected to help the public understand musical terms. We defined auditory presence based on the contents of the literature review [10] that it is difficult to generalize about auditory presence, and the participant information is shown in Table 6.

3.3. Step 3: Survey and Statistical Analysis

To derive an evaluation component of auditory experience that takes into account the presence of the metaverse environment, a survey was conducted using a 5-point Likert scale on the auditory factors that enhance the presence of the metaverse environment. We recruited participants who met the following criteria: they had used a VR device at least once to understand their familiarity with the virtual world, played games and metaverse platforms at least once or twice a week, and listened to music or media content at least 2-3 times a week to be able to discriminate auditory stimuli. The total number of participants was 232 (139 males and 93 females), with 67 participants aged 10-19, 60 participants aged 20-29, 60 participants aged 30-39, 25 participants aged 40-49, and 20 participants aged 50-59, with 80.6% of participants aged 10-39. The survey was also subjected to PCA to reduce the data with as little loss of information as possible. The PCA was performed in IBM SPSS® Statistics, with a varimax factor rotation, and a RA was performed on the derived values to confirm the appropriateness of the newly derived evaluation components of auditory experience

4. Results and Discussion

4.1. Results of Grouping for Auditory Evaluation Factors

From the primary expert interviews, the auditory evaluation factors identified in the literature review were grouped and presented in Table 7.

As shown in Table 7, the factors of ‘the concentration of contents due to hearing’ [20], and ‘presence with or without auditory backgrounds’ [30] were grouped as 'focusing the metaverse environment with sound’ because they are classified as the same concept of ‘sense of presence’ in the reviewed literature [28,30,39]. The factors of ‘the spatial characteristics of sound’ [12], ‘spatialized sound’ [38], ‘spatialized sound source’ [39], and binary audio (spatial acoustics) [40] were grouped into ‘implementing the spatial properties of sound' because they use the common concept of ‘space’ according to P1’s opinion. ‘Similarity to real-world sounds [30], ‘realization of real sounds’ [34], ‘reproduction of soundscapes’ [35], and ‘everyday soundscapes’ [36] were grouped ‘implementing realistic soundscapes in a metaverse environment’ because they use the common auditory concept of ‘real soundscapes ‘according to P1’s opinion. ‘Perceived location factor of sound’ [20], three-dimensional spatial representation of sound [34], and ‘three-dimensional representation of sound’ [37] were grouped into ‘three-dimensional perception of sound’ because they use the common concept of ‘location’ according to P2’s opinion.

The factors of ‘good sound’ [17], interesting sound [17], boring sound [17], unpleasant sound [48], wanting to stay again [49], and unpleasant sound [49] were grouped under the common term ‘willingness to return due to sound’ because of their common meaning of measuring the impression of sound, according to all expert opinions. The factors of ‘quality of sound’ [12], ‘sound source quality’ [34], ‘degree of sound echoing’ [48] ‘degree of sound softness’ [48], ‘loudness of sound’ [48,49], and ‘keeping sounds short and smooth elements’ [52] were grouped into ’the quality of the sound source’ because they use the common auditory concept of ‘quality of sound’ according to P2's opinion. It seems that the factors of ‘how confusing it is to hear the sound’ [17], ‘distractions from noise outside the metaverse environment’ [32], and the ‘attracting attention without being distracting’ [52] can be grouped into 'how distracting the sound is' as they use the common auditory concept of 'distraction' according to P1 and P3's opinions. The factors of ‘auditory clarity’ [30], ‘the right amount of sound resolution’ [51], and ‘how clear the sound is’ [51] can be grouped into ‘clarity of sound’ because they use the common auditory concept of ‘resolution of sound’ in P2 and P3's opinions. The factor of ‘delivering the sound you expect visually’ [35], ‘ambience sounds to match the visual experience’ [37], and ‘degree of harmony between surroundings and sound’ [49] could be grouped together as ' sound experiences that live up to visual expectations’ as they use a common auditory concept in the sense that 'sight and sound should match' according to P3's opinion.

The factors of ‘consistency between sounds’ [12], ‘how difficult it is to understand the sound being varied’ [17], and ‘consistency between sounds within products and services’ [50] are believed to use a common auditory concept of ' how consistent the overall sound is in a sound variation situation’, and thus can be grouped into 'the degree to which the overall sound is consistent in the context of sound variation', by P3's opinion. The factors of ‘how much the sound helps’ [17], ‘how easy it is to understand the sound’ [17], ‘how easy it is to understand what the sound represents’ [17], ‘matching sound and meaning’ [17], ‘identification of the sound’ [20], and ‘auditory cues to help you navigate’ [39] can be grouped under ‘auditory cues to help you navigate’ as they share a common auditory concept of the role of sound in metaverse environments as an alternative to data navigation and visual interfaces based on a literature review [17]. As mentioned above, we conducted primary expert interviews and found that the grouping within the auditory presence evaluation factor was dominated by the tendency to realize a realistic auditory experience, while the grouping within the AUX evaluation factor was dominated by acoustic characteristics, such as the sufficiency of volume size control, the distribution of the sound range, and the ease of access to sound control. In the case of grouping between the auditory presence and AUX evaluation factors, we found that the factors covered by the existing AUX evaluation factors, such as 'auditory utility to help navigation', are also covered by auditory presence, confirming that the scope of auditory presence can be expanded.

4.2. Creatinge an Auditory Experience Evaluation Questionnaire that Considers the Public Understanding of Presence

Secondary expert interviews were conducted to explain the auditory presence and AUX evaluation factors in the public’s terms, and to create evaluation questionnaires that make it easier for the public to understand presence. As a result of the secondary expert interviews, the definition of auditory presence in a metaverse environment is shown in Table 8, the considerations for creating an evaluation questionnaire for auditory experiences with presence are shown in Table 9, and the terminology for the evaluation questionnaire is shown in Table 10.

Based on the secondary expert interviews shown in Table 8, we found that the experts' opinions on the question ‘How can we define auditory presence in a metaverse environment?’ were generally described as related to the auditory sensation of ‘being there,’ i.e., the transfer of sounds from the physical world to a virtual space, which is consistent with the theories advocated in the literature review{18, 19}, and we considered them when creating the presence evaluation questionnaire.

As shown in Table 9, we found that the auditory presence and AUX factors are subject to subjective interpretation and should be accompanied by examples, and that the examples should be described in as intuitive and everyday terms as possible, rather than in technical terms.

As shown in Table 10, we found that existing terms such as ‘three-dimensional’ can be concretized into ‘location and distance identification,’ and terms such as ‘approach to sound control’ can be simplified to be understood by the general public, such as ‘sound you want to reduce. After completing the expert interviews in Step 1 and Step 2, the questionnaire for evaluating the auditory experience with a final sense of presence is shown in Table A1 in the Appendix.

4.3. Results of the Survey and Statistical Analysis

The Kaiser-Meyer-Olkin (KMO) and Bartlett's test of sphericity indicators indicate the degree to which the correlation between variables is well explained by other variables, and low values suggest that the variables are not appropriate for PCA. The KMO value for the survey in this study is 0.830, and the probability of significance (p) is less than the significance level (0.001), so the model fit is high. RA shows that a Cronbach's α value of 0.6 or higher is generally accepted as reliable, and the reliability of this study is very high with a total component of Cronbach's α=0.896, and the Cronbach's α of component 1, component 2, component 3, component 4, and component 5 was 0.867, 0.832, 0.829, 0.903, and 0.744 respectively. In addition, the eigenvalue of the extracted components indicates the amount of variance in the component, with a value greater than 1 being significant, and all five components were found to be the main auditory components considered in the metaverse environment. Commonality values below 0.4 are subject to removal, but all of the components were above 0.5, so no components were removed, and the results are shown in Table 12.

From the results in Table 12, the cumulative variance of the five components, including 'realistic auditory background' and 'acoustic aesthetics', is 70.458%, which can explain more than 70% of the auditory experience of the metaverse environment, so it is believed that improving these components can improve the sense of presence. Component 1, which is explained by 35.320% (variance) of the total factors, consists of QNs 1, 2, 10, 5, and 16, and has the characteristics of ‘realistic auditory background implementation’ in that it considers the positional and spatial characteristics of sounds in the metaverse environment and the implementation of everyday soundscapes and auditory backgrounds (such as background music and everyday noises) that can evoke a place. In particular, considering that 'Component 1' is derived from the AUX factors 'harmonization between digital and natural sounds' (QN 16) and 'auditory presence factors related to constructing a realistic sound environment' (QNs 1, 2, 10, 5), it can be interpreted that the auditory environment of the metaverse should not be implemented as it is in reality, but should seek harmonization with existing digital sounds, which can enhance the sense of presence.

Component 2, described as 12.330% of the total factor, consists of QNs 3, 4, 18, and 20. Component 2 is characterized as ‘acoustic aesthetics’ in terms of fewer bad sounds, clearer sounds, auditory discomfort, and user revisits due to sounds in the metaverse environment. The reason why factors such as (QNs 3, 4, 18, and 20) were derived as ‘Component 2’ is that the auditory presence factor (QNs 3, 4), which is related to acoustic quality, considers ‘willingness to return due to sound’ (QN 20), which tends to be a strong AUX evaluation factor These results suggest that users' "willingness to return due to sound" (QN 20) does not directly increase presence, but can function as a checklist for presence by ensuring that "Component 2" has no issues with sound quality if metaverse use is low (QNs 3, 4).Comprising QNs 6, 19, 15, and 14, 'Component 3', which explains 8.590% of the total factor, can be described as 'consideration of acoustic control and accessibility' in terms of the granularity of sound volume and sound effect control in the metaverse environment and the ability to turn different kinds of sounds on and off. In particular, since 'Component 3' is the component with the most AUX evaluation factors, including 'Sufficiency of volume control' (QN 19), 'ease of access to sound control' (QN 14), and the auditory presence evaluation factor 'distraction from sound' (QN 6), it is judged that the AUX evaluation factor itself does not improve the sense of presence, but the sense of presence can be improved by controlling 'distraction from sound' (QN 6) through acoustic control and accessibility considerations. Comprised of QNs 11, 12, and 9, 'Component 4', explained as 7.848% of the total factor, is characterized by 'auditory utility and minimalist design' in the sense that AUI such as notifications, signals, etc. that help navigate in the metaverse environment should be designed in such a way that users do not notice the sound looping. In particular, ‘Component 4’ states that ‘signals that help with navigation’ (QN 11, 12) are not just AUI, but auditory signals that are experienced in the real world (such as car horns), and if the auditory signals are mechanically repeated, the sense of presence is diminished. Therefore, the auditory design of the metaverse environment should be minimalist, with sounds that can be useful to the user, rather than importing all the sounds of the real world.

Comprising QNs 13, 8, 17, and 7, 'Component 5' explains 6.371% of the total factor and can be described as 'auditory consistency', considering the coherence of sound and visual motion and atmosphere in the metaverse environment, as well as the coherence of sound changes in the content. 'Component 5' is grouped into 'sound matching visual motion timing' (QN 7), 'sound experience meeting visual expectations' (QN 8), 'relevance of sound to ideas' (QN 17), and 'degree of consistency of overall sound across sound variations' (QN 13), which is a strong tendency of AUX evaluation factors, suggesting that the unified experience of sight and sound covered by traditional auditory presence should be harmonized with the auditory experience across the metaverse environments. Based on these results, it is difficult to conclude that AUX factors improve presence on their own because they tend to be more functional, such as acoustic control and accessibility, but they can improve presence if they improve factors that reduce auditory presence factors, such as the degree of distraction from sound. In addition, since auditory presence factors tend to embody the auditory experience of the real environment in the metaverse, such as realizing spatial characteristics and matching the visual atmosphere, we believe that it is possible to improve presence by expanding its scope through AUX evaluation factors such as the degree of consistency of the overall sound in sound variation situations. Therefore, it is believed that improving the auditory experience of the metaverse environment based on the five components derived in this study can improve the user's sense of reality, and furthermore, it can be used as a basis for creating sound design guidelines and identifying sound development priorities.

5. Conclusion

As the realistic experience of users in a metaverse environment becomes important, we developed an evaluation questionnaire for auditory experience considering presence to improve the presence of the metaverse environment, and derived evaluation components considering auditory presence and AUX through a survey, PCA, and RA. As a result of grouping the auditory evaluation factors through primary expert interviews (Step 1), we found that the tendency to realize a realistic auditory experience was strong in the grouping within an auditory presence evaluation factor, and that the grouping within the AUX evaluation factor had strong acoustic characteristics such as ‘sufficiency of volume size control’, ‘distribution of the sound range ‘ and 'ease of access to sound control'. In the case of grouping between the auditory presence and AUX evaluation factors, we found that the factors covered by the existing AUX evaluation factors, such as ‘auditory cues to help you navigate’, are also covered by auditory presence, confirming that the evaluation area of auditory presence can be expanded.

Through secondary expert interviews (Step 2), we created an evaluation questionnaire that can be easily understood by the public and confirmed that the following should be included in the auditory experience evaluation questionnaire: consideration of auditory sensation such as the transfer of sounds from the physical world to the virtual space, attachment of auditory examples to the questionnaire, and use of intuitive and everyday terms. We also identified that existing broad terms such as three dimensions could be refined to ‘location and distance identification’ and that difficult terms such as 'ease of access to sound control' could be simplified to make them more understandable to the public, such as ‘sounds you want to reduce’. Among the five components extracted from the results of survey, PCA and RA (Step 3), the eigenvalue for 'realistic auditory background' was the highest, so it is a prioritized evaluation component of auditory experience in the metaverse environments. In particular, although AUX evaluation factors such as 'ease of access to sound control' have limitations in improving the sense of presence, negative factors of presence such as 'distraction due to sound' can be improved by utilizing AUX evaluation factors, so it is judged that the sense of presence in the metaverse environment can be improved by improving the auditory sense of presence and AUX evaluation factors according to the composition of the five evaluation components derived in this study.