TouchView: Mid-Air Touch on Zoomable 2D View for Distant Freehand Selection on Virtual Reality User Interface

1. Introduction

Selection is one of the basic user interactions in virtual reality (VR) and 3D user interfaces (UIs) [1]. Poor selection techniques may severely damage the VR interaction as the selection is a prerequisite to the manipulation of virtual objects in general. Virtual pointing techniques (a.k.a. raycasting) are considered the most popular among 3D selection techniques due to their simplicity and generality [2,3,4,5], resulting in better selection performance compared to substitute 3D selection techniques [1].

However, the selection of small or distant objects using raycasting has been known to be difficult. Raycasting is sensitive to natural hand tremors as the selection occurs in mid-air while no physical support is given to hands [6], which leads to high error rates when selecting smaller targets [7,8,9,10,11]. A change in the tool orientation happens by the action of selection trigger (i.e. the Heisenberg effect of spatial interaction; Bowman et al., 2001) also induces target misses, accounting for nearly 30% of the overall errors [13]. When the tracking noise becomes substantial such as in vision-tracked freehand input, the negative influence of positional and rotational jitter from the tracked input device on the selection accuracy can become non-negligible [14,15].

Many studies have proposed new ways to improve 3D selection. Some early research introduced volume-based pointing to enlarge the selection area itself [16,17], although it required disambiguation when multiple objects were inside the area for selection. Disambiguation could be done progressively by allowing users to specify the target [18,19,20,21] or heuristically by assigning scores to potential targets [11,22,23,24,25,26,27]. Another approach to improve 3D selection was to dynamically adjust the control-display ratio of the cursor depending on the precision requirement, which could be done either manually [28,29] or automatically based on the user’s hand velocity [30,31,32]. Attempts to build and apply the model to correct systematic mid-air pointing offset that is caused by humans’ limited inherent pointing accuracy [33] were also made [34,35].

Some techniques have used an indirect approach where the user manipulates virtual objects by interacting with their copied representation, although it has been typically used to enhance the manipulation instead of the selection of objects. World-in-Miniature [36] and Voodoo Dolls [37] were the first few techniques that introduced the use of proxies for object manipulation. Indirect proxy techniques can allow users to not only observe the world with different views thus better understanding the layout of objects [38] but also interact directly through hands hence avoiding body distortion and disownership [39]. A recent advance of this approach covers interaction with proxies of spaces instead of objects [40], proxies of multiple virtual hands—Ninja Hands [41], or even a handle connected by a tether to an out-of-reach object to decrease the distance to targets [42].

Adaptations were made to benefit object selection with proxies by transforming the 3D environments into 2D representations, as studies have shown 2D selection can outperform 3D selection [43,44,45,46,47]. An early example of leveraging the benefits of 2D interaction in 3D can be found in the Through-the-lens metaphor [48], where the use of an additional viewport of the virtual world in a form of a floating virtual panel was proposed. Follow-up works have flattened targets and the surrounding environment onto various mediators including a small virtual screen [49], a cylindrical virtual window displaying a 360° panoramic image [50], and a virtual viewport that reflects like a mirror [51]. The use of a tangible viewport utilizing a physical proxy has been also explored for selecting objects in VR [52] and handheld augmented reality [53,54,55,56].

Meanwhile, recent advances in computer vision technology have brought vision-based hand tracking using the built-in cameras of consumer VR headsets like the Meta Quest series to a point where it can actually be used seamlessly for interactions like object selection. This has greatly increased the accessibility of freehand mid-air interaction in VR without the need for gloves, markers, or other attachments. Not only do freehand mid-air inputs benefit from higher naturalness, intuitiveness, and unobtrusiveness, which are important values in VR, but they also reduce the cost and portability burden of ubiquitous computing by eliminating the need for additional wearable sensors or devices. In addition, when coupled with well-designed interaction techniques, there is great potential in terms of usability, as hands are finely-tuned devices that provide strength, precision, and expressiveness in our real-world actions [1].

Research on vision-based freehand interaction in VR is still scarce. Of the 48 papers included in a review article on evaluation of VR object selection and manipulation, only 8 (16.7%) investigated bare-hand interaction [57]. Of these, the number of studies utilizing vision-based tracking would be even lower, although the exact number was unreported. In VR ray-based pointing studies, VR controllers are used [58,59], or in the case of freehand interaction, an optical motion capture system is used to track markers attached to the body parts including hand [34,35] to minimize the effect of tracking noise thereby increase internal validity in understanding human pointing behavior. However, we would argue studies that include the tracking noise of vision-based hand tracking, are also valuable in terms of external validity as tracking noise is unavoidable in real-world applications, especially in the current circumstances where comparative studies on the market-dominating Hybrid Ray technique, which is the Meta’s implementation of freehand raycasting [60], are lacking.

The study by Kim & Xiong [61] was the first study that attempted to reveal the full potential of the Through-the-lens metaphor with the advantages of the direct touch interaction and optimization of the interaction space in the freehand VR selection task. They proposed ViewfinderVR, a technique that allows users to configure the interaction space projected onto a virtual panel in reach and to select objects inside the panel through direct touch interaction. They compared ViewfinderVR with Hybrid Ray, Meta’s implementation of freehand raycasting [60] on a 2D Fitts’ law-based test and found ViewfinderVR induced significantly better performance and lower perceived workload. Another modern adaptation of the Through-the-lens metaphor—although not concerning freehand interaction—is the work by Berwaldt et al. [62]. They introduced a method for enhancing VR interaction and navigation by employing multiple virtual cameras that project their views onto windows encircling the user, enabling users to simultaneously monitor, navigate, and engage with multiple occluded locations.

However, earlier works required a considerable amount of extra time and effort to configure the panel to improve the object selection. This indicates earlier techniques are not appropriate for scenarios where the view needs to be changed frequently since the user had to reconfigure the panel whenever the target beyond the panel view needs to be selected. To solve this problem, we propose the TouchView technique. A virtual panel with the Through-the-lens metaphor is used, but the panel is always coupled with the user’s head movement and large enough to cover the whole view thereby significantly reducing the time needed for configuration. We conducted a user study to measure its performance against Hybrid Ray to understand the benefits and limitations of the proposed technique.

2. Techniques

2.1. Hybrid Ray

If any method of cursor movement that establishes the position of the cursor by a ray's intersection with a surface or object in the distance is defined as generic raycasting, then there are several variants: laser pointing, arrow pointing, image-plane pointing, and fixed-origin pointing [63]. Laser pointing specifies the ray directly by the position and direction of a physical device. In contrast, arrow pointing works similarly to laser pointing but the use of the laser pointer is confined to be aligned with the user’s eye. On the other hand, image-plane pointing allows the ray to be determined through the users’ eye location and another controllable point in space, while fixed-origin pointing relaxes image-plane pointing by directing one of the two points of the ray onto any fixed location rather than the eye.

In this study, Meta’s implementation of freehand raycasting called Hybrid Ray [60] was used, because it has been one of the most commonly used freehand ray-based pointing techniques as the default selection method embedded in the current market-dominating VR headset that occupied 72% of the market as of Q4 2023 [64]. Hybrid Ray is a variant of the fixed-origin pointing that uses the secondary position on the body to anchor the ray’s direction to stabilize the ray to minimize the negative influence of tracking jitter when the vision-tracked freehand input is used. It is described that the optimal point of origin for this secondary position varies between shoulder and hip depending on whether you're standing or sitting, but Meta has not disclosed how exactly the secondary position is determined.

2.2. TouchView

Figure 1 illustrates the working mechanism of our proposed technique, TouchView. TouchView was designed as a technique that may be optimal for interacting with 2D UI elements with 3D planar panels, which may be a typical form of VR UIs. TouchView aims to aid the selection of distant objects by utilizing a virtual panel that shows the 3D environment behind it, taking advantage of the Through-the-lens metaphor [48]. The virtual panel located in front of the user’s head is large enough to cover the whole view and follows the user’s head movement, thereby manipulation of the panel becomes unnecessary. The view reflected on the panel can be magnified by moving the hand up or down while pinching with the index finger, and magnification of the view can be initialized by turning their hand over and looking at the palm.

Users can choose distant targets by selecting the targets reflected inside the panel view instead of selecting the actual targets. The panel cursor is coupled with the control cursor which moves on the control plane located in front of the view origin. The cursor for selection is rendered on the intersection point between the target object and the ray from the view origin to the control cursor. The panel cursor is located at the intersection point between the panel and the ray originating from the index fingertip and directed perpendicular to the panel. The selection occurs when the index fingertip touches the panel surface. View magnification and object selection can be done by both hands.

In this study, the panel was perpendicular to the front direction of the head and located 0.4m away and 0.13m below the head. This setting was determined based on various circumstances such as field of view, hand tracking range of the VR headset, and location to comfortably reach the panel [65,66,67]. The virtual hands were kept semi-transparent during the whole experiment to prevent the occlusion of the object behind the hands (Figure 1).

3. User Study and Data Analysis Methods

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

3.1. Participants

Twenty-three Korean young adults (16 males and 7 females) with a mean age of 24.3 (SD=3.3) and with normal or corrected to normal vision, which satisfies the recommended minimum number of participants for evaluation of object selection in VR [57], participated in the experiment. Twenty-two participants were right-handed and 1 participant was left-handed. Twenty participants have experienced any kind of headset-based VR applications before this experiment, but 14 among them used the VR headset no more than once a year, showing the majority of participants were light VR users. Eight participants reported that they experienced mid-air interaction with the tracked virtual hand in VR. All participants gave consent for the experiment protocol approved by the University Institutional Review Board (IRB NO.: KH2021-009).

3.2. Experimental Settings

The participants were equipped with the Oculus Quest (resolution: 1440×1600 per eye; refresh rate: 72 Hz) VR headset. The headset was connected to the PC with an Intel Core i7-7700 processor running at 3.6 GHz, 16 GB of RAM, and an NVIDIA GeForce GTX 1080 Ti GPU, running Windows 10 through Oculus Link via a compatible USB 3.0 cable. Participants conducted the given task in a room while standing in front of a black screen fence (Figure 2). Hands were tracked in real-time by the four fisheye monochrome cameras embedded in the Oculus Quest headset [68], so the background was covered by a black screen fence to prevent any potential deterioration of tracking performance. The distance between the participant and the screen fence was 0.9m, and no physical interruptions were caused by the screen fence during the whole experiment. The virtual environment used for the experiment was developed using Unity 2019.4.15f1.

3.3. Experimental Design and Procedure

First, participants filled in a pre-test questionnaire asking about their demographic information and prior experience with VR. Then, they put on the VR headset and performed a multitarget selection task [69]. In this task, 12 blue circular targets and 12 magenta circular distractors were placed randomly using a Poisson disk distribution on a 5m away rectangular plane with a width × height of 11.9175m × 4.6631m, which corresponds to a visual angle of 100° and 50° (Figure 3). Here, targets are arranged on a planar space to simulate the selection on a UI. Participants were asked to select all targets as accurately and quickly as possible while prioritizing accuracy over speed. Whenever the selection occurred, the participants were informed with a click sound when the target was hit, and a beep sound when the target was missed. The selected target changed its color to yellow. A sequence of trials consisted of 12 target selections, and five sequences were conducted for each of the experimental conditions. The experiment followed a within-subject 2 × 3 full-factorial design with two factors: technique (2 levels: Hybrid Ray and TouchView, see Figure 4-a) and target size (3 levels: 7.5°, 4.5°, and 1.5°, see Figure 4-b), where experimental conditions were presented in randomized order. The targets of 0.6554m, 0.3929m, and 0.1309m in diameter were used to represent large, middle, and small sizes, which were equal to the visual angle of 7.5°, 4.5°, and 1.5° from the distance of 5m, respectively. It is worth noting that the distance from the user to the target ranged from 5 to 8.1205m, so the perceived visual angle of actual targets ranged 4.62– 7.50°, 2.77– 4.50°, and 0.92– 1.50°, accordingly. Target visual angle and distance were determined based on the settings in previous VR target selection studies [7,47,70,71]. Participants were asked to conduct the first sequence with their dominant hand only, and the second sequence with both hands, then they could freely choose to conduct the rest three sequences unimanually or bimanually based on their prior attempts. For TouchView, participants were allowed to change the magnification of the view at any time before or during the task. After all sequences in each test condition were finished, participants gave ratings to NASA-TLX [72] and then 30 seconds of rest, determined by a series of internal pilot tests, were given before moving on to the next test condition. Before the experiment ends, participants were asked to select the preferred technique per target size with reasons for their choice. It should be noted that TouchView is primarily designed for use in VR UIs, where a typical scenario involves a large 3D planar panel with 2D UI elements floating in front of the user. Therefore, the experimental task in this study does not cover the varying depths of targets and cluttered environments that are often encountered in a general VR environment.

3.4. Data Analysis

Out of five sequences, the first two sequences were considered as practice and thus excluded from the analysis, and the data was recorded from when the participant selected the first target in each sequence. Two performance measures: task completion time and miss rate (the number of misses divided by the total number of selections) and seven NASA-TLX ratings for perceived workload: mental demand, physical demand, temporal demand, performance, effort, frustration, and weighted rating were collected and arranged for repeated-measures analysis of variance (RM-ANOVA) where the variation from participants was blocked. In addition, behavioral measures: movement of the dominant/nondominant hand and head, and target visual angle along with the distribution of selections made by each hand were calculated and analyzed to have a better understanding of the results. Hand movements were defined as the length of movement trajectory of the tracked dominant/nondominant hands. Target visual angle was defined as the visual angle of the target calculated based on the distance from the head origin to the original target or the adjusted target (TouchView only). The Shapiro-Wilk test for normality and inspection of Q-Q plots found no measures were severely deviated from the normal distribution. The degree of freedom was corrected with Greenhouse-Geisser correction if the p-value of Mauchly’s test was equal to or less than 0.05. Matlab R2019a, R 4.2.2, and “rstatix” R package were used to conduct all data processing and statistical analyses at a significance level of 0.05, and the effect size in terms of generalized eta-squared (${\eta }_G^2)$ [73] was further calculated to check practical significance.

4. Results

We highlight a subset of significant results related to the technique for better clarity. The entire descriptive statistics and RM-ANOVA results are presented in Appendix A.

4.1. Performance

Figure 5 shows performance measures by technique and target size. A significant main effect of the technique was observed in task completion time ($F_{\mathrm{1,22}}$ = 45.19, p < 0.001, ${\eta }_G^2$ = 0.290). The task completion time was shorter at TouchView (M = 19.3s, SD = 10.5) than at Hybrid Ray (M = 29.0s, SD = 16.8), but no significant difference in miss rate ($F_{\mathrm{1,22}}$ = 0.66, p = 0.425, ${\eta }_G^2$ = 0.005) between Hybrid Ray (M = 16.8%, SD = 12.6) and TouchView (M = 15.8%, SD = 10.0) was found. A significant interaction effect of technique × target size was found for task completion time ($F_{\mathrm{1.2,26.51}}$ = 9.88, p = 0.003, ${\eta }_G^2$ = 0.082) but not for miss rate ($F_{\mathrm{2,44}}$ = 3.04, p = 0.058, ${\eta }_G^2$ = 0.025). According to the Bonferroni post hoc analyses, participants completed the task faster with TouchView in all target size conditions. Task completion time and miss rate with small targets increased compared to bigger targets in both techniques, but Hybrid Ray showed a larger increase compared to TouchView.

4.2. Perceived Workload

Figure 6 indicates NASA-TLX workload ratings by technique and target size. A significant main effect of the technique was observed on all perceived workload measures: mental demand ($F_{\mathrm{1,22}}$ = 24.24, p < 0.001, ${\eta }_G^2$ = 0.046), physical demand ($F_{\mathrm{1,22}}$ = 6.45, p = 0.019, ${\eta }_G^2$ = 0.016), temporal demand ($F_{\mathrm{1,22}}$ = 20.13, p < 0.001, ${\eta }_G^2$ = 0.049), performance ($F_{\mathrm{1,22}}$ = 12.74, p = 0.002, ${\eta }_G^2$ = 0.082), effort ($F_{\mathrm{1,22}}$ = 15.01, p < 0.001, ${\eta }_G^2$ = 0.054), frustration ($F_{\mathrm{1,22}}$ = 10.78, p = 0.003, ${\eta }_G^2$ = 0.030), and weighted rating ($F_{\mathrm{1,22}}$ = 21.74, p < 0.001, ${\eta }_G^2$ = 0.069). All NASA-TLX workload ratings had similar results that TouchView was lower than Hybrid Ray, where the weighted rating of TouchView (M = 27.9, SD = 22.4) was lower than Hybrid Ray (M = 39.0, SD = 28.3) by 11.1. A significant interaction effect of technique × target size was found in mental demand ($F_{\mathrm{2,44}}$ = 6.29, p = 0.004, ${\eta }_G^2$ = 0.017), physical demand ($F_{\mathrm{2,44}}$ = 5.17, p = 0.010, ${\eta }_G^2$ = 0.015), frustration ($F_{\mathrm{2,44}}$ = 6.66, p = 0.003, ${\eta }_G^2$ = 0.021), and weighted rating ($F_{\mathrm{2,44}}$ = 4.47, p = 0.017, ${\eta }_G^2$ = 0.013). A similar tendency was found across all workload ratings that the rating of Hybrid Ray was generally higher yet occasionally similar compared to TouchView at large and middle-sized targets but the discrepancy became more obvious at small targets.

4.3. User Behavior and Preference

There was a significant main effect of the technique on all user behavior measures (Figure 6-8): movement of the dominant hand ($F_{1,22}$ = 71.68, p < 0.001, ${\eta }_G^2$ = 0.484), nondominant hand ($F_{1,22}$ = 23.12, p < 0.001, ${\eta }_G^2$ = 0.215) and head ($F_{1,22}$ = 7.60, p = 0.012, ${\eta }_G^2$ = 0.042), and target visual angle ($F_{\mathrm{1.01,22.14}}$ = 10.54, p = 0.004, ${\eta }_G^2$ = 0.132). The hand and head movements of TouchView (Dominant hand: M = 1.89m, SD = 0.74; Nondominant hand: M = 0.87m, SD = 0.66; Head: M = 0.43m, SD = 0.17) were significantly longer than Hybrid Ray (Dominant hand: M = 0.86m, SD = 0.25; Nondominant hand: M = 0.39m, SD = 0.16; Head: M = 0.38m, SD = 0.16). For target visual angle, the adjusted target visual angle of TouchView (M = 4.22°, SD = 1.83) was larger compared to the original target visual angles of Hybrid Ray (M = 3.68°, SD = 2.01) and TouchView (M = 3.70°, SD = 2.03), where no significant difference was found between those two.

A significant interaction effect of technique × target size was found for the movement of the nondominant hand ($F_{\mathrm{2,44}}$ = 11.43, p < 0.001, ${\eta }_G^2$ = 0.084) and target visual angle ($F_{\mathrm{1.86,41.03}}$ = 11.40, p < 0.001, ${\eta }_G^2$ = 0.156). Movement of the nondominant hand of TouchView tended to decrease with the target size, showing no significant difference with Hybrid Ray at small targets (Figure 7). Figure 8 shows more detailed information about the usage of dominant and nondominant hands, which is in line with the hand movement. The percentage of trials using only the dominant hand almost doubled when Hybrid Ray was used compared to TouchView at large and middle targets, but not at small targets. Likewise, selections with the nondominant hand were made more frequently with TouchView at large and middle targets but the gap became less noticeable at small targets. The original target visual angle was consistent across both techniques, whereas the adjusted target visual angle was significantly larger than the original one at small targets (Figure 9). It is worthy to note that some participants intentionally reduced the target visual angle for selecting large and middle targets.

A larger number of participants preferred TouchView over Hybrid Ray in all target size conditions (Figure 10). However, preference differed depending on the target size, showing a difference in the percentage of 21%, 48%, and 74% of all participants for large, middle, and small targets, respectively.

5. Discussion

5.1. Main Findings

The major limitation of ViewfinderVR [61] was that it required a considerable amount of extra time and effort to configure the panel before object selection. This made ViewfinderVR not practically applicable for cases where the view needs to be changed frequently since the user had to reconfigure the panel whenever the target beyond the panel view should be selected. TouchView simplified the configuration procedure by using the head-coupled panel large enough to cover the whole view, making manipulation of the panel unnecessary while the benefits could be maintained by allowing magnification of the panel view. The average duration participants used to magnify the view while completing all three sequences was only 3.3 seconds (SD = 3.8), which is 84.4% shorter than the duration required for panel configuration of ViewfinderVR (M = 21.1s, SD = 11.8).

As the fundamental property of direct touch interaction transferred from ViewfinderVR to TouchView, results similar to a certain extent were expected. Direct touch interaction was known to require less number of mental resources compared to indirect methods for an intuitive way of interacting with objects [8,74,75,76]. Many participants who preferred TouchView over Hybrid Ray commented, “P4, P5, P6, P9, P12, P13, P17, P21, P22: The way of interaction of TouchView was more direct and intuitive thus less demanding.” Direct input techniques have no intermediary to directly touch objects, unlike indirect input techniques that require additional cognitive processing of spatial translation between the body movement and the cursor movement [77]. Accordingly, the weighted workload rating of TouchView was 28.0–29.1% lower than Hybrid Ray. Although TouchView required users to move their upper body closer to targets to select them thereby resulting in longer movements of hands and head (Figure 7), participants could hold up their arms for a shorter period for performing the same amount of tasks when using TouchView. As physical demand is heavily influenced by the time that the upper arm muscles remain active for mid-air interaction [78,79,80], the rating of physical demand of TouchView turned out to be 6.0–24.5% lower than Hybrid Ray.

The directedness of interaction could further enable quicker task performance. Studies have found direct touch input allowed users to perform the task faster and more accurately with higher satisfaction compared to indirect mouse input [69,81,82]. In this study, TouchView significantly outperformed Hybrid Ray in task completion time, lowering by 30.6–37.2% compared to Hybrid Ray depending on the target condition. It is worth noting that the duration used for magnification of the view is included in the task completion time of TouchView since participants were allowed to magnify the view at any time during the task if needed.

Bimanual interaction might also take part in improving the selection speed of TouchView, as proven in research that speed advantages of bimanual touch over a pair of mice [69,83]. In this study, participants chose to select 16.6% more large targets and 19.4% more middle targets with the nondominant hand when using TouchView compared to when using Hybrid Ray, whereas they had difficulties in selecting small targets with their nondominant hand and thereby chose to use the nondominant hand in only 4.3% more trials (Figure 8-a). Accuracy demands of the task are known to influence both performance and learning of unimanual and bimanual motor sequences [84,85]. In other words, the bimanual advantage is present for tasks that have relatively low accuracy requirements. As participants were asked to perform the task as accurately and quickly as possible, they likely chose to utilize bimanual advantage for large and middle targets that require low accuracy but not for small targets with high accuracy demands to avoid the bimanual disadvantage. This result is in line with previous studies that have reported a bimanual advantage and disadvantage associated with the task difficulty in selection tasks [84,85,86,87]. Nevertheless, the majority of targets were selected with the dominant hand (Figure 8-b).

Likewise, it was anticipated that TouchView would outperform Hybrid Ray in terms of miss rate, as some participants also pointed out, “P8, P11, P15, P16, P23: An option to magnify the view was useful in improving accuracy for selecting smaller targets as the cursor jitter of Hybrid Ray was more sensitive to hand tremors and occurrence of the pinch gesture for selection.” However, improvements in accuracy were minor in comparison with ViewfinderVR, as the miss rate of TouchView was not significantly lower than Hybrid Ray at all target sizes. This is because the view magnification of TouchView is inevitably accompanied by a decrease in the field of vision and an increase in the rate of change of the view, as the view was always coupled with the head movement. Consequently, excessive magnification could place more challenges on searching for targets out of the visual field and keeping the targets stationary within the view. Therefore, participants chose to adjust the target visual angle to be similar for large targets and larger by a small margin (8.6%) for middle targets to minimize such downsides, but by a much bigger margin (109.0%) for small targets (Figure 9) where benefits of magnification could surpass disbenefits. This indicates TouchView is not a successor that can fully replace ViewfinderVR, but a complement that can be used together with ViewfinderVR to cover the usage with frequent view shifts.

5.2. Limitations and Future Work

The proposed technique has some limitations. First, as TouchView projects the 3D environment onto a 2D screen, a slight distortion occurs in the view. Although distortion may not reach the level that significantly degrades the selection performance, it may harm the naturalness of the interaction since users can notice that the view shifts a bit differently from what they would expect when the users rotate their heads in the real world. Second, controlling the cursor of TouchView can be demanding to some users as a few participants complained, “P3, P7, P20: It required me to fully concentrate on keeping the cursor inside the target while thrusting the finger.” This might be because the indication of the cursor location (i.e., the intersection point between the panel and the ray originating from the index fingertip and directed perpendicular to the panel) was less obvious to certain users. Future work can explore ways to further improve the proposed technique.

In addition, it would be valuable to investigate whether the findings can be generalized to other options for the implementation of raycasting, other types of the VR environment, and other VR headsets with different specifications. While the comparison with Meta's Hybrid Ray can be very valuable in a practical sense, it is lacking in terms of understanding human pointing behavior and exploring better alternatives, thereby more diverse ray-based pointing methods such as finger, eye-finger, and head-based raycasting [35,88] can be further compared in future work. The experimental task used in this study was tailored for VR UIs, so it would be worthy to test the technique in a variety of VR environments in future work. For instance, TouchView omits the depth information thus occluded objects cannot be selected similar to the case of Hybrid Ray, thus expected to suffer with occluded targets unlike techniques deliberately designed for such scenarios [89,90,91]. Finally regarding the headset specifications, the accuracy of hand tracking might affect the result, although it should be noted that the VR headset used in this study, Oculus Quest reported a comparable hand tracking accuracy in touch tasks compared to another commonly used hand tracking sensor, Leap Motion [92].

6. Conclusions

We have proposed a new technique TouchView for the improved selection of distant objects in VR UI that can accommodate view shifts. It allows users to configure the interaction space projected onto a virtual panel in reach and to select objects inside the panel through touch interaction, but the virtual panel is large enough to cover the whole view and follows the user’s head movement, thereby manipulating the panel becomes unnecessary. A follow-up user study was conducted to evaluate and compare the proposed technique with Hybrid Ray through a multitarget selection test. Experimental results showed that for all investigated target sizes, TouchView induced shorter movement time and lower perceived workload compared to Hybrid Ray, and has the advantages of the directness of interaction and target magnification inside the view. Such benefits of TouchView enabled easier bimanual selection hence contributing to shortening task completion time. The findings prove that users can benefit from our proposed technique for distant selection in VR UI due to direct mid-air touch interaction and customization of the interaction space.