Comparing 20Hz Steady-State Somatosensory Neural Responses for Contact Vibrotactile and Ultrasound Mid-Air Haptic Stimulation

1. Introduction

Ultrasound mid-air haptics (UMAH) is a technology that enables users to experience the feeling of touch without direct physical contact and has been gaining significant attention due to its versatile applications. Recently, its potential has been explored in various fields and contexts, including human-computer interaction (HCI; e.g., touchless controls and haptic icons), virtual and augmented reality (VR and AR; e.g., mid-air feedback during virtual object interaction) [1,2], automotive user interfaces (e.g., gesture-controlled infotainment) [3,4], and digital advertising (e.g., interactive digital signage) [5].

UMAH generate a feeling of touch using focused ultrasound waves generated by an array of ultrasound transducers. The phases of different ultrasound waves are individually controlled in a way that the acoustic pressure of the different waves converge at a certain location in space (i.e., focal point). At this focal point, mechanoreceptors in the skin (e.g., Pacinian corpuscles, Meissner corpuscles, Ruffini endings, and Merkel cells) are stimulated by the acoustic radiation forces of ultrasound waves [6,7] resulting in perceptible sensations.

While UMAH is a versatile and promising technique, it has its limitations. First, the perceptible strength of UMAH is small. [8] describes a total tactile force of ≈16 mN. This is only a small fraction of the force that physical hardware buttons produce (e.g., 1.5 N). A second important limitation is related to safety. Ultrasound exceeds the range of human hearing, and dangerous levels of ultrasound might go undetected since they do not cause discomfort that warns the user. Within a focal point, pressure can reach more than 140 dB [9,10,11]. To mitigate these potential risks, safety measures are incorporated in available devices limiting the pressure levels of ultrasound. These limitations are based on recommendations from [12,13], which recommend a limit of pressure levels of 100 dB for the general public. These limitations impose constraints on all applications of UMAH. Consequently, it becomes essential to understand how users perceive UMAH under these restricted conditions. To address this, we investigate user experience (UX) through neural markers.

Neural responses to somatosensory stimulation provide an objective way to assess perception under these limitations. Electroencephalography (EEG) is commonly used to measure such responses, as tactile stimulation elicits transient somatosensory evoked potentials (SEPs). [14] demonstrated that UMAH can generate SEPs similar to those produced by vibrotactile stimulation (VTS), suggesting comparable neural processing. More recently, [15] further showed that spatiotemporally modulated UMAH can elicit intensity-dependent transient SEPs. Specifically, they reported that a negative N275 component and a positive P450 component showed larger absolute amplitudes and stronger phase synchronization at higher stimulation intensities. Their behavioral results followed a similar pattern, as participants more reliably discriminated between intensity levels that also produced distinct SEP amplitudes. Together, these findings indicate that EEG can capture perceptually meaningful neural responses to UMAH.

Beyond SEPs, steady-state potentials (i.e., sinusoidal brain responses to sinusoidally modulated stimuli) [16] offer another promising approach due to their 1-on-1 mapping between stimulation frequency and neural oscillation. While steady-state visual and auditory potentials have been widely studied, their somatosensory counterpart (i.e., steady-state somatosensory evoked potentials, SSSEPs) remains less explored. For example, [17] showed that repetitive pneumatic stimulation at 20 Hz elicits SSSEP at the same frequency in the primary somatosensory cortex (S1). Stimulation around the 20 Hz range was found to yield the largest signal-to-noise ratios (SNR) for SSSEP, and matches the preferred frequency range for the S1 [17,18,19].

Building on these findings, we examine whether UMAH can evoke SSSEPs and how these responses reflect UX. Using VTS as a benchmark, we first sought to replicate established SSSEP effects in the S1 using a custom-built VTS device. We then examined whether SSSEPs are also present in stimulus types with reduced stimulation intensity, including matched-intensity VTS and full-intensity UMAH stimulation (maximum intensity under current safety regulations). Matched-intensity refers to matching the VTS intensity to the UMAH stimulus using per-participant perceived intensity matching, allowing us to compare SSSEPs in both stimulus types with subjectively similar intensities. Based on previous SSSEP work with contact stimulation, we hypothesized that full-intensity VTS would elicit robust contralateral SSSEPs, whereas reduced-intensity VTS would result in attenuated responses reflecting differences in stimulus intensity. Given evidence that UMAH can elicit perceptually meaningful transient cortical responses, we additionally examined whether full-intensity UMAH would be sufficient to evoke detectable 20 Hz SSSEPs under the constraints of a commercially available device.

2. Materials and Methods

The materials, and (analyses) scripts of this study can be accessed through OSF.

2.1. Participants

A total of thirty-two participants were recruited through social media groups and mailing lists. Six participants were excluded from the final sample: two participants due to incomplete recordings, three due to technical artifacts that could not be removed during preprocessing, and one for performing the experiment incorrectly. The final sample comprised 26 participants (18 female, 8 male; ${\mathit{M}}_{age}$ = 26). Participants received a monetary compensation of €10 upon completing the experiment.

2.2. Apparatus

EEG data were recorded using the commercially available eego^TM64 amplifier by ANT Neuro using a sampling frequency of 512 Hz with an EEG cap with 65 electrodes (64 EEG channels, 1 EOG channel) following the extended 10-20 system.

VTS was delivered by a custom-built device consisting of a microcontroller (Arduino Uno R3) connected to three linear resonant actuators (LRA; Precision Microdrives C10-100, 10 mm diameter, 4 mm type). The actuators were driven by a dedicated LRA driver board (Adafruit DRV2605L) in real-time playback mode. Stimulation onset was triggered by the experimental computer through USB serial communication. The actuators were positioned at the ventral side of the distal phalanx on the index, middle and ring finger of the right hand. They were held in place using tape to ensure consistent contact.

UMAH stimulation was delivered using the Ultraleap Haptics Development Kit (HDK REC192) which consists of an array of 190 ultrasonic transducers (Murata MA4-0S4S). Hand tracking, necessary to locate the stimulation area, was performed using the integrated Leap Motion Controller 2. The haptic device was controlled through Unity and the sensation was designed in the Sensation Designer software provided by Ultraleap. UMAH stimulation was delivered to the right hand.

2.3. Stimuli

During the experiment, participants received repetitive sensory stimulation with 20 Hz pulses. The VTS sensation was designed as a periodic burst pattern at 20 Hz. Each cycle consisted of a 20 ms vibration burst followed by a 30 ms pause. The VTS amplitude was held constant within each burst and trial. VTS intensity for the matched-intensity stimulus type was subjectively chosen by the participants on a 0–100 scale, this value was binned into one of 52 ordered amplitude steps and mapped to a driver intensity value $i\in [2,127]$ using $i=\mathrm{round}(127·k/52)$, where $k\in \{1,\dots ,52\}$ is the selected step index. UMAH sensations were delivered as a static focal stimulus with perceived pulsing generated via amplitude modulation of output intensity over time. The intensity envelope was defined on a normalized 0–1 (no intensity – maximum intensity) scale and repeated with an approx. 50 ms cycle. The draw frequency was set to 250 Hz, while the envelope controlled the instantaneous intensity.

The stimulation protocol was adapted from [17] and consisted of 2000 ms of 20 Hz stimulation with a 5000 ms inter-stimulus interval. The experiment consisted of three blocks with different stimulation types: 1) VTS at maximum intensity for replication purposes (VTS), 2) VTS matched to the intensity of the UMAH stimulation (matched-intensity VTS) and 3) UMAH stimulation with maximal achievable intensity. During stimulation blocks, all participants were shown an identical nature documentary video that is publicly available online, following the protocol from [17].

2.4. Procedure

Upon arrival, participants filled out an informed consent form. Then, participants were seated in front of the experimental computer. The first part of the experiment consisted of an online questionnaire (Qualtrics) in which participants provided demographic information and answered questions about examples of VTS and UMAH stimulation (these data were, however, not included in the analyses). Following the questionnaire, the EEG sensors were applied to the participant. Next, participants were provided with a 5 minute block (45 trials) of VTS set at maximum intensity. Following this block, participants performed a calibration procedure in which they subjectively matched the VTS intensity to that of the UMAH device. Participants used both hands (left: VTS, right: UMAH) simultaneously to gauge the intensity of both stimulation types, while the experimenter altered the VTS intensity (scale 0–100) based on feedback from the participants until they reported the intensity to match. This value determined the matched-intensity VTS. After completing the calibration procedure, the next block was presented. To control for potential order effects, even-numbered participants were presented with the full-intensity UMAH stimulation block first followed by the matched-intensity VTS block, and vice versa for odd-numbered participants. Each block consisted of 174 trials. Upon completing the experiment, participants provided information for monetary compensation. During the UMAH condition, the fingers of the participant were held in place at ≈22 cm using a custom 3D-printed hand support, see Figure 1.

2.5. Data Analyses

EEG preprocessing was performed using Python 3.12 and statistical analyses using R 4.5.2. The raw EEG data were bandpass-filtered with cut-off frequencies of 0.1 and 40 Hz and re-referenced to the average signal. Due to the generally high SNR in SSSEP, preprocessing was limited to the exclusion of the T7, T8, M1 and M2 electrodes due to consistent noise and the exclusion of epochs that showed excessive noise (e.g., movement artifacts).

The region of interest (ROI) was defined as the electrodes close to the S1 region contralateral to the stimulated hand as described in [20] (ROI: C1, C3, C5, CP1, CP3, CP5, P1, P3, P5, P7, TP7). Epochs were created by extracting the signal 500 ms pre- and 7000 ms post stimulus onset. PSD was estimated for each condition using Welch’s method (Hann window, 0 overlap) over 0.5–2 s and 1–50 Hz. PSD describes how signal power is distributed across frequencies, it includes both stimulus-locked activity and background noise. Using this PSD, we can assess the strength of the SSSEP response relative to background noise by calculating the SNR. SNR is calculated by dividing the power at the target frequency (20 Hz) with the average power of neighboring frequencies, this reflects how strongly the stimulation frequency stands out from surrounding noise. Baseline (no stimulation) measures were obtained by computing the PSD during the no-stimulation segment within the inter-stimulus interval (4000 – 5500 ms post-stimulus onset). SNR and PSD channel averages across trials for the 20 Hz frequency were calculated for each participant and condition. To account for possible inconsistencies in EEG-cap placement across participants, the three electrodes within the ROI with the highest PSD for 20 Hz relative to surrounding frequencies were selected for each participant separately. To avoid double-dipping, electrode selection was performed using the full-intensity VTS condition, and the selected electrodes were then held fixed for all other stimulus types. The average SNR and PSD (log-transformed) for these electrodes (and their contralateral counterparts) were used for further interpretation and statistical analyses.

To investigate our hypotheses, two linear mixed-effect models were conducted with SNR and PSD as dependent variables. The fixed effects included the four stimulus types (Baseline, VTS, matched-intensity VTS and UMAH stimulation) and electrode laterality (ipsilateral vs. contralateral). A random intercept for participants was included to account for individual differences. Post-hoc pairwise comparisons were performed using estimated marginal means with Tukey adjustments for multiple comparisons. As an additional sanity check, the Pearson correlation between the value of the matched-intensity VTS intensity and the SNR/PSD for the matched-intensity VTS stimulus type was calculated.

3. Results

3.1. Signal-to-Noise Ratio (SNR)

Mean SNR values were analyzed using a linear mixed-effects model with fixed effects of stimulus type, hemisphere, and their interaction, and a random intercept for participant. The model revealed a significant main effect of stimulus type F(3,175) = 3.93, p = .01, but no main effect of laterality F(1, 175) = 0.002, p = .962. However, a significant stimulus type x laterality interaction was found F(3, 175) = 43.75, p < .001, indicating that laterality differences depended on stimulation type (Figure 2).

Post-hoc comparisons showed that for the VTS and matched-intensity VTS stimulus types, mean SNR was substantially higher contralateral compared to ipsilateral, p < 001 and p = 001, respectively. But no laterality differences for UMAH or baseline measures. Stimulus type contrasts showed that, contralateral to stimulation, SNR was highest in VTS, significantly higher than matched-intensity VTS, UMAH and baseline (all p < .001). SNR in matched-intensity VTS was significantly higher than UMAH and baseline (p < .001 and .002, respectively), while UMAH and baseline did not differ from each other (p = .995). Ipsilateral to stimulation, SNR was found to be larger only for VTS compared to baseline and UMAH (p = .033 and p = .01, respectively). A moderate positive correlation was found between SNR for the matched-intensity VTS stimulus type and the calibrated intensity of the VTS device (contralateral ROI; Pearson, r = 0.39, p = .049).

3.2. Power Spectral Density (PSD)

Mean PSD values were analyzed using an equivalent linear mixed-effects model. This revealed a significant main effect of stimulus type but no main effect of laterality; F(3, 175) = 5.63, p = .001, F(1, 175) = 0.01, p = .914, respectively. A significant interaction effect was found for stimulus type and laterality with F(3, 175) = 37.88, p < .001 (Fig. Figure 2).

Laterality contrasts revealed that the PSD in the contralateral ROI was significantly higher than the ipsilateral ROI for VTS (p < .001), and, to a smaller extent, for matched-intensity VTS, p = .049. No significant laterality differences were found for UMAH or baseline. For stimulus types, contralateral PSD was highest for full-intensity VTS, exceeding all other conditions (all p < .001). Matched-intensity VTS also yielded higher contralateral PSD than baseline and UMAH (p = .001 and p < .001), while baseline and UMAH did not differ. Ipsilateral, PSD was higher for full-intensity VTS compared to baseline and UMAH (p = .006 and p = .001), with no other differences reaching significance. A positive correlation (contralateral ROI, Pearson, r = 0.39, p = .048) was found between PSD for matched-intensity VTS in the contralateral ROI and the absolute intensity of the VTS device (0–100).

4. Discussion

This study investigated whether UMAH, delivered at the maximum intensity under current intensity constraints, elicits SSSEPs comparable to those produced by VTS. Building on prior work demonstrating reliable SSSEPs for periodic tactile stimulation [17,19], we aimed to assess the feasibility of using SSSEP-based neural markers as an objective complement to self-reported UX measures in the evaluation of UMAH interfaces. Three main findings emerge from the results.

First, full-intensity VTS elicited a clear and robust SSSEP at 20 Hz, expressed as increased PSD and SNR over contralateral somatosensory regions. This replicates earlier findings using pneumatic and VTS [17,19] and confirms the suitability of our experimental setup and analysis pipeline for detecting SSSEPs. The spatial distribution of the effect, centered over contralateral central-parietal electrodes, is consistent with activation of primary somatosensory cortex (S1) as reported in the literature [18,20].

Second, the positive correlation between stimulation intensity and SSSEP magnitude in the matched-intensity VTS condition aligns with previous evidence that SSSEPs scale with stimulus amplitude [19]. Together, these findings support the validity of SSSEP measures as sensitive neural markers of tactile stimulus strength.

Lastly, full-intensity UMAH stimulation did not elicit statistically significant SSSEPs relative to baseline, despite being delivered at the maximum intensity allowed by the device. Both PSD- and SNR-based analyses showed similar results and led us to conclude that neural responses to UMAH were indistinguishable from no-stimulation periods.

There are several explanations that may account for the absence of SSSEPs for the UMAH stimulus type. First, the low forces generated by UMAH as described by [8]. In their study, an output force of 16 mN was measured which is orders of magnitude lower than forces from typical vibrotactors. Similarly, [21] emphasize that UMAH at maximum output produces weaker skin displacements than other conventional mechanical stimulators. Therefore, even full intensity UMAH stimulation, under the current safety constraints, provides only a small mechanical input which may not be sufficient to induce a detectable SSSEP. This may additionally explain why an SSSEP was found for the matched-intensity stimulus type and why this was not present for UMAH.

Second, UMAH stimulation differs from VTS not only in amplitude but also in mechanical coupling. Vibrotactile actuators produce direct skin indentation and shear forces, whereas UMAH produces forces through acoustic radiation pressure acting on the skin surface [6]. These different modes of stimulation likely engage mechanoreceptor populations differently, which may affect the ability to entrain oscillatory cortical responses even when perception is preserved [21].

Third, differences in spatial and temporal characteristics of stimulation may reduce neural entrainment. In contrast to VTS, the focal point generated by UMAH can be spatially diffuse and sensitive to small hand movements, potentially reducing temporal precision and mechanoreceptor synchronization. Minor tracking jitter or movements of the hand could further degrade consistent phase-locking between stimulus and neural response, even if the stimulus remains perceptually detectable.

Importantly, our finding does not mean that UMAH is imperceptible or that it cannot elicit cortical responses. Other research has previously shown that UMAH pulses can evoke SEPs [14,15]. These findings are compatible with the present results when considering the type of neural response that was measured. Both studies focused on transient responses to discrete mid-air haptic events, whereas the present study examined whether a continuous 20 Hz stimulation pattern produced sustained frequency-specific entrainment. UMAH may therefore be sufficient to trigger discrete cortical responses related to stimulus detection, intensity, or salience, while still being insufficient to drive a robust SSSEP at 20 Hz under the present stimulation and analysis parameters. Alternatively, very small UMAH-induced SSSEPs may have been present but were not captured by the current SNR and PSD approach.

From an applied perspective, these findings have important implications for the evaluation of UMAH interfaces. While SSSEPs appear to be a robust and sensitive measure for VTS, our approach currently seems unsuitable as an independent, objective evaluation tool for UMAH UX under the output constraints of commercially available devices. This does not imply that UMAH lacks perceptual or experiential value, but rather that its neural signature may be too weak, too variable, or too transient to be captured using the current methods.

Several limitations should be considered. First, stimulation was restricted to a single frequency (20 Hz), chosen based on prior SSSEP literature. It remains possible that UMAH may more effectively entrain neural responses at other modulation frequencies [21]. Second, although intensity matching was carefully implemented, subjective equivalence does not guarantee physiological equivalence. Future work could combine psychophysical thresholds, force measurements, and neurophysiological outcomes more directly. Additionally, future research could explore alternative neural markers for UMAH, such as transient SEPs, time–frequency analyses, or multimodal integration paradigms, rather than relying on SSSEPs.

In conclusion, while VTS reliably elicited SSSEPs, UMAH did not produce detectable SSSEPs when delivered at the maximum intensity permitted by current commercial implementations. These findings suggest that, under existing constraints, UMAH may be insufficient to drive SSSEPs in the somatosensory cortex. This places important limits on the independent use of steady-state based methodologies for evaluating UX of UMAH interfaces. Further research into alternative objective measures of UMAH experience is recommended.