PiEEG XR: A WebXR Brain-Computer Interface Platform for Neural-Adaptive Avatar Control in Mixed Reality

1. Introduction

Immersive virtual and mixed reality systems have reached a level of fidelity sufficient to sustain meaningful social, educational, and therapeutic applications (Liberatore et al. 2021). Head-mounted displays (HMDs) such as the Meta Quest series now support six-degrees-of-freedom tracking, full-colour video passthrough, and sub-millisecond controller latency. Despite this richness of sensory and motor I/O, one bandwidth-rich channel remains almost entirely unexploited: the electrophysiological state of the user.

A user's avatar in current social VR platforms is an animated proxy whose facial expressions are driven exclusively by tracked head and hand motion. The avatar does not know whether the user is deeply focused, mildly anxious, or experiencing flow. It cannot smile spontaneously when the user feels joy, nor modulate the luminance of its aura when attention peaks. This absence of affective mirroring represents a fundamental limitation for applications in mental health monitoring, neurofeedback training, empathy research, and next-generation social presence.

Brain-computer interfaces (BCIs) offer a natural complement to this gap. Electroencephalography, in particular, provides a non-invasive, wearable means of measuring neural correlates of attention, relaxation, cognitive load, and emotional valence at the timescales relevant to interactive behaviour. Prior work has demonstrated EEG-based affective classification in desktop settings (Xiong et al., 2025), but integration with real-time immersive 3D environments—especially on consumer hardware—remains underexplored.

We present PiEEG XR, an open-source platform that couples a low-cost, BLE5-connected EEG front-end (IronBCI) with a WebXR application stack to produce a complete, browser-deployable neural-adaptive avatar system. The contributions of this work are:

(1)

A full-stack, open-source architecture connecting consumer EEG hardware to a WebXR avatar pipeline via a standards-based OSC protocol.

(2)

A mapping heuristic from five EEG frequency-band powers to six VRM facial expression categories, designed for real-time, low-latency deployment.

(3)

An evaluation of end-to-end BCI-to-avatar latency and render-loop throughput on commodity XR hardware.

(4)

A Focus-to-Action API enabling thought-modulated interaction with virtual objects, demonstrated in a prototype mixed reality environment.

2. Related Work

2.1. EEG-Based Affective Computing

The mapping of EEG signals to affective states has a well-established research lineage. Theta and alpha band power are broadly associated with meditative and relaxed states, while elevated beta and gamma activity correlates with active cognitive processing and focused attention. Cahn et al. (2006). Valence and arousal—the canonical dimensions of the circumplex model of affect—have been decoded from frontal asymmetry and parieto-occipital alpha suppression, respectively (Greco et al. 2021). Commercial platforms such as NeuroSky MindWave and Emotiv EPOC have brought simplified attention and meditation metrics to consumer applications, though typically at lower spatial resolution and sampling rates than research-grade systems.

2.2. BCI in Virtual and Augmented Reality

Prior investigations of BCI in VR have largely focused on motor imagery-based control (Wen et al., 2021) and passive monitoring of presence and workload.

More recently, attempts to synchronise avatar facial expressions with affective EEG indices have appeared in the social VR community principally as bespoke VRChat modifications. So PiEEG XR provides the first fully open-source, browser-native implementation of this paradigm.

2.3. WebXR and Browser-Native Immersive Experiences

WebXR exposes immersive session capabilities (VR and AR) directly to JavaScript through a standardised API, enabling hardware-agnostic deployment without application stores. Recent frameworks such as React Three Fiber and @react-three/xr have made it practical to construct production-quality immersive applications in TypeScript. The VRM avatar format, maintained by the Virtual Cast community and adopted by VRoid Hub, provides a portable, humanoid model specification with built-in blend-shape expression semantics, making it well-suited to programmatic affective mapping.

2.4. Open-Source BCI Hardware

The PiEEG ecosystem builds on PiEEG, JNEEG, and ardEEG. The IronBCI module used in the XR variant adds wireless BLE5 connectivity and 24-bit ADS1299-family front-end circuitry optimised for ambulatory use, with a GitHub repository accumulating over 600 stars at the time of writing.

3. System Architecture

PiEEG XR comprises three loosely coupled layers:

• the hardware acquisition layer,
• the signal processing and server layer
• the XR rendering layer.

An optional fourth path routes band-power parameters to external social VR clients via OSC.

3.1. Hardware Acquisition Layer

The EEG front-end is the IronBCI module (Figure 1), which integrates a Texas Instruments ADS1299 analogue front-end with a BLE5 system-on-chip. Key specifications are summarised in Table 1.

The electrode placement on the PiEEG XR is shown in Figure 2.

A conductive-gel wet-electrode or dry-electrode cap is supported.

3.2. Signal Processing and Server Layer

The PiEEG Server is a Python daemon that receives raw ADC samples over BLE5, performs digital signal processing, and exposes derived metrics via two interfaces: a WebSocket API and, optionally, a UDP OSC stream. The server is distributed via PyPI (`pip install pieeg-server`) and documented at docs.pieeg.com.

The primary processing steps are:

(1) hardware-level common-mode rejection and digital notch filtering at mains frequency (50 or 60 Hz);

(2) a 4th-order Butterworth band-pass cascade segmenting the signal into delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–100 Hz) sub-bands;

(3) rolling-window power spectral density estimation using a 2-second Hamming-windowed FFT with 50% overlap;

(4) cross-channel averaging and min-max normalization to produce band power scores in [0, 1].

Two composite metrics are computed from the band powers. Attention is estimated as a weighted combination of beta and gamma power, consistent with the neurophysiological literature on frontal midline theta and parietal beta. Meditation (relaxation) is estimated from alpha dominance. Dominant state classification uses threshold logic: if alpha-normalized power exceeds 0.7 and is 0.2 units above beta, the state is labelled relax; if beta and gamma jointly exceed their respective thresholds, the state is labelled focus; elevated beta alone triggers an alert state; otherwise, the state is neutral.

3.3. WebXR Rendering Layer

The XR application is a React + TypeScript single-page application built with Vite and deployed as a static bundle. The rendering engine is Three.js, accessed through the React Three Fiber (R3F) declarative wrapper. Immersive sessions (both VR and AR passthrough) are managed by @react-three/xr, which handles WebXR session lifecycle, reference spaces, and controller event routing without requiring a native application container.

The primary scene component, VrmAvatarScene, subscribes to the EEG data stream and drives the VrmAvatar component—a Three.js scene graph node wrapping a @pixiv/three-vrm model loader. Avatar placement in mixed reality space uses a spatial anchor system: on session start, the avatar is positioned at a configurable distance (default 1.2 m) in front of the user's forward gaze vector and locked to a world-space pose. The user may reposition the avatar by grip-dragging with an XR controller, which updates the anchor without frame-locking the avatar to head motion, preserving the sense of the avatar as an independent entity in shared space.

4. EEG-to-Expression Mapping

The central contribution of the PiEEG XR pipeline is the mapping from continuous EEG-derived affective state estimates to the discrete and continuous expression parameters of a VRM avatar. The mapping is implemented as a deterministic state function evaluated at the server's output cadence (default 4 Hz, configurable) and propagated to the avatar's expression manager via the React state tree.

Table 2 specifies the mapping used in the current prototype.

The expression manager of the @pixiv/three-vrm runtime accepts blend-shape weights in [0, 1] for each named expression slot. In each render frame, the VrmAvatar component zeroes all expression weights, then applies a weight of 1.0 to the currently active expression, with an additional overlay for autonomous blinking (scheduled at Poisson-distributed intervals with a mean inter-blink period of 4 seconds). This approach avoids weight accumulation artefacts from missed clean-up calls in prior frame.

A Focus-to-Action affordance extends the expression mapping to environmental interaction. When the dominant EEG state is focus and concentration metrics sustain above threshold for a configurable dwell time (default 1.5 s), a selectstart event is synthesised on the XR controller pointing at the currently gazed interactable object.

5. Webxr Avatar Rendering

5.1. VRM Model and Animation

The default avatar asset is a VRM 1.0 model distributed with an open licence. VRM encodes a standardised skeletal hierarchy over 54 human bone slots and a named expression set. PiEEG XR applies an idle pose at every render frame (overriding Three.js's default T-pose) by directly setting normalised bone node rotations: upper arms are rotated about the Z-axis to bring them to the anatomical neutral position, and the elbows receive a slight forward flexion.

Figure 3. General view of the first Mask prototype with integrated silver/silver chloride electrodes.

Figure 3. General view of the first Mask prototype with integrated silver/silver chloride electrodes.

5.2. Spatial Anchoring in AR

In passthrough AR mode on Meta Quest, the avatar occupies a fixed world-space position. The anchor is initialised by projecting along the user's forward gaze direction from the head pose at session start, with a vertical offset to place the avatar's feet at estimated floor level for a standing adult user. The outer group node of the scene graph receives the anchor pose each frame rather than the VRM node itself, preserving VRM's internal transform hierarchy.

Figure 4. Avatar control fragment via PiEEG XR.

Figure 4. Avatar control fragment via PiEEG XR.

6. Discussion

6.1. Strengths

The principal strength of PiEEG XR is the completeness and openness of the stack. From raw ADC samples to rendered facial expression, every component is open-source, documented, and deployable without proprietary SDKs or application stores. The WebXR delivery mechanism means that a researcher with a Meta Quest and a local network can run the system with a single npm command. The OSC bridge extends the reach of the platform to an established social VR user community without requiring those users to understand EEG signal processing.

The VRM format is a particularly well-chosen abstraction boundary. Because VRM standardises expression slot names across avatar models, the EEG-to-expression mapping is model-agnostic: any VRM avatar downloaded from VRoid Hub or created with VRoid Studio will respond to the same blend-shape commands. This decouples avatar customisation from system development and enables a diverse avatar ecosystem.

6.2. Limitations

Several limitations must be acknowledged. First, the EEG-to-expression mapping is heuristic rather than trained. The band-power thresholds were set empirically and have not been validated against ground-truth affective labels in a controlled study. Individual differences in resting-state EEG, session-to-session variability, and motion artefacts from head-mounted display donning may reduce mapping reliability.

EEG recorded near a VR headset is susceptible to electromagnetic interference from the HMD's display and wireless subsystems. Shielded electrode cables and the digital notch filter mitigate this partially, but dedicated characterisation of HMD-induced artefacts in the IronBCI frequency bands has not yet been performed.

6.3. Future Work

Planned extensions include: (1) integration of a real-time BLE5 Web Bluetooth adapter in the browser, eliminating the Python server for simple single-device deployments; (2) a per-user calibration wizard that learns mapping thresholds from 2 minutes of labelled affective imagery; (3) multi-user synchronisation to support shared avatar spaces where each participant's neural state modulates their own avatar; (4) expansion of the Focus-to-Action API to continuous modulation of object properties (scale, colour, particle emission rate) proportional to instantaneous band power; and (5) formal user studies evaluating presence, social connectedness, and neurofeedback efficacy in PiEEG XR-enabled environments.

7. Conclusions

We have presented PiEEG XR, an open-source platform that connects consumer-grade 24-bit, 250 Hz EEG hardware to a WebXR avatar system through a standards-based pipeline of BLE5 acquisition, frequency-band power analysis, OSC streaming, and VRM expression control. PiEEG XR demonstrates that the technical barrier to neural-adaptive avatars in consumer mixed reality is no longer hardware cost or rendering performance, but rather the calibration and validation of affective state decoders at the individual user level. As wearable EEG hardware miniaturises further and browser-native BLE APIs mature, we anticipate that BCI-driven avatar affect will transition from a research curiosity to an expected feature of immersive social platforms. We offer PiEEG XR as an open foundation for that transition.