Filter Before Mixing: Per-Modality Denoising for Multimodal RL with Application to Health Management

1. Introduction

Reinforcement learning agents operating in the physical world receive information through multiple sensors: cameras capture spatial structure, proprioceptive sensors report joint angles, language instructions convey goals, and reward signals provide sparse feedback. These modalities differ vastly in noise level, sampling rate, and information density. A robust multimodal RL system must combine them without allowing noise from one channel to degrade the others.

Recent RL architectures address multimodal fusion in two broad ways. Monolithic approaches process all modalities through a single shared backbone. DreamerV3 [1] feeds observations through a shared encoder–RSSM pipeline with fixed hyperparameters, mastering over 150 tasks spanning Atari to robotic control. $\pi$0 [2] tokenizes vision, language, state, and action noise into a 3B-parameter PaliGemma transformer with flow matching for continuous action generation. Genie 2 [3] and DIAMOND [4] apply diffusion to world model prediction and video generation respectively. These systems rely on model scale to implicitly absorb distributional differences across modalities.

Modular approaches assign a dedicated encoder to each modality, then fuse the outputs. MSDP [5] pretrains a multisensory encoder via masked autoencoding across vision, force, and proprioception, then fuses embeddings through cross-attention. VTDexManip [6] concatenates CLIP visual features with tactile MLP features for dexterous manipulation, finding that adding sparse touch signals improves success rates by 20%. MIB [7] compresses a joint representation via the information bottleneck principle, filtering task-irrelevant information after fusion. These methods use separate encoders but do not equalize signal quality before fusion: the noisy output of a sparse reward encoder enters the same fusion layer as a well-encoded CLIP embedding.

Both paradigms thus share a common vulnerability: cross-modal noise contamination at the point of fusion. When a high-variance modality (e.g., sparse reward, raw IMU acceleration) is fused with a low-variance one (e.g., CLIP vision embedding), noise from the former can corrupt the latter. This problem is especially acute in sensor-driven domains such as wearable health management, where modalities have radically different noise profiles and temporal scales—physical movement precedes heart-rate elevation by several seconds, and language instructions precede visual confirmation by many steps.

A second gap is the absence of principled spectral-domain fusion. Standard attention computes instantaneous pairwise similarities and cannot naturally represent temporally delayed cross-modal correlations. A third gap is the limited application to healthcare. MedDreamer [8] applied an RSSM to electronic health records, and KANDI [9] used diffusion policies for elderly activity promotion, but no prior work has combined per-modality generative refinement with spectral fusion for wearable-sensor health intervention.

This paper addresses these gaps through a design principle we call filter-before-mixing: each modality’s representation is independently refined by a dedicated Flow Matching module before cross-modal fusion, analogous to the signal processing practice of filtering each channel before mixing into a master bus. The refinement intensity adapts to each modality’s signal quality: weak for already well-encoded signals (e.g., frozen CLIP embeddings) and strong for noisy or sparse signals (e.g., raw reward, IMU data). The refined representations are then fused in the spectral domain via a Fourier Neural Operator (FNO), whose complex-valued weights naturally encode phase-shifted cross-modal correlations. An information-theoretic analysis (Section 3.3) shows that pre-fusion denoising preserves more policy-relevant mutual information than post-fusion alternatives, with a residual gate mechanism ensuring that the refinement is never harmful.

The resulting four-layer modular architecture integrates modality-specific encoding (frozen CLIP with Slot Attention, or IMU foundation encoders for health applications), per-modality Flow Matching, FNO spectral fusion, and DNC episodic memory. The modular design enables domain transfer: the same downstream pipeline (Flow Matching → FNO → DNC → PPO) is shared between MiniGrid navigation and PAMAP2 health management, with only the modality-specific encoders replaced.

We validate the framework in three domains. In MiniGrid navigation, the system achieves 94.0% success on MultiRoom-N2-S4 (+51 pp over IMPALA under identical conditions) and 93.0% on N4-S4, while IMPALA fails entirely on harder configurations (0% on N4-S5 and N6). In Crafter, a procedurally generated open-world environment with no language instructions, the agent achieves an official score of 16.0%, exceeding DreamerV3 (14.5%) despite operating without the language modality. In wearable-sensor health management on the PAMAP2 dataset—where no pre-trained encoder exists and per-modality denoising is critical—the foundation encoder achieves 2.4× higher cumulative reward ($268.3\pm 12.0$) and 16× lower cross-seed variance than a vanilla MLP baseline. This progression—MiniGrid (4 modalities, CLIP available), Crafter (3 modalities, no language), PAMAP2 (3 modalities, no pre-trained encoder)—illustrates a key insight: the advantage of filter-before-mixing grows with encoder noise and is robust to missing modalities.

The contributions of this paper are as follows:

1.

We propose the filter-before-mixing design principle for multimodal RL, in which per-modality Flow Matching denoises each representation before spectral-domain fusion, and provide an information-theoretic justification with an explicit approximation bound (Eq. (A11)).

2.

We instantiate this principle in FreamerV1 (Filter-before-mixing dreamer), a modular four-layer architecture (93M parameters, 0.4M trainable), and validate it on MiniGrid navigation with controlled baselines (IMPALA under identical conditions, PPO).

3.

We validate across three domains—MiniGrid (4 modalities), Crafter (3 modalities, no language), and PAMAP2 (3 modalities, no pre-trained encoder)—showing that the architecture degrades gracefully with missing modalities and that the filter-before-mixing advantage grows with encoder noise.

Table 1 summarizes how these design choices differentiate FreamerV1 from $\pi$0. These are not claims of superiority; they reflect optimization for a different regime—sensor-driven decision-making with discrete actions, limited data, and the need for interpretable, modular policies.

2. Related Work

We organize related work along the three problems identified in the Introduction: (1) how existing architectures fuse multimodal inputs and where cross-modal noise contamination arises, (2) how generative models have been applied in RL and where our per-modality approach diverges, and (3) how world models have been used in healthcare.

2.1. Multimodal Fusion in RL Foundation Models

The dominant approach to multimodal fusion in recent RL foundation models is monolithic: all modalities are tokenized and processed by a single large transformer. $\pi$0 [2] feeds image tokens (from PaliGemma), language tokens, robot-state tokens, and noisy action tokens into a shared self-attention backbone, relying on the model’s 3B-parameter capacity to implicitly disentangle heterogeneous inputs. Gato [10] similarly serializes text, images, and actions into a single token sequence for a 1.2B-parameter transformer. DreamerV3 [1] fuses observations through a shared encoder–RSSM pipeline with fixed hyperparameters across both discrete and continuous action domains, achieving human-level Atari performance and competitive robotic control.

These architectures have proven effective when the input modalities are relatively homogeneous (e.g., images and proprioception in robotic manipulation) or when the model is large enough to absorb distributional differences. However, they do not explicitly prevent noise from one modality from propagating to another during fusion—the problem we term cross-modal noise contamination. Our framework addresses this by denoising each modality independently via per-modality Flow Matching before fusion.

A related line of work explores modality-specific processing. CLIP [11] demonstrated the power of separate vision and language encoders aligned through contrastive learning. Slot Attention [12] decomposes visual inputs into object-centric slots, enabling structured representation learning. SAVi [13] extended this to video with temporally consistent slot tracking. We adopt both CLIP and Slot Attention as modality-specific encoders in Layer 1 of our architecture, using them as established building blocks rather than claiming novelty for their individual designs.

2.1.1. Multimodal Sensor Fusion in RL

Recent work has increasingly recognized that naive fusion of heterogeneous sensor modalities can degrade RL performance. Meng et al. [7] demonstrated that concatenating egocentric images and proprioception can fail to match single-modality performance, and proposed a Multimodal Information Bottleneck (MIB) that compresses joint representations while retaining task-relevant information. Their approach filters out task-irrelevant information after fusion, in contrast to our per-modality pre-fusion denoising.

In contact-rich robotic manipulation, MSDP [5] proposed self-supervised multisensory pretraining using masked autoencoding across vision, force, and proprioception. Their key insight is an asymmetric actor-critic architecture: the critic uses cross-attention over frozen sensor embeddings for dynamic feature extraction, while the actor receives a stable pooled representation. This achieves robustness to sensor noise with as few as 6,000 online interactions on real hardware. However, MSDP does not apply modality-specific noise reduction before fusion—all sensor embeddings enter the same transformer encoder and are masked uniformly.

VTDexManip [6] presented a benchmark for visual-tactile dexterous manipulation, comparing 17 pretrained and non-pretrained methods. Their results showed that adding sparse binary tactile signals to vision improves success rates by approximately 20%, and joint visual-tactile pretraining gains a further 20%. Fusion is achieved by concatenating visual features (from CLIP, R3M, or ResNet) with tactile MLP features—a simple strategy that does not account for the very different noise characteristics of high-resolution images versus sparse binary contact signals.

These approaches share a common pattern: modality-specific encoders followed by concatenation, cross-attention, or information-theoretic fusion. None applies generative refinement (e.g., Flow Matching) to individual modality representations before fusion. Our per-modality FM fills this gap by allowing the refinement intensity to be tuned per modality—weak for already well-encoded signals (e.g., CLIP vision) and strong for noisy or sparse signals (e.g., reward, raw IMU)—before spectral fusion combines the cleaned representations.

2.2. Generative Models and Memory in RL

Generative models in RL have been applied almost exclusively at the output stage: Decision Diffuser [14] generates trajectories, Diffusion Policy [15] produces action chunks, $\pi$0 [2] applies flow matching to 50-step action generation, and DIAMOND [4] learns world models via diffusion over observations. All operate after multimodal fusion. Our approach inverts this: Flow Matching operates on internal representations before fusion, refining each modality’s encoding via learned optimal transport. For fusion itself, we employ a Fourier Neural Operator (FNO) [16], whose complex-valued spectral weights naturally encode phase-shifted cross-modal correlations—a property absent from attention-based fusion.

For episodic memory, the Differentiable Neural Computer (DNC) [17] provides content-based external memory, previously applied to RL navigation in MERLIN [18]. In our framework, the DNC restores episodic context that is lost when the FNO produces a single-timestep fused representation.

2.3. World Models: From Games to Healthcare

World models—learned environment simulators that predict future states from actions—have become a dominant paradigm for sample-efficient RL. DreamerV2 [19] introduced categorical latent representations for discrete-action Atari games. DreamerV3 [1] generalized this across 150+ tasks with fixed hyperparameters, spanning Atari, DMControl, Minecraft, and Crafter, handling both discrete and continuous actions. MuZero [20] combined learned models with Monte Carlo tree search for board games and Atari. More recently, DIAMOND [4] and Genie 2 [3] have explored diffusion-based world models at foundation scale.

Despite this success, world model-based RL remains underexplored in healthcare. MedDreamer [8] is the closest precedent: it adapts the DreamerV3 RSSM to electronic health records (EHRs) for sepsis treatment and mechanical ventilation, introducing an Adaptive Feature Integration module to handle irregular clinical time series. KANDI [9] addresses physical activity promotion for fall-risk elderly using wearable accelerometers, combining Diffusion Policies with offline inverse RL. However, neither MedDreamer nor KANDI employs structured multimodal encoding (Slot Attention, per-modality Flow Matching) or spectral-domain fusion, and neither targets continuous wearable-sensor health management with online imagination-based policy optimization.

2.4. Positioning of This Work

Table 2 positions our framework against existing methods along five capability axes that correspond to the design choices motivated in the Introduction. No prior method combines per-modality generative representation refinement, spectral-domain cross-modal fusion, language-grounded reward shaping, external episodic memory, and world model-based health intervention.

3. Proposed Method

This section describes the architecture that instantiates the filter-before-mixing principle introduced in Section I.

3.1. Architecture Overview

The architecture consists of four layers (Figure 1):

1.

Layer 1 — Modality-Specific Encoding. Each of four input modalities (state, vision, language, reward) is encoded by a dedicated module. Vision uses a frozen CLIP ViT-B/32 [11] with Slot Attention [12]; language uses a Transformer encoder [21] with Slot Attention; state uses an FNO-based encoder [16]; reward uses a temporal Conv1D encoder. All encoders produce d-dimensional embeddings.

2.

Layer 2 — Per-Modality Flow Matching. Each modality embedding is refined by an independent Flow Matching module [22] that learns an optimal transport map from a Gaussian prior to the modality’s data manifold, denoising the representation before cross-modal fusion.

3.

Layer 3 — FNO Spectral Fusion. The refined modality embeddings are stacked as a $M\times d$ matrix and fused via spectral-domain convolution [16] (SpectralConv1d), producing a unified representation.

4.

Layer 4 — DNC Memory + Policy. The fused representation is augmented with episodic context from a DNC memory module [17], and fed to a PPO [23] policy head that produces a discrete action.

The total parameter count is 93.05M, of which 92.68M (99.6%) are the frozen CLIP encoder. The trainable parameters amount to 0.37M, enabling efficient learning on small datasets.

3.2. Layer 1: Modality-Specific Encoding

Each modality is encoded by a dedicated module producing a d-dimensional embedding (details in Appendix E).

For vision, a frozen CLIP ViT-B/32 [11] extracts 49 patch features ($7\times 7$ grid, 768-dim), projected to ${\mathbb{R}}^d$. Slot Attention [12] decomposes the features into K object-centric slots via iterative competitive assignment. Four numerical stabilization techniques reduce the NaN occurrence rate from 23.5% to 0% (Table 5).

For language, the mission text is processed by a Transformer encoder and decomposed into $K_{\mathrm{lang}}$ slots via Slot Attention. An LLM (Qwen2.5 [24]) parses the mission into structured components for reward shaping (Appendix A).

For state, the MiniGrid observation ($7\times 7\times 3$) is flattened and encoded by a 1D FNO block [16].

For reward, the past H-step reward history is encoded by a temporal Conv1D.

Bidirectional multi-head cross-modal attention between visual and language slots enables component-level correspondence (e.g., “green goal” slot ↔ green object slot). The resulting slots are mean-pooled to produce modality embeddings $e_m\in {\mathbb{R}}^d$.

3.3. Layer 2: Per-Modality Flow Matching

For each modality $m\in \left\{\mathrm{state},\mathrm{vision},\mathrm{language},\mathrm{reward}\right\}$, we apply an independent Flow Matching module ${\varphi }^m$ to refine the encoded representation $e_m$ before cross-modal fusion. This implements the “filter-before-mixing” principle (Section III-A-2).

Each module learns a velocity field $v_{\theta }^m$ via Conditional Flow Matching (CFM) [22] and refines $e_m$ by Euler integration from $t=0$ to $t=1$ over N steps (see Appendix E for the full formulation):

$$\begin{array}{cc}\hfill x_0^m& =e_m\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill x_{i+1}^m& =x_i^m+{\displaystyle \frac{1}{N}}{v}_{\theta }^m(x_i^m,i/N),i=0,\dots ,N-1\hfill \end{array}$$

(2)

A learnable residual gate ensures training stability:

$${\widehat{e}}_m=e_m+\sigma \left({\alpha }^m\right)·(x_N^m-e_m)$$

(3)

where ${\alpha }^m$ is initialized to $-3$ so that $\sigma \left({\alpha }^m\right)\approx 0.047$ at the start of training, ensuring that the FM module acts as a near-identity function until its velocity field is sufficiently trained. Crucially, the refinement intensity can be set independently per modality: weak for already well-encoded signals (e.g., frozen CLIP: $\alpha =-5$, $\sigma \approx 0.007$) and strong for noisy signals (e.g., sparse reward: $\alpha =-1$, $\sigma \approx 0.27$).

We provide an information-theoretic argument for why applying Flow Matching to each modality independently before fusion is preferable to applying it after fusion (see Appendix J for the full derivation).

Let ${\left\{X^m\right\}}_{m=1}^M$ denote the encoded representations of M modalities, each corrupted by modality-specific noise: ${\tilde{X}}^m=X^m+{ϵ}^m$, where ${ϵ}^m\sim \mathcal{N}(0,{\sigma }_m^{2}I)$ and the noise variances ${\sigma }_m^2$ differ across modalities.

In post-fusion denoising (as in $\pi$0), the noisy representations are first fused as $Z=f({\tilde{X}}^1,\dots ,{\tilde{X}}^M)$ and then denoised. By the data processing inequality [25], information lost during noisy fusion cannot be recovered. Cross-modal noise propagates through shared projection weights.

In pre-fusion denoising (our approach), a modality-specific denoiser $g^m$ is applied to obtain ${\widehat{X}}^m\approx X^m$, then the cleaned representations are fused. If each FM achieves near-optimal denoising:

$$I\left((X^1,\dots ,X^M);f({\widehat{X}}^1,\dots ,{\widehat{X}}^M)\right)\ge I\left((X^1,\dots ,X^M);f({\tilde{X}}^1,\dots ,{\tilde{X}}^M)\right)$$

(4)

In practice, the FM modules are imperfect denoisers with error ${\delta }^m={\widehat{X}}^m-\mathbb{E}\left[X^m\right|{\tilde{X}}^m]$. Pre-fusion denoising remains beneficial whenever $\parallel {\delta }^m{\parallel }^2<{\sigma }_m^2$—a condition substantially weaker than optimal denoising (see Appendix J for the full derivation). The residual gate further tightens this bound: since $\sigma \left({\alpha }^m\right)\to 0$ when the velocity field is untrained, pre-fusion denoising is never worse than no denoising.

3.4. Layer 3: FNO Spectral Fusion

The four refined modality embeddings ${\widehat{e}}_m\in {\mathbb{R}}^d$ are stacked into a matrix $E\in {\mathbb{R}}^{M\times d}$ and fused via a Fourier Neural Operator (FNO) [16]. The FNO applies spectral convolution: FFT along the embedding dimension, multiplication by learnable complex-valued weights $R_k\in {\mathbb{C}}^{M\times M}$ for the first K Fourier modes, and inverse FFT, with a pointwise residual path (see Appendix E for the full formulation). The fused representation is obtained by mean-pooling over the modality axis.

The key advantage of spectral-domain fusion over attention-based fusion is the ability to represent phase-shifted cross-modal correlations. When two modalities have a temporal delay $\tau$ in their correlation (e.g., IMU activity precedes heart-rate elevation), this appears as a frequency-dependent phase shift $e^{-j\omega \tau }$ in the Fourier domain. The FNO’s complex-valued weights can directly encode such shifts through $arg\left(R_k\right)$, whereas real-valued attention computes instantaneous inner products and cannot represent phase delays without auxiliary mechanisms.

3.5. Layer 4: DNC Memory and PPO Policy

The fused representation $e_{\mathrm{fused}}$ is augmented with episodic context via a Differentiable Neural Computer (DNC) [17] with content-based addressing, producing a memory-augmented representation $e_{\mathrm{aug}}$. A categorical PPO [23] policy head then produces discrete actions:

$${\pi }_{\theta }(a\mid s)=\mathrm{softmax}(W_{\pi }{e}_{\mathrm{aug}}+b_{\pi })$$

(5)

The DNC architecture details (content-based addressing, write/read operations) and PPO objective formulation are provided in Appendix F.

3.6. Auxiliary Components

The framework includes several auxiliary mechanisms whose details are in Appendix G: Language-grounded reward shaping provides dense supervision in sparse-reward environments by matching LLM-parsed mission structure against observations [24]. Adaptive reward shaping [26] decays bonus rewards as the success rate increases. Success Buffer [27] stores and replays successful episodes to prevent catastrophic forgetting. Score-based adaptive complexity estimation [28] dynamically adjusts the number of FM inference steps per modality.

3.7. Training Objective

The overall loss function combines four terms:

$$\mathcal{L}={\mathcal{L}}_{\mathrm{PPO}}+{\lambda }_{\mathrm{FM}}{\mathcal{L}}_{\mathrm{FM}}+{\lambda }_{\mathrm{CL}}{\mathcal{L}}_{\mathrm{CL}}+{\lambda }_s{\mathcal{L}}_{\mathrm{smooth}}$$

(6)

where ${\mathcal{L}}_{\mathrm{FM}}$ is the per-modality Flow Matching loss (Eq. (A6)), ${\mathcal{L}}_{\mathrm{CL}}$ is a vision–language contrastive alignment loss [11], and ${\mathcal{L}}_{\mathrm{smooth}}$ is the FNO smoothness regularization (Eq. (A9)).

The total loss combines four objectives operating at different levels of the architecture: ${\mathcal{L}}_{\mathrm{FM}}$ trains the per-modality Flow Matching velocity fields (Layer 2), ${\mathcal{L}}_{\mathrm{smooth}}$ and the FNO parameters govern cross-modal fusion (Layer 3), ${\mathcal{L}}_{\mathrm{CL}}$ aligns vision and language representations (cross-layer), and ${\mathcal{L}}_{\mathrm{PPO}}$ optimizes the policy (Layer 4).

A potential concern with multi-objective optimization is gradient interference: updates to the FM velocity fields $v_{\theta }^m$ that reduce ${\mathcal{L}}_{\mathrm{FM}}$ might increase ${\mathcal{L}}_{\mathrm{PPO}}$ by changing the representation landscape that the policy has adapted to. We mitigate this through two mechanisms. First, each FM module produces its output through a learnable residual gate ${\widehat{X}}^m=X^m+\sigma \left({\alpha }^m\right)·(\mathrm{ODE}\left(X^m\right)-X^m)$, initialized near zero (${\alpha }_0^m=-3$, $\sigma \left({\alpha }_0^m\right)\approx 0.047$). During early training, ${\widehat{X}}^m\approx X^m$, so the FM modules do not disrupt the representations that the policy is learning from. As training progresses, $\sigma \left({\alpha }^m\right)$ grows, gradually introducing the FM refinement. This staged introduction ensures that ${\mathcal{L}}_{\mathrm{PPO}}$ and ${\mathcal{L}}_{\mathrm{FM}}$ do not interfere during the critical early phase. Second, the CLIP visual encoder (92.68M of 93.05M total parameters) is frozen, eliminating the largest source of potential gradient interference. The FM velocity fields, FNO weights, and policy parameters constitute only 0.37M trainable parameters, operating in a low-dimensional optimization landscape where multi-objective conflicts are empirically manageable.

Regarding convergence, the conditional Flow Matching loss ${\mathcal{L}}_{\mathrm{CFM}}={\mathbb{E}}_{t,q\left(z\right),p_t\left(x\right|z)}[\parallel u_t-v_{\theta }{\parallel }^2]$ is a regression loss with a unique global minimum, and Lipman et al. [22] showed that its gradient estimator has bounded variance under the optimal transport conditional path. Combined with PPO’s clipped surrogate objective [23], which bounds policy updates to a trust region, and the Adam optimizer with gradient clipping ($\parallel \nabla \parallel \le 1.0$), the overall training procedure converges reliably in practice.

The full PPO loss is:

$${\mathcal{L}}_{\mathrm{PPO}}=-min\left(r_t{A}_t,\mathrm{clip}(r_t,1-ϵ,1+ϵ)A_t\right)+c_1{\mathcal{L}}_{\mathrm{value}}-c_{2}H\left[\pi \right]$$

(7)

4. Experiments

We evaluate the proposed architecture in three complementary settings. First, we verify that the Slot Attention stabilization techniques are effective on the MSR-VTT video–text dataset, as numerical stability is a prerequisite for the downstream Flow Matching and FNO layers. Second, we conduct comprehensive experiments on MiniGrid navigation tasks to assess the overall performance of the integrated system and to quantify the contribution of each component through ablation. Third, we apply the architecture to wearable-sensor health management on the PAMAP2 dataset (Section 5) to validate whether the filter-before-mixing principle produces better world models than flat-concatenation or attention-only encoders.

4.1. Experimental Setup

4.1.1. MSR-VTT Experiments

We used the MSR-VTT dataset [29] comprising 7,010 videos with 20 captions each (90%/10% train/validation split) to evaluate the numerical stability of Slot Attention under heterogeneous multimodal inputs. Images were resized to $224\times 224$ for the CLIP encoder.

4.1.2. MiniGrid Experiments

We used MiniGrid [30], a 2D grid-world environment for goal-oriented navigation and instruction-following tasks [31]. Experiments were conducted across several configurations: Empty ($5\times 5$, $8\times 8$), DoorKey ($5\times 5$, $8\times 8$), and MultiRoom (N2–N4, S4–S5). Each configuration was trained with a fixed random seed; IMPALA baselines used 3 seeds with standard deviation reported. Multi-seed evaluation of the proposed method with error bars is reported for the full architecture experiments (Section 4.3.5).

Table 3 shows the model configuration. The LLM mission parser uses Qwen2.5-3B-Instruct with a regex-based fallback. Table 4 shows that the trainable parameters constitute only 0.4% of the total, with the frozen CLIP encoder accounting for 99.6%.

4.2. Slot Attention Stability

Table 5 shows that the four stabilization techniques (Section IV-B-1) progressively eliminate NaN occurrences. The full set reduces the NaN rate from 23.5% to 0%, which is a prerequisite for the downstream Flow Matching and FNO fusion layers—if Slot Attention produces NaN, the entire pipeline collapses.

Table 5. Effect of Slot Attention Stabilization Techniques (MSR-VTT)

Configuration	NaN Rate	Training Completion
No stabilization	23.5%	76.5%
+ Mask value correction	8.2%	91.8%
+ NaN fallback	2.1%	97.9%
+ GRU init + var. clipping	0.0%	100.0%

Table 5. Effect of Slot Attention Stabilization Techniques (MSR-VTT)

Configuration	NaN Rate	Training Completion
No stabilization	23.5%	76.5%
+ Mask value correction	8.2%	91.8%
+ NaN fallback	2.1%	97.9%
+ GRU init + var. clipping	0.0%	100.0%

Table 6 confirms that the stabilized CLIP + Slot Attention encoder achieves the lowest cross-modal alignment loss, validating that the stabilization does not compromise representation quality.

4.3. MiniGrid Results

4.3.1. Overall Performance

Table 7 summarizes results across MiniGrid configurations. The framework achieves its strongest results on MultiRoom-N4-S4 (93.0% success) and N2-S4 (94.0%), demonstrating effective navigation through up to 4 interconnected rooms.

Two patterns are notable. First, room size 4 is consistently easier than size 5 (N4-S4: 93% vs. N4-S5: 36%), suggesting that the language reward shaping (“traverse rooms to reach the goal”) provides sufficient guidance when each room is small enough for the agent to observe the door and goal simultaneously. The N4-S5 from-scratch experiment (0% at 2000 episodes) confirms that curriculum learning is essential for complex multi-room configurations: the 35.7% achieved with curriculum initialization from N2-S4 cannot be reached by training from scratch within the same budget. Second, DoorKey-8x8 plateaus at 20%, indicating that the current framework struggles with tasks requiring key acquisition followed by door opening—a two-stage compositional skill that may benefit from hierarchical planning not included in the current architecture.

4.3.2. Comparison with PPO Baseline

Table 8 shows that the integrated framework improves over standard PPO by +44.6 pp on N2-S4 and +31.4 pp on N3-S4. The PPO baseline uses FlatObsWrapper (symbolic observations), whereas our method operates on pixel-level visual inputs processed through CLIP, making the comparison conservative: our method achieves higher success rates despite receiving a harder input modality.

4.3.3. Ablation Study

Table 9 quantifies the contribution of each component by removing one at a time from the full system. Three observations connect directly to the design rationale. Removing the frozen CLIP encoder (−19.8 pp) causes the largest performance drop, confirming that internet-scale pre-trained visual representations are the most critical component; the CLIP encoder brings knowledge that cannot be learned from MiniGrid data alone (Section 3). Removing language reward shaping (−8.0 pp) degrades performance substantially, confirming that the LLM-parsed mission structure provides essential dense supervision in the sparse-reward MultiRoom environment. Replacing PPO with SAC (−34.9 pp) yields near-zero success, confirming that categorical PPO is appropriate for discrete action spaces.

4.3.4. Literature-Based Positioning

Table 10 contextualizes our results against other methods reported in the literature. These comparisons are indicative, not definitive, as experimental conditions differ across papers.

DreamerV3—despite its success across 150+ domains including discrete-action Atari—converges to suboptimal policies on MiniGrid [34], where IMPALA outperforms it. To strengthen our comparison, we reproduced IMPALA under identical conditions (FlatObsWrapper, 2000 episodes, MLP with two 256-unit hidden layers, 3 seeds). IMPALA achieved 43.0% ± 7.8% on N2-S4—below the PPO reference value of 65%—and failed entirely on N4-S5 and N6 (0% across all seeds), confirming that model-free methods without external memory or planning mechanisms cannot solve multi-room navigation beyond the simplest configuration. FreamerV1’s 94% success rate on N2-S4 (+51 percentage points over IMPALA) and 93% on N4-S4 demonstrate the effectiveness of the integrated architecture.

4.3.5. Full Architecture Evaluation

To isolate the contribution of the filter-before-mixing pipeline (per-modality FM, FNO spectral fusion, DNC memory) from the modality-specific encoder (Layer 1), we evaluate multiple configurations on MultiRoom-N2-S4:

Table 11 and Figure 2 reveal two important findings. First, FreamerV1 converges more slowly than Layer 1 alone (79% vs. 94% at 2000 episodes) due to the additional 1.4M parameters in the filter-before-mixing pipeline. However, with continued training, FreamerV1 reaches 87.7% ± 8.2% at 5000 episodes (one seed reaching 100%), surpassing the Layer 1-only baseline. Second, Layer 1 alone degrades from 94% to 78% with continued training—a clear instance of catastrophic forgetting visible in the learning curve (Figure 2, red line). The filter-before-mixing pipeline prevents this degradation: per-modality FM stabilizes the learned representations by enforcing modality-specific structure, while the DNC provides episodic recall that anchors the policy against distributional shift in the replay buffer.

The removal of DNC has minimal impact at 2000 episodes (80% vs. 79%), consistent with the observation that N2-S4 rooms are visited sequentially and episodic memory provides limited benefit at short training horizons.

4.4. Transfer Learning

For the N3-S4 environment, a model pre-trained on N2-S4 was used as initialization, followed by continued training. This improved the success rate from 28.0% (training from scratch) to 46.0% (transfer + fine-tuning), demonstrating that the modular architecture learns representations that transfer across environments of different complexity.

4.5. Crafter Experiments

To evaluate the architecture beyond MiniGrid, we apply it to Crafter [36]—a procedurally generated open-world environment with 22 hierarchically structured achievements (e.g., collect wood → place table → make pickaxe → collect stone). Crafter differs from MiniGrid in two critical ways: (1) it provides no language instructions, so the language modality line receives dummy input and language reward shaping is unavailable; (2) the observation is a $64\times 64$ RGB image requiring visual understanding of a complex, procedurally generated world.

The architecture uses three active modality lines (state: 16-dimensional inventory, vision: CLIP-encoded image, reward) with the language line disabled. We train for ${10}^6$ environment steps on a single GPU (RTX 4090, ∼12 hours).

Table 12 summarizes the results. The agent unlocks 16–17 of 22 achievements within ${10}^6$ steps, including intermediate crafting chains (place table, make wood pickaxe, collect stone, place furnace, make stone sword). The average per-episode achievement count of 4.5 indicates that the agent consistently executes multi-step plans, not merely achieving each item once by chance.

Using the official Crafter score ($exp({\displaystyle \frac{1}{22}}\sum ln(1+s_i))-1$), FreamerV1 achieves 16.0% (with achievement reward shaping) and 14.9% (without), compared to DreamerV3’s 14.5% and PPO’s 4.6%. The key observation is that FreamerV1 slightly surpasses DreamerV3 without any language modality—the language line receives dummy input and no language reward shaping is applied. This confirms that per-modality encoding with FNO spectral fusion provides competitive performance even when a primary modality is absent, and suggests that adding language instructions (e.g., achievement descriptions as mission text) could further improve exploration efficiency. We note that EMERALD, a masked latent transformer-based world model, achieves 58.1% on Crafter but requires $10\times$ the training budget (${10}^7$ steps).

5. Application: PAMAP2 Health Management

To validate that the filter-before-mixing principle transfers beyond grid worlds, we apply the same four-layer pipeline to wearable-sensor health management on the PAMAP2 dataset [37]. This domain lacks pre-trained encoders (no CLIP equivalent for IMU/heart-rate data), making per-modality FM refinement critical. Full architectural details (RSSM world model, training procedure, imagination-based PPO, anomaly detection, and health system positioning) are provided in Appendix I.

Figure 3. Architecture of the health management system. Wearable sensor observations (IMU at three body sites and heart rate monitor) are encoded by the foundation encoder (Slot Attention + FM + FNO + DNC), and the RSSM world model learns physiological dynamics in a latent space for imagination-based policy optimization.

Figure 3. Architecture of the health management system. Wearable sensor observations (IMU at three body sites and heart rate monitor) are encoded by the foundation encoder (Slot Attention + FM + FNO + DNC), and the RSSM world model learns physiological dynamics in a latent space for imagination-based policy optimization.

5.1. Domain Transfer via Encoder Replacement

The observation $o_t$ consists of IMU sensors at three body locations (36 dimensions) and heart-rate features (16 dimensions). The foundation encoder applies the same four-stage pipeline as MiniGrid, with modality-specific encoders replaced: Stage 1 treats IMU sites as slots and fuses them with heart rate via Cross-Attention; Stage 2 applies per-modality FM to each sensor group—without a pre-trained encoder, the FM must compensate for raw encoder noise; Stage 3 uses FNO fusion to encode the phase delay between physical movement and heart-rate response; Stage 4 provides episodic recall via DNC. The downstream pipeline (FM → FNO → DNC → PPO) is identical to MiniGrid, demonstrating the modularity claimed in Contribution 3.

5.2. Encoder Comparison

To validate that the foundation encoder improves policy quality beyond simpler encoders, we compare four variants sharing the same RSSM core, reward function, and PPO optimizer (Table 13).

The foundation encoder achieves 2.4× higher reward than the MLP baseline with 16× lower cross-seed variance ($\pm 12.0$ vs. $\pm 195.2$), confirming that the per-modality FM stabilizes learning when no pre-trained encoder is available. The progression MLP → CNN → SlotAttn → Foundation shows that each additional layer of the proposed architecture contributes: structured attention (+28%), then FM+FNO+DNC (+47%). Safety violations decrease from 1.3 to 0.95 per episode.

This result, combined with the MiniGrid experiments where CLIP provides a strong pre-trained encoder and FM’s marginal effect is smaller, supports the central claim: the advantage of filter-before-mixing grows with encoder noise.

Figure 4. Agent decision timeline over a 48-hour scenario. Top: life phases (sleep, commute, exercise, rest). Middle: heart rate with zone coloring (Light/Moderate/Vigorous/Danger). Bottom: agent decisions. Jogging raises HR to 175 bpm, triggering Rest/Alert. After recovery, stair-climbing with groceries raises HR to 180 bpm due to residual fatigue, triggering immediate Alert. KL spikes (↑KL) mark unexpected physiological changes.

Figure 4. Agent decision timeline over a 48-hour scenario. Top: life phases (sleep, commute, exercise, rest). Middle: heart rate with zone coloring (Light/Moderate/Vigorous/Danger). Bottom: agent decisions. Jogging raises HR to 175 bpm, triggering Rest/Alert. After recovery, stair-climbing with groceries raises HR to 180 bpm due to residual fatigue, triggering immediate Alert. KL spikes (↑KL) mark unexpected physiological changes.

6. Discussion

6.1. Filter-Before-Mixing: When Does It Help?

The full architecture evaluation (Table 11) reveals a nuanced picture of when filter-before-mixing is beneficial. In MiniGrid with CLIP, the full architecture initially underperforms the Layer 1-only baseline (79% vs. 94% at 2000 episodes) because the additional FM/FNO/DNC parameters slow convergence. However, with continued training, the trajectories diverge dramatically: FreamerV1 reaches 87.7% ± 8.2% (one seed reaching 100%) while Layer 1 alone degrades to 78% due to catastrophic forgetting (Figure 2). This reveals a previously unrecognized role of per-modality FM: by enforcing modality-specific structure on the representations, FM acts as an implicit regularizer that prevents the policy from overfitting to recent experience and forgetting earlier knowledge.

In PAMAP2, where no pre-trained encoder exists and raw IMU/HR signals are noisy, the effect is immediate: the foundation encoder with FM achieves 2.4× higher reward and 16× lower variance than a vanilla MLP (Table 13). The central finding is therefore twofold: filter-before-mixing improves final performance in all settings through both representational enrichment and forgetting prevention, with the convergence cost largest when the encoder is already strong (MiniGrid + CLIP).

A noteworthy corollary is the dissociation between reconstruction accuracy and policy quality: the MLP encoder achieves the lowest reconstruction MSE (20.1) but the worst reward (113.6), while the foundation encoder shows the opposite. This suggests that per-modality FM optimizes representations for decision-making rather than reconstruction—consistent with the information-theoretic argument of Section 3.3.

6.2. The Role of Each Architectural Layer

The ablation and cross-domain results illuminate the relative contribution of each layer. At Layer 1, the frozen CLIP encoder accounts for +19.8 pp (Table 9), confirming that pre-trained visual representations are the most critical component in MiniGrid. In PAMAP2, Slot Attention + Cross-Attention alone improves reward from 113.6 (MLP) to 182.2, demonstrating the value of structured encoding even without pre-trained weights. At Layer 2, adding FM + FNO + DNC to SlotAttn+CrossAttn improves PAMAP2 reward from 182.2 to 268.3 (+47%), with the residual gate ensuring stable training by defaulting to identity until the velocity field is sufficiently trained. At Layer 3, the FNO’s contribution in MiniGrid is modest because modalities lack strong temporal delays, but in PAMAP2, where IMU activity precedes HR elevation by several seconds, the FNO’s phase-shift capability contributes to the 16× variance reduction. At Layer 4, the DNC’s contribution in MiniGrid is limited (rooms are visited sequentially), but in PAMAP2, episodic recall enables the agent to reference past physiological patterns across activity transitions.1

6.3. Computational Efficiency

The framework requires only 0.37M trainable parameters. The frozen CLIP encoder (92.6M) accounts for 99.6% of the total count but requires no gradient computation, and Score-based adaptive complexity estimation dynamically adjusts FM inference steps per state.

7. Conclusion

This paper proposed a design principle for multimodal reinforcement learning—filter before mixing—in which each modality’s representation is denoised by a dedicated Flow Matching module before cross-modal fusion via a Fourier Neural Operator in the spectral domain. This principle addresses the problem of cross-modal noise contamination that arises when heterogeneous modalities with different noise profiles are fused in a shared backbone, as is standard practice in monolithic VLA architectures such as $\pi$0.

We instantiated this principle in FreamerV1, a four-layer modular architecture integrating a frozen CLIP encoder with Slot Attention, per-modality Flow Matching, FNO spectral guidance, DNC episodic memory, and LLM-based language reward shaping. The framework achieves competitive performance with 0.37M trainable parameters—two orders of magnitude smaller than $\pi$0’s 3B—by exploiting the modular structure to freeze pre-trained components and train only the integration layers.

Experiments in three domains validated the approach. On MiniGrid navigation tasks, the full architecture (FM + FNO + DNC) reached 100% success on MultiRoom-N2-S4 at 5000 episodes, surpassing the 94% ceiling of the Layer 1-only baseline, though at the cost of slower initial convergence. On Crafter, an open-world environment without language instructions, the agent slightly surpassed DreamerV3’s official score (16.0% vs. 14.5%; single seed) using only three of four modality lines, demonstrating graceful degradation with missing modalities. On the PAMAP2 wearable-sensor health management task, the foundation encoder with per-modality Flow Matching, FNO fusion, and DNC memory achieved 2.4× higher cumulative reward ($268.3\pm 12.0$ vs. $113.6\pm 195.2$), 27% fewer safety violations, and 16× lower cross-seed variance compared to a vanilla MLP-based RSSM world model. The dissociation between reconstruction accuracy and policy quality—the MLP encoder reconstructs observations better but produces worse policies—provides empirical support for the information-theoretic argument that pre-fusion denoising preserves policy-relevant information.

The health management application demonstrates that world model-based RL, which has seen remarkable success in games and robotics, can be extended to wearable-sensor health intervention with minimal architectural modification. The RSSM world model enables imagination-based policy optimization in the latent space, avoiding the ethical and practical difficulties of exposing real patients to dangerous physiological states during training. KL-divergence-based anomaly detection provides an additional safety layer for real-time physiological monitoring.

Several directions remain for future work. First, controlled comparisons against DreamerV3 on MiniGrid under identical conditions would further strengthen the experimental claims; preliminary results with the NM512 PyTorch reimplementation are ongoing. Second, extension to continuous-control robotic tasks and 3D environments would test the generality of the filter-before-mixing principle beyond discrete action spaces. Third, clinical validation of the health management application with domain experts and real patient outcomes is essential before deployment. Finally, the modest contribution of the FNO and DNC components in MiniGrid—where temporal delays between modalities are limited—motivates evaluation in domains with richer cross-modal dynamics, where the phase-shift capabilities of spectral-domain fusion are expected to be more fully realized. A direct experimental comparison of pre-fusion vs. post-fusion denoising (e.g., applying FM after FNO fusion rather than before) would further strengthen the information-theoretic argument for the filter-before-mixing principle.

7.1. Limitations

We acknowledge several limitations. For MiniGrid baselines, we conducted a controlled comparison against IMPALA under identical conditions, confirming the substantial performance gap (Table 10), but the PPO comparison relies on SB3 Zoo reference values [32] rather than identical conditions, and a controlled comparison against DreamerV3 remains for future work.

Regarding environment scope, MiniGrid is a 2D grid-world with discrete observations, and transfer to 3D environments, continuous-control tasks, and real robotic systems is unvalidated.

The ablation study removes one component at a time, so pairwise interaction effects (e.g., whether FM helps more or less when DNC is present) are not characterized.

For statistical reporting, the initial MiniGrid experiments (Table 7, Table 8 and Table 9) and the full architecture evaluation (Table 11) report single-seed results; multi-seed evaluation is ongoing. The IMPALA comparison uses 3 seeds with standard deviations, and the PAMAP2 encoder comparison (Table 13) uses 3 seeds.

The health management demonstration uses simulated rewards based on physiological heuristics, and clinical validation with domain experts is essential before deployment.

Finally, the FNO guidance layer and DNC memory show modest contributions in MiniGrid (Table 9), where the environment lacks strong temporal delays and long-horizon dependencies; their full potential is hypothesized to emerge in more complex domains, which remains to be validated.

Appendix A.1. Architecture and Caching

We employ Qwen2.5-3B-Instruct [24] as the mission parser. Given a mission string m, the model receives structured prompts and produces a JSON response from which four elements are extracted:

$$(g_{\mathrm{final}},{\left\{g_i\right\}}_{i=1}^{n_g},{\left\{a_j\right\}}_{j=1}^{n_a},{\left\{o_k\right\}}_{k=1}^{n_o})=\mathrm{Parse}\left(\mathrm{Qwen}2.5\left(m\right)\right)$$

(A1)

where $g_{\mathrm{final}}$ is the final goal, $\left\{g_i\right\}$ are intermediate goals, $\left\{a_j\right\}$ is the action sequence, and $\left\{o_k\right\}$ are mentioned objects.

An LRU cache (capacity 1,000, keyed by MD5 hash) avoids redundant inference for identical missions, achieving >99% cache hit rates in practice. When LLM inference fails, a regex-based fallback parser ensures robustness:

$$\mathcal{P}\left(m\right)=\left\{\begin{array}{cc}\mathrm{Qwen}2.5\left(m\right)\hfill & \mathrm{if}\mathrm{LLM}\mathrm{succeeds}\hfill \\ \mathrm{Regex}\left(m\right)\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(A2)

Appendix A.2. Integration into Reward Computation

Parsing results drive the language-based reward:

$$R_{\mathrm{lang}}=s_{\mathrm{match}}·r_{\mathrm{scale}}+b_{\mathrm{goal}}+b_{\mathrm{prox}}$$

(A3)

where $s_{\mathrm{match}}$ is the semantic match between parsed goals and the current state, $b_{\mathrm{goal}}$ is a goal-reached bonus, and $b_{\mathrm{prox}}$ is a proximity bonus.

Appendix A.3. Parsing Accuracy

Table A1 compares parsing accuracy across mission formats. The LLM parser substantially outperforms the regex baseline, particularly for compound instructions (+25 pp), novel expressions (+48 pp), and multilingual inputs (+94 pp).

Table A1. Mission Parsing Accuracy

Mission Format	Regex	Qwen-1.5B	Qwen-3B
Simple instructions	95%	98%	99%
Compound instructions	72%	94%	97%
Novel expressions	45%	89%	93%
Multilingual	0%	91%	94%
Average	53%	93%	96%

Table A1. Mission Parsing Accuracy

Mission Format	Regex	Qwen-1.5B	Qwen-3B
Simple instructions	95%	98%	99%
Compound instructions	72%	94%	97%
Novel expressions	45%	89%	93%
Multilingual	0%	91%	94%
Average	53%	93%	96%

Appendix A.4. Effect on RL Performance

The parsing accuracy advantage translates to measurable improvements in downstream RL performance. Table A2 compares success rates at episode 2000 across the three parsers. The LLM parser yields higher success rates in both environments, with the improvement more pronounced in the complex N4-S5 environment (+6.2 pp for Qwen-3B over Regex), where compound missions require accurate decomposition into sub-goals.

Table A2. Effect of Parser on RL Performance (Episode 2000)

Environment	Parser	Success Rate	Avg. Steps
N2-S4	Regex	68.0%	45.2
	Qwen2.5-1.5B	70.5%	42.8
	Qwen2.5-3B	71.2%	41.5
N4-S5	Regex	42.3%	78.5
	Qwen2.5-1.5B	46.8%	73.2
	Qwen2.5-3B	48.5%	71.8

Table A2. Effect of Parser on RL Performance (Episode 2000)

Environment	Parser	Success Rate	Avg. Steps
N2-S4	Regex	68.0%	45.2
	Qwen2.5-1.5B	70.5%	42.8
	Qwen2.5-3B	71.2%	41.5
N4-S5	Regex	42.3%	78.5
	Qwen2.5-1.5B	46.8%	73.2
	Qwen2.5-3B	48.5%	71.8

Figure A1 shows the learning curves for the MultiRoom-N2-S4 environment with the three parser configurations. The LLM-based parsers (Qwen2.5-1.5B and 3B) achieve faster initial learning and higher asymptotic performance than the regex baseline, confirming that accurate mission parsing provides more effective dense reward signals from the early stages of training. The Qwen-3B model shows a slight advantage over Qwen-1.5B, consistent with its higher parsing accuracy (Table A1).

Figure A1. Learning curves for MultiRoom-N2-S4 with different mission parsers. The LLM-based parsers achieve faster convergence and higher asymptotic success rates than the regex baseline.

Figure A1. Learning curves for MultiRoom-N2-S4 with different mission parsers. The LLM-based parsers achieve faster convergence and higher asymptotic success rates than the regex baseline.

Appendix A.5. Effect of Time Penalty

The language reward system includes a time penalty $P_{\mathrm{time}}=\alpha ·(1+\rho /100)·t$ that discourages excessively long episodes, where $\alpha$ is a scale parameter, $\rho$ is the current success rate, and t is the elapsed steps. Figure A2 illustrates the effect of the time penalty coefficient $\alpha$ on the reward distribution.

With the default setting $\alpha =0.05$, a successful episode completing at step 145 can receive a reward as low as $-9.39$ due to the accumulated time penalty, creating a misleading signal where successful behavior is penalized. Reducing $\alpha$ to 0.02 alleviates this issue, yielding a reward distribution in which successful episodes receive consistently positive rewards while still discouraging unnecessarily long trajectories.

Figure A2. Effect of time penalty coefficient $\alpha$ on reward distribution. Reducing $\alpha$ from 0.05 to 0.02 prevents successful episodes from receiving negative total rewards due to excessive time penalties.

Figure A2. Effect of time penalty coefficient $\alpha$ on reward distribution. Reducing $\alpha$ from 0.05 to 0.02 prevents successful episodes from receiving negative total rewards due to excessive time penalties.

Appendix A.6. Computational Cost

Table A3 shows that the LRU cache reduces effective LLM inference time to <1 ms, limiting the total training time overhead to approximately 10%.

Table A3. Computational Cost of Mission Parsing

Method	Inference	Memory	Cache Hit	Total Time
	(ms/ep)	(GB)	Rate	(h/2000ep)
Regex	0.1	0.1	—	2.5
Qwen-1.5B	35.2 (0.3*)	2.1	99.2%	2.8
Qwen-3B	52.8 (0.5*)	4.2	99.1%	3.1
*Effective time on cache hit.

Table A3. Computational Cost of Mission Parsing

Method	Inference	Memory	Cache Hit	Total Time
	(ms/ep)	(GB)	Rate	(h/2000ep)
Regex	0.1	0.1	—	2.5
Qwen-1.5B	35.2 (0.3*)	2.1	99.2%	2.8
Qwen-3B	52.8 (0.5*)	4.2	99.1%	3.1
*Effective time on cache hit.

Appendix A.7. Parsing Examples and Failure Cases

Table A4 shows representative parsing outputs. The LLM parser correctly extracts goals and actions from compound instructions with nested sub-goals.

Table A4. Parsing Examples

Mission	Final Goal / Interm.	Actions
“traverse the rooms to get to the goal”	goal / [room]	[traverse, get]
“pick up the blue key and open the blue door”	door / [key]	[pick, open]
“find the yellow key then unlock the door to reach the goal”	goal / [key, door]	[find, unlock, reach]

Table A4. Parsing Examples

Mission	Final Goal / Interm.	Actions
“traverse the rooms to get to the goal”	goal / [room]	[traverse, get]
“pick up the blue key and open the blue door”	door / [key]	[pick, open]
“find the yellow key then unlock the door to reach the goal”	goal / [key, door]	[find, unlock, reach]

The following failure cases were observed: (1) extremely long mission descriptions (>100 words), (2) ambiguous instructions (e.g., “do something interesting”), and (3) references to concepts absent from the environment (e.g., “fly to the ceiling”). In all cases, the regex fallback maintains system stability.

Appendix D.1. Conditional Flow Matching Formulation

The flow is defined by the ODE:

$${\displaystyle \frac{d{\psi }_t\left(x\right)}{dt}}=v_t\left({\psi }_t\left(x\right)\right),{\psi }_0\left(x\right)=x$$

(A5)

The CFM training loss for modality m is:

$${\mathcal{L}}_{\mathrm{CFM}}^m={\mathbb{E}}_{t,z,x_0}\left[\parallel v_{\theta }^m(x_t,t)-(z-x_0){\parallel }^2\right]$$

(A6)

Appendix D.2. FNO Spectral Convolution

The spectral convolution applies FFT, complex multiplication by learnable weights $R_k$, and IFFT:

$$\begin{array}{cc}\hfill {\widehat{E}}_k& =\mathrm{FFT}{\left(E\right)}_k\hfill \end{array}$$

(A7)

$$\begin{array}{cc}\hfill E_{\mathrm{fused}}& =\mathrm{GELU}\left(\mathrm{LN}(\mathrm{IFFT}\left(R_k{\widehat{E}}_k\right)+WE)\right)+E\hfill \end{array}$$

(A8)

with smoothness regularization:

$${\mathcal{L}}_{\mathrm{smooth}}={\lambda }_s{\displaystyle \sum _{m=1}^{M-1}}{\parallel E[m,:]-E[m+1,:]\parallel }^2$$

(A9)

Appendix D.3. Phase-Shift Representation

The cross-correlation delay $\tau$ between modalities appears as:

$$\mathcal{F}\left\{{\rho }_{12}\right\}\left(\omega \right)=S_{12}\left(\omega \right)·e^{-j\omega \tau }$$

(A10)

Appendix D.4. Approximation Bound

The pre-fusion mutual information bound:

$$\begin{array}{c}I\left((X^1,\dots ,X^M);f({\widehat{X}}^1,\dots ,{\widehat{X}}^M)\right)\hfill \\ \hfill \ge I_{\mathrm{post}-\mathrm{fusion}}-\mathcal{O}(\sum _m\parallel {\delta }^m{\parallel }^2/{\sigma }_m^2)\end{array}$$

(A11)

See Appendix J for the full derivation.

Appendix E.1. Vision Encoder

The frozen CLIP ViT-B/32 [11] produces patch features $F_{\mathrm{patch}}\in {\mathbb{R}}^{49\times 768}$, projected via $F_{\mathrm{vision}}=W_{\mathrm{proj}}{F}_{\mathrm{patch}}+b_{\mathrm{proj}}\in {\mathbb{R}}^{49\times d}$. Slot Attention [12] decomposes these into K slots through iterative competitive assignment:

$$\begin{array}{cc}\hfill A_{ij}& ={\displaystyle \frac{exp\left(\frac{1}{\sqrt{d}}{\left(W_q{s}_i\right)}^{\top }\left(W_k{F}_j\right)\right)}{{\sum }_{i^{\prime }}exp\left(\frac{1}{\sqrt{d}}{\left(W_q{s}_{i^{\prime }}\right)}^{\top }\left(W_k{F}_j\right)\right)}}\hfill \end{array}$$

(A12)

$$\begin{array}{cc}\hfill s_i^{\prime }& =\mathrm{GRU}\left(\sum _j{\displaystyle \frac{A_{ij}}{{\sum }_{j^{\prime }}{A}_{i{j}^{\prime }}}}{W}_v{F}_j,s_i\right)\hfill \end{array}$$

(A13)

Numerical stabilization techniques: (1) Mask values use $-{10}^9$ instead of $-\infty$. (2) NaN fallback substitutes uniform $\mathbf{1}/K$. (3) GRU weights: Xavier (gain 0.5) + Orthogonal (gain 0.5). (4) Variance clipping: $\sigma =max(exp(log{\sigma }_{\mathrm{raw}}),{10}^{-6})$.

Appendix E.2. Language, State, and Reward Encoders

Language: A Transformer encoder [21] produces $H_{\mathrm{lang}}=\mathrm{TransformerEnc}(\mathrm{Embed}\left(T\right)+\mathrm{PE})$. Slot Attention [12] is applied to decompose mission components.

State: FNO block [16] with spectral convolution $e_{\mathrm{state}}=\mathrm{IFFT}(R_k·\mathrm{FFT}\left(o\right))+W·o$.

Reward: $e_{\mathrm{reward}}=\mathrm{ReLU}\left(\mathrm{Conv}1\mathrm{D}\left(r_{t-H:t}\right)\right)$.

Appendix E.3. Cross-Modal Attention

Bidirectional multi-head attention [21] with residual connections:

$$\begin{array}{cc}\hfill S_{\mathrm{vis}}^{\prime }& =S_{\mathrm{vis}}+\mathrm{MHA}(S_{\mathrm{vis}},S_{\mathrm{lang}},S_{\mathrm{lang}})\hfill \end{array}$$

(A14)

$$\begin{array}{cc}\hfill S_{\mathrm{lang}}^{\prime }& =S_{\mathrm{lang}}+\mathrm{MHA}(S_{\mathrm{lang}},S_{\mathrm{vis}},S_{\mathrm{vis}})\hfill \end{array}$$

(A15)

Appendix F.1. DNC Memory

The DNC [17] uses content-based addressing: $w_c\left(i\right)=\mathrm{softmax}(\beta ·cos(k,M\left[i\right]))$. Write: $M^{\prime }=M\odot (1-w_w{\mathbf{e}}^{\top })+w_w{\mathbf{a}}^{\top }$. Read: $r={\sum }_i{w}_r\left(i\right)M^{\prime }\left[i\right]$. Output: $e_{\mathrm{aug}}=\mathrm{GELU}(W_{\mathrm{out}}[e_{\mathrm{fused}};r]+b)$.

Appendix F.2. PPO Objective

The PPO [23] clipped surrogate: ${\mathcal{L}}_{\mathrm{PPO}}=-\mathbb{E}[min({\rho }_t{\widehat{A}}_t,\mathrm{clip}({\rho }_t,1-ϵ,1+ϵ){\widehat{A}}_t)]$ with $ϵ=0.2$ and GAE [38] for advantage estimation.

Appendix G.1. Language-Grounded Reward

Three-component reward [24]: $R_{\mathrm{total}}={\lambda }_1{R}_{\mathrm{lang}}+{\lambda }_2{R}_{\mathrm{VL}}+{\lambda }_3{R}_{\mathrm{intrinsic}}$. $R_{\mathrm{lang}}$: semantic match between LLM-parsed goals and observations. $R_{\mathrm{VL}}$: cosine similarity [11] between vision and language FM embeddings. $R_{\mathrm{intrinsic}}$: prediction-error curiosity reward [39].

Appendix G.2. Adaptive Reward Shaping

Following the potential-based reward shaping framework [26]: $r_{\mathrm{shaped}}=r_{\mathrm{env}}+max(1-1.25\overline{S},0)·r_{\mathrm{bonus}}$, where $\overline{S}$ is the recent success rate.

Appendix G.3. Success Buffer

Inspired by Hindsight Experience Replay [27], successful episodes are stored and replayed at 25% mix ratio during training to prevent catastrophic forgetting.

Appendix G.4. Score-Based Adaptive Complexity

Using the score function from score-based generative models [28]: Complexity $={\mathbb{E}}_t[\parallel s_{\theta }(x,t,c){\parallel }^2]$ determines the number of FM steps via Gumbel-Softmax [40] selection from $\{1,2,3,5,7,10\}$.

Appendix H.1. Addressing RSSM Limitations via Input-Side Design

The standard RSSM architecture [1] processes observations through a single encoder before the deterministic–stochastic state transition, which introduces three structural limitations for multimodal sensor-driven applications. First, a single encoder conflates modalities with fundamentally different noise profiles (e.g., sparse reward signals and high-frequency IMU readings), losing modality-specific structure. Second, the GRU-based deterministic pathway provides only short-term memory, insufficient for recalling episodic patterns such as prior heart-rate spikes across activity transitions. Third, standard real-valued fusion (concatenation or attention) cannot represent phase-shifted cross-modal correlations, such as the several-second delay between physical movement (IMU) and heart-rate elevation.

Our approach addresses these limitations not by modifying the RSSM’s internal dynamics, but by improving the quality of the observation representation that enters the RSSM posterior. Per-modality Flow Matching preserves modality-specific noise characteristics by denoising each signal independently before fusion. FNO spectral fusion encodes phase-shifted correlations through learnable complex-valued spectral weights (Eq. (A10)). DNC external memory supplements the GRU’s short-term state with content-addressable episodic recall. This input-side design is complementary to approaches that modify the RSSM itself, such as MedDreamer’s Adaptive Feature Integration module [8] for irregular clinical time series, and could in principle be combined with such internal modifications.

Appendix I.1. RSSM World Model Architecture

The world model follows the RSSM of DreamerV3 [1], representing the latent state $s_t$ as the concatenation of a deterministic component $h_t$ (256 dimensions) and a stochastic component $z_t$ (64 dimensions), yielding a 320-dimensional state vector (Figure A3).

Figure A3. RSSM world model for PAMAP2 health management. Top: temporal unrolling showing the deterministic pathway $h_t$ (GRU transitions) and stochastic pathway $z_t$ (prior/posterior distributions). Bottom: four-phase training pipeline with experimental results.

Figure A3. RSSM world model for PAMAP2 health management. Top: temporal unrolling showing the deterministic pathway $h_t$ (GRU transitions) and stochastic pathway $z_t$ (prior/posterior distributions). Bottom: four-phase training pipeline with experimental results.

The deterministic pathway retains temporal context of activity patterns through a GRU:

$$h_{t+1}={\mathrm{GRU}}_{\theta }\left(h_t,[z_t;e\left(a_t\right)]\right)$$

(A16)

The stochastic pathway models uncertainty through prior and posterior distributions:

$$\begin{array}{cc}\hfill \mathrm{Prior}\text{:}{p}_{\theta }(z_t\mid h_t)& =\mathcal{N}\left({\mu }_{\theta }\left(h_t\right),{\sigma }_{\theta }\left(h_t\right)\right)\hfill \end{array}$$

(A17)

$$\begin{array}{cc}\hfill \mathrm{Posterior}\text{:}{q}_{\varphi }(z_t\mid h_t,o_t)& =\mathcal{N}\left({\mu }_{\varphi }(h_t,o_t^{\mathrm{embed}}),{\sigma }_{\varphi }(h_t,o_t^{\mathrm{embed}})\right)\hfill \end{array}$$

(A18)

where $o_t^{\mathrm{embed}}$ is the output of the foundation encoder (Section 5). The posterior conditions on observations via the encoder, while the prior enables imagination without observations.

Appendix I.2. Foundation Encoder Details

The observation $o_t$ consists of IMU sensors at three body locations (accelerometer, gyroscope, magnetometer; 36 dimensions) and heart-rate-related features (16 dimensions). The foundation encoder applies the four-stage pipeline:

Stage 1 (Slot Attention): The three IMU sites ($3\times 12$ dimensions) are treated as slots and processed by Self-Attention, then fused with heart rate features via Cross-Attention:

$$o_t^{\mathrm{slot}}=\mathrm{CrossAttn}\left(\mathrm{SelfAttn}\left({\mathrm{IMU}}_t\right),\mathrm{MLP}\left({\mathrm{HR}}_t\right)\right)$$

(A19)

Stage 2 (Per-Modality Flow Matching): Independent FM modules refine the IMU and HR representations before fusion. Without a pre-trained encoder (no CLIP equivalent for IMU data), the FM refinement compensates for the raw encoder’s noise—this is the setting where filter-before-mixing is most critical.

Stage 3 (FNO Spectral Fusion): The refined modality representations are fused via SpectralConv1d. The phase-shift property of the FNO (Eq. (A10)) is particularly relevant here, as physical movement (IMU) precedes heart rate elevation by several seconds.

Stage 4 (DNC Memory): The fused representation is augmented with episodic memory via content-based addressing [17], enabling the agent to recall past physiological episodes (e.g., a prior heart-rate spike that triggered a rest recommendation).

The final encoded observation $o_t^{\mathrm{embed}}$ enters the RSSM posterior (Eq. (A18)).

Appendix I.3. World Model Training

The world model is trained by minimizing:

$${\mathcal{L}}_{\mathrm{WM}}={\mathcal{L}}_{\mathrm{rec}}+{\beta }_d{D}_{\mathrm{KL}}(q_{\varphi }\parallel p_{\theta })+{\mathcal{L}}_{\mathrm{rew}}+{\mathcal{L}}_{\mathrm{con}}$$

(A20)

where ${\mathcal{L}}_{\mathrm{rec}}$ reconstructs observations (MSE on IMU + HR features), ${\mathcal{L}}_{\mathrm{rew}}$ predicts rewards (health outcomes), ${\mathcal{L}}_{\mathrm{con}}$ predicts episode continuation (activity transitions), and ${\beta }_d=1.0$ balances KL regularization.

Appendix I.4. Imagination-Based Policy Optimization

Using only the prior distribution, imagined rollouts of horizon $H=15$ are generated to optimize the policy without real-world interaction:

$$\begin{array}{cc}\hfill {\widehat{h}}_{t+1}& ={\mathrm{GRU}}_{\theta }({\widehat{h}}_t,[{\widehat{z}}_t;e\left({\widehat{a}}_t\right)])\hfill \\ \hfill {\widehat{z}}_{t+1}& \sim p_{\theta }(·\mid {\widehat{h}}_{t+1}),{\widehat{a}}_t\sim {\pi }_{\psi }(·\mid {\widehat{s}}_t)\hfill \end{array}$$

(A21)

The policy ${\pi }_{\psi }$ and value function $V_{\xi }$ are updated via $\lambda$-returns computed over imagined trajectories.

The reward function combines physiological targets (heart rate within safe range, activity level maintenance, fatigue prevention) with safety constraints (penalizing prolonged exposure to dangerous heart rate zones >160 bpm). This imagination-based training is particularly valuable for health management: learning appropriate Alert actions requires repeated exposure to high-risk states, which is ethically and practically infeasible in real patients. The world model generates such scenarios safely in the latent space.

Appendix I.5. Physiological Anomaly Detection

As a byproduct of the world model, the KL divergence $D_{\mathrm{KL}}(q_{\varphi }\parallel p_{\theta })$ at each time step quantifies how much the actual sensor observation deviates from the world model’s prediction. Time steps where $D_{\mathrm{KL}}>{\mu }_D+2{\sigma }_D$ (where ${\mu }_D$ and ${\sigma }_D$ are the running mean and standard deviation of KL values) are flagged as anomaly candidates, indicating unexpected heart rate elevation or atypical fatigue patterns. This mechanism provides an additional safety layer not present in MedDreamer or KANDI (Table A6).

Appendix I.6. Positioning Among Health Intervention Systems

Table A6 compares our approach with recent world model-based health intervention systems. Direct numerical comparison is not meaningful across different clinical domains; the table highlights architectural differences.

Table A6. Comparison with World Model-Based Health Intervention Systems

	MedDreamer [8]	KANDI [9]	DreamerV3 [1]	FreamerV1 (ours)
Domain	EHR (Sepsis)	Wearable (fall risk)	General RL	Wearable (health)
World model	RSSM+AFI	None	RSSM	RSSM+Foundation
Policy	Actor-Critic	Diffusion	Actor-Critic	PPO
Learning	Online+Imag.	Offline IRL	Online+Imag.	Online+Imag.
Encoding	AFI	MLP	CNN/MLP	SlotAttn+FM+FNO
Anomaly	—	—	—	KL divergence

Table A6. Comparison with World Model-Based Health Intervention Systems

	MedDreamer [8]	KANDI [9]	DreamerV3 [1]	FreamerV1 (ours)
Domain	EHR (Sepsis)	Wearable (fall risk)	General RL	Wearable (health)
World model	RSSM+AFI	None	RSSM	RSSM+Foundation
Policy	Actor-Critic	Diffusion	Actor-Critic	PPO
Learning	Online+Imag.	Offline IRL	Online+Imag.	Online+Imag.
Encoding	AFI	MLP	CNN/MLP	SlotAttn+FM+FNO
Anomaly	—	—	—	KL divergence

Our system shares with MedDreamer the paradigm of RSSM-based imagination for safe policy learning, but addresses a different data regime: continuous, high-frequency, multi-site wearable signals rather than sparse, irregular EHR records. Compared to KANDI, our approach learns an explicit world model and optimizes policies through imagination, whereas KANDI operates offline with pre-collected expert demonstrations.

Appendix I.7. Generalization Across Subjects

After pre-training the world model on data from multiple subjects, the policy can be adapted to new subjects with minimal data. While inter-subject variability exists in resting heart rate and cardiopulmonary capacity, higher-level decision structures—such as recommending rest under high exertion or alerting upon fatigue accumulation—are shared in the latent space of the world model, enabling few-shot adaptation.