CM-S6: Cross-Modal Selective Scan via Parameter-Level Modulation for Multisource Remote Sensing

1. Introduction

High-precision Earth observation increasingly benefits from the complementary information provided by multimodal remote sensing data. Hyperspectral imagery (HSI) records fine-grained spectral responses across continuous narrow bands, synthetic aperture radar (SAR) captures surface structure and scattering properties, and light detection and ranging (LiDAR) and digital surface models (DSMs) provide three-dimensional geometric information [1,2,3,4,5]. These modalities observe the same land-cover targets from different perspectives, but each is affected by modality-specific limitations. HSI can suffer from spectral confusion in mixed pixels or high-reflectance regions, SAR is sensitive to speckle noise and shadow effects, and LiDAR or DSM responses may become less reliable beneath vegetation canopies or over complex terrain. Multimodal remote sensing classification therefore needs to select and combine complementary evidence according to scene content and spatial context rather than simply aggregate the input modalities.

Existing multimodal remote sensing fusion methods mainly use CNN, Transformer, or Mamba backbones. CNNs are effective at extracting local spatial structures, but may require enlarged receptive fields or deeper operations to model long-range relationships. Transformers establish global token interactions through cross-attention; however, standard global self-attention has quadratic computational and memory scaling with sequence length. Mamba updates hidden states through selective scanning and makes the state-update process input-dependent, enabling long-range dependency modeling with linear scaling in sequence length. MambaHSI [6] and S²-Mamba [7] demonstrate SSM effectiveness for hyperspectral classification; HLMamba [8] and MCAMamba [9] extend SSMs to multisource classification. However, these methods fuse modalities before or outside selective scanning through concatenation, weighting, or attention so auxiliary information alters only the scan input and cannot separately influence state retention (Δ), input writing (B), or state readout (C). Strengthening such indirect interaction typically requires extra attention modules or larger capacity.

Selective SSMs generate token-dependent Δ, B, and C an internal interface that auxiliary information can exploit to influence state evolution directly. Standard S6 generates these parameters solely from the current-modality input, forcing auxiliary information to act indirectly through pre-fused features. We instead condition Δ, B, and C on co-registered auxiliary tokens, extending multimodal fusion from the representation layer to the parameter-generation layer of selective SSMs.

The key question is therefore how auxiliary modalities can move beyond indirect representation-level transmission and participate in distinct selective state-evolution functions while controlling model overhead. To address this question, we propose Cross-Modal Selective Scan (CM-S6): lightweight projections map auxiliary tokens into modulation signals for Δ, B, and C, complemented by cross-modal input injection (CII) before scanning and output calibration after scanning. Five bounded coefficients coordinate this input–parameter–output pathway. A lightweight variant, CM-S6(L), retains all modulation paths while simplifying scan branches for resource-constrained deployment, trading some classification accuracy for greater resource efficiency. Full CM-S6 and CM-S6(L) thus provide accuracy-oriented and efficiency-oriented configurations for different computational budgets.

The main contributions of this work are summarized as follows.

• We propose CM-S6, which conditions the Δ/B/C parameter-generation pathways of selective SSMs on auxiliary-modality signals via lightweight projections, enabling parameter-level cross-modal modulation without widening or deepening the backbone.
• We design the parameter-modulation mechanism with pathway-specific operations: multiplicative perturbation for Δ and additive corrections for B and C, preserving the primary-modality generation baseline while assigning auxiliary information distinct roles in state updating, input writing, and state readout.
• We integrate CII, Δ/B/C modulation, and output calibration into a three-stage mechanism coordinated by five bounded learnable coefficients. Ablations confirm that input/output modulation and internal parameter modulation are complementary, with dataset-dependent relative contributions.
• We develop CM-S6(L), which retains all five modulation paths at ~0.19 M parameters with 3.86–5.82× throughput, providing an efficiency-oriented counterpart to the accuracy-oriented full model.

Figure 1. Conceptual comparison between the standard S6 module and the proposed CM-S6 module.

Figure 1. Conceptual comparison between the standard S6 module and the proposed CM-S6 module.

2. Related Work

2.1. Multisource Remote Sensing Fusion Methods

The goal of multisource remote sensing fusion is to construct complementary representations across modalities. Early CNN-based methods integrated HSI, LiDAR, SAR, and DSM data through multibranch feature extraction, feature concatenation, gated weighting, or multiscale fusion [5,10]. Although effective at local modeling, their fixed receptive fields limit adaptation to modality-specific physical responses, and interaction is generally confined to feature concatenation or channel weighting. Transformers extend fusion to global token interactions [11,12]. MFT organizes modalities as parallel token streams and uses self-attention to model long-range dependencies and cross-modal relationships [13]. PICNet combines prototype-based information compensation with cross-modal feature interaction for joint HSI–LiDAR classification, effectively capturing local discriminative structures and class prototypes [14,15,16]. These methods improve global contextual modeling, but cross-attention and multibranch token processing increase computational and memory costs, while fusion still operates on formed feature representations. Mamba offers a linear-complexity alternative. MSFMamba uses multiscale feature fusion within an SSM backbone for multisource classification [17]. MambaHSI and S²-Mamba demonstrate the effectiveness of SSMs for spatial–spectral hyperspectral modeling [6,7], while HLMamba and MCAMamba extend Mamba to multisource classification [8,9]. MCAMamba achieves strong classification performance by using multilevel cross-modal attention to enhance modality complementarity, but retains feature-level interaction. MTMixer combines Mamba with Transformer/Mixer-style token processing to capture sequential and global dependencies at a relatively high parameter and storage cost [18]. Despite this progression, cross-modal interaction in existing Mamba methods operates at the feature or hidden-state level; the Δ/B/C parameter-generation pathways remain underexploited for cross-modal modulation.

2.2. State Space Models in Remote Sensing

State space models (SSMs) originate from continuous-time dynamical systems. S4 [19,20,21] and Mamba [22] establish structured and selective state-space modeling; Vision Mamba [23,24] extends SSMs to 2D features. In remote sensing, MambaHSI [6], S²-Mamba [7], SpectralMamba [25], and related methods validate SSMs for spatial–spectral classification. These methods treat SSMs as efficient backbones; Δ/B/C selectivity is driven by a single stream and has not been used as a cross-modal fusion interface. Rather than replacing the Mamba backbone, this work introduces auxiliary-modality signals into these parameter-generation pathways so that SSM selectivity participates directly in multisource fusion.

2.3. Taxonomy of Cross-Modal Fusion Paradigms

According to where cross-modal information enters the SSM, multimodal Mamba fusion methods can be categorized into feature-level, hidden-state-level, and parameter-level fusion, as illustrated in Figure 2. Feature-level fusion (e.g., HLMamba [8], MCAMamba [9]) combines modalities before scanning via concatenation or attention, leaving Δ/B/C driven by the fused input. Hidden-state-level fusion (e.g., Coupled Mamba [27], a visual-language model not directly comparable in our remote-sensing setting) couples intermediate states during recurrence but does not alter parameter generation. Parameter-level fusion our approach conditions Δ/B/C directly on auxiliary signals, changing how the SSM responds to each input rather than only changing its inputs or intermediate states. Appendix A discusses this distinction from a function-space perspective.

2.4. Granularity of Cross-Modal Interaction

Cross-modal fusion can also be compared according to the processing stage and granularity at which auxiliary information affects a model. Logit-level methods such as TMC and Evidential Deep Learning combine modalities at the network output through evidence distributions or confidence weighting [28,29]. Because they operate on final class probabilities, they provide limited access to intermediate state processing. Dynamic, conflict-aware, and uncertainty-aware multimodal methods further adapt output- or representation-level interaction to cross-modal discrepancies [30,31,32]. Feature-level methods use cross-attention, token weighting, or channel recalibration to modify formed representations. MFT, for example, establishes fine-grained cross-modal relationships through token interaction [13]. These mechanisms provide local feature adaptation but do not explicitly distinguish how auxiliary information affects state updating, information writing, and state readout. Parameter-level interaction instead conditions the generation of Δ, B, and C. CM-S6 uses this interface to assign auxiliary information to the three state-space functions while complementing internal parameter modulation with input injection and output calibration.

3. Methods

CM-S6 assumes co-registered multimodal inputs. Registration, resampling, and cropping place observations of the same ground location at consistent tensor coordinates across modalities. After lightweight modality-specific encoding, the resulting two-dimensional feature maps are unfolded under the same deterministic scan order, so primary- and auxiliary-modality tokens at a common sequence index retain their spatial correspondence, as expressed in (1).

$${\mathrm{x}}_{\mathrm{t}}^{\mathrm{M}}\leftrightarrow {\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}}\leftrightarrow \mathrm{p}\left(\mathrm{t}\right),\hspace{1em}\mathrm{t}=1,\dots ,\mathrm{L}.$$

(1)

Here, $X_t^M$and $X_t^A$denote the primary- and auxiliary-modality tokens at the t-th scan position, respectively, and p(t) denotes the corresponding two-dimensional coordinate. The shared ordering does not by itself align the semantic distributions of different modalities, but it preserves the position-wise correspondence required for cross-modal conditioning. On this basis, CM-S6 conditions the Δ/B/C parameter-generation pathways on auxiliary signals, as detailed below.

3.1. The Selective State Space Model and Cross-Modal Parameter Generation

A state space model represents a continuous-time sequence as the input-driven evolution of a hidden state. Given an input x(t), hidden state h(t), and output y(t), its continuous form is given in (2):

$$\mathrm{h}\mathrm{\text{'}}\left(\mathrm{t}\right)=\mathrm{A}\cdot \mathrm{h}\left(\mathrm{t}\right)+\mathrm{B}\cdot \mathrm{x}\left(\mathrm{t}\right),\hspace{1em}\mathrm{y}\left(\mathrm{t}\right)=\mathrm{C}\cdot \mathrm{h}\left(\mathrm{t}\right)$$

(2)

Here, A governs the intrinsic evolution of the hidden state, B controls how the input is written into the state space, and C controls how the hidden state is read out. S4 improves long-sequence modeling through structured parameterization and discretization. Mamba further introduces selectivity by making B, C, and the discretization step Δ dynamic functions of the current input token, as shown in (3):

$${\mathrm{B}}_{\mathrm{t}}={\mathrm{S}}_{\mathrm{B}}({\mathrm{x}}_{\mathrm{t}}),{\mathrm{C}}_{\mathrm{t}}={\mathrm{S}}_{\mathrm{C}}({\mathrm{x}}_{\mathrm{t}}){\mathsf{\Delta }}_{\mathrm{t}}=\mathrm{softplus}\left({\mathsf{\tau }}_{\mathsf{\Delta }}+{\mathrm{S}}_{\mathsf{\Delta }}\left({\mathrm{x}}_{\mathrm{t}}\right)\right)$$

(3)

Zero-order-hold discretization gives (4), and the resulting discrete recurrence is defined in (5):

$${\stackrel{\mathrm{̄}}{\mathrm{A}}}_{\mathrm{t}}=\exp ({\mathsf{\Delta }}_{\mathrm{t}}\text{⋅}\mathrm{A}){\stackrel{\mathrm{̄}}{\mathrm{B}}}_{\mathrm{t}}=({\mathsf{\Delta }}_{\mathrm{t}}\text{⋅}\mathrm{A}{)}^{-1}\left(\exp ({\mathsf{\Delta }}_{\mathrm{t}}\text{⋅}\mathrm{A})-\mathrm{I}\right)\text{⋅}{\mathsf{\Delta }}_{\mathrm{t}}\text{⋅}{\mathrm{B}}_{\mathrm{t}}$$

(4)

$${\mathrm{h}}_{\mathrm{t}}=\stackrel{\mathrm{̄}}{\mathrm{A}}\mathrm{t}\text{⋅}{\mathrm{h}}_{\mathrm{t}-1}+{\stackrel{\mathrm{̄}}{\mathrm{B}}}_{\mathrm{t}}\text{⋅}{\mathrm{x}}_{\mathrm{t}},{\mathrm{y}}_{\mathrm{t}}={\mathrm{C}}_{\mathrm{t}}\text{⋅}{\mathrm{h}}_{\mathrm{t}}$$

(5)

Equations (2)–(5) show that S6 is an input-adaptive recurrence: Δ, B, and C jointly control state retention, input writing, and state readout at each token position. In standard S6, Δ/B/C are generated solely from the primary input. CM-S6 instead conditions them on co-registered auxiliary tokens (Section 3.3), enabling parameter-level cross-modal modulation.

3.2. Overall Design of CM-S6

CM-S6 takes a pair of aligned modality sequences as input. Let the primary-modality sequence be $X^M=\{x_1^M,x_2^M,\mathrm{...},x_L^M\}$, with $X_t^M\in R^D$, and the auxiliary-modality sequence be $X^A=\{x_1^A,x_2^A,\mathrm{...},x_L^A\}$, with $X_t^A\in R^D$. The two sequences are generated by separate lightweight encoding branches and unfolded under the same scan order, producing one-dimensional sequences of equal length. CM-S6 outputs an enhanced sequence from the primary-modality perspective, $Y^M=\{y_1^M,y_2^M,\mathrm{...},y_L^M\}$, where $y_t^M\in R^D$. The primary modality provides the main stream for state-space modeling, while the auxiliary modality modulates input injection, parameter generation, and output calibration. These roles are not permanently assigned to particular sensors. Depending on the network topology, HSI, LiDAR, SAR, or DSM may serve as the primary or auxiliary modality in different modules, enabling alternating modulation in dual- or tri-modal configurations. Cross-modal information enters through five paths: CII (η) before the SSM, Δ/B/C modulation (α/β/γ) inside parameter generation, and output calibration (ζ) after the SSM. These form a three-stage input–parameter–output chain; Figure 3 shows the overall CM-S6 architecture, and Section 3.3 details the corresponding operations.

3.3. The Five-Coefficient Cross-Modal Modulation Mechanism

Building on the primary–auxiliary interaction framework above, this section specifies where the five learnable scalar coefficients act during the CM-S6 forward pass. Let$X_t^M$, $X_t^A\in R^D$denote the primary- and auxiliary-modality features at position t, respectively, where D is the channel dimension. The symbol${\tilde{X}}_t^M$denotes the primary feature after cross-modal input injection, and$y_t^M$denotes the primary-modality output of CM-S6. The coefficients η, α, β, γ, and ζ control input injection, step-size modulation, input-writing modulation, state-readout modulation, and output calibration, respectively.

• Cross-Modal Input Injection: CII acts on the primary-modality stream before the SSM parameter-generation functions are evaluated, as defined in (6):

$${\stackrel{\mathrm{~}}{\mathrm{x}}}_{\mathrm{t}}^{\mathrm{M}}={\mathrm{x}}_{\mathrm{t}}^{\mathrm{M}}+\widehat{\mathsf{\eta }}\text{⋅}\tan \mathrm{h}\left({\mathrm{W}}_{\mathrm{in}}\mathrm{LN}({\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}})\right)$$

(6)

Here, Wᵢₙ is a lightweight linear projection that aligns the auxiliary feature with the primary channel dimension, and tanh bounds the injected response. Equation (6) preserves the original primary input and adds a bounded correction, so the subsequent Δ, B, and C generation functions operate on the cross-modally calibrated feature ${\tilde{X}}_t^M$. The projected auxiliary response determines token- and channel-specific variations, whereas η controls the overall strength of the injection path. A more discriminative auxiliary response produces a stronger bounded correction, while an uninformative response produces a weaker adjustment.

• Step-Size Modulation: The Δ path applies a multiplicative perturbation to the discretization step. The primary branch produces the original step-size signal Δself through a linear projection of ${\tilde{X}}_t^M$

, while LayerNorm and a separate projection of $X_t^A$ produce the cross-modal perturbation Δcross, as defined in (7):

$${\mathsf{\Delta }}_{\mathrm{t}}^{\mathrm{self}}={\mathrm{W}}_{\mathrm{dt}}{\stackrel{\mathrm{~}}{\mathrm{x}}}_{\mathrm{t}}^{\mathrm{M}},{\mathsf{\Delta }}_{\mathrm{t}}^{\mathrm{cross}}={\mathrm{W}}_{\mathrm{dt},\mathrm{cross}}\mathrm{LN}({\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}})$$

(7)

The modulated step is obtained by applying softplus to the multiplicatively adjusted signal, as defined in (8):

$${\mathsf{\Delta }}_{\mathrm{t}}=\mathrm{softplus}\left({\mathsf{\Delta }}_{\mathrm{self}}\text{⋅}\left[1+\mathsf{\alpha }\text{⋅}\tan \mathrm{h}({\mathsf{\Delta }}_{\mathrm{cross}})\right]\right)$$

(8)

This multiplicative form adjusts the relative scale of the primary step-size signal before softplus while retaining a positive output step. The factor 1 + α·tanh(Δcross) provides an upward or downward adjustment relative to the primary-modality baseline. Because Δ controls the discretization timescale, this path allows auxiliary features to alter the relative balance between integrating the current input and retaining historical state information.

3.

Input-Writing Matrix Modulation: The B path applies an additive cross-modal perturbation to the B generation function, as defined in (9):

$$\mathrm{B}={\mathrm{B}}_{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}+\mathsf{\beta }\cdot \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathrm{W}}_{\mathrm{B}}\cdot {\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}}\right),{\mathrm{B}}_{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}={\mathrm{S}}_{\mathrm{B}}\left({\tilde{\mathrm{x}}}_{\mathrm{t}}^{\mathrm{M}}\right)$$

(9)

Here, B_self is generated from the pre-aligned sequence produced by (6) through the original input-dependent function of S6. The additive formulation preserves the primary-modality component Bself while introducing a separate cross-modal writing component. Because B determines how input dimensions are written into the hidden state, (9) allows auxiliary information to condition the writing pattern at each token position.

4.

State-Readout Matrix Modulation: The C path adopts an additive structure symmetric to that of B, as defined in (10):

$$\mathrm{C}={\mathrm{C}}_{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}+\mathsf{\gamma }\cdot \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathrm{W}}_{\mathrm{C}}\cdot {\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}}\right),\hspace{1em}{\mathrm{C}}_{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}={\mathrm{S}}_{\mathrm{C}}\left({\tilde{\mathrm{x}}}_{\mathrm{t}}^{\mathrm{M}}\right)$$

(10)

C specifies the channel weights used to read the hidden state in $y_t=C_t\cdot h_t$. Equation (9) regulates how information is written, whereas (10) regulates how it is read out. The two paths therefore modulate the input and output sides of state-space information flow.

5.

Output-Level Residual Calibration: The ζ path is the most downstream of the five modulation paths and is defined in (11):

$${\mathrm{y}}_{\mathrm{t}}^{\mathrm{M}}\text{←}{\mathrm{y}}_{\mathrm{t}}^{\mathrm{M}}+\mathsf{\zeta }\text{⋅}\tan \mathrm{h}\left({\mathrm{W}}_{\mathrm{out}}\text{⋅}{\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}}\right)$$

(11)

Unlike CII and the Δ/B/C paths, which intervene in parameter generation or state evolution, ζ adds a projected auxiliary response after the selective scan. It provides an output-side correction when auxiliary semantics are not fully retained through the recurrent state representation. This path complements, rather than replaces, the internal parameter-level modulation.

6.

Coordination of the Five Coefficients: Equations (6)–(11) implement the three-stage chain: (6) input injection; (7)–(10) Δ/B/C parameter modulation; (11) output calibration. This grouping directly corresponds to the ablation design in Section 4.3. The bounded reparameterization is defined in (12), and Algorithm 1 summarizes the full forward pass.

Algorithm 1. CM-S6 Forward Pass for One Layer.

Input: Primary modality $x^M\in R^{（B\times N\times D）}$Auxiliary modality $x^A\in R^{（B\times N\times D）}$Learnable scalars α, β, γ, η, ζ with bounds αmax, βmax, γmax, ηmax, ζmax pretrained SSM parameters A, D Output: Primary modality output $y^M\in R^{(B\times N\times D)}$

1: Stage 0: Reparameterize five coefficients 2: for ρ in {α, β, γ, η, ζ} do$3\text{:}\mathsf{\rho }\text{̂}\text{←}\mathrm{clip}\left(\tan \mathrm{h}\right(\mathsf{\rho }),0,{\mathsf{\rho }}_{\max })$4: end for

5: Stage 1: Cross-Modal Input Injection $6\text{:}\mathrm{x}{\text{̃}}^{\mathrm{M}}\text{←}{\mathrm{x}}^{\mathrm{M}}+\mathsf{\eta }\text{̂}\text{·}\tan \mathrm{h}({\mathrm{W}}_{\mathrm{in}}\text{·}\mathrm{LN}({\mathrm{x}}^{\mathrm{A}}\left)\right)$

7: Stage 2: Generate Δ, B, C with cross-modal modulation 8: Δself ← Wdt · x̃ $9\text{:}{\mathsf{\Delta }}_{\mathrm{cross}}\text{←}\mathrm{Dropout}({\mathrm{W}}_{(\mathrm{dt},\mathrm{c})}\text{·}\mathrm{LN}({\mathrm{x}}^{\mathrm{A}}\left)\right)$$10\text{:}\mathsf{\Delta }\text{←}\mathrm{Softplus}({\mathsf{\Delta }}_{\mathrm{self}}\text{·}[1+\mathsf{\alpha }\text{̂}\text{·}\tan \mathrm{h}({\mathsf{\Delta }}_{\mathrm{cross}}\left)\right])$$11\text{:}\mathrm{B}\text{←}{\mathrm{W}}_{\mathrm{B}}\text{·}\mathrm{x}\text{̃}+\mathsf{\beta }\text{̂}\text{·}\tan \mathrm{h}({\mathrm{W}}_{(\mathrm{B},\mathrm{c})}\text{·}\mathrm{LN}({\mathrm{x}}^{\mathrm{A}}\left)\right)$ $12\text{:}\mathrm{C}\text{←}{\mathrm{W}}_{\mathrm{C}}\text{·}\mathrm{x}\text{̃}+\mathsf{\gamma }\text{̂}\text{·}\tan \mathrm{h}({\mathrm{W}}_{(\mathrm{C},\mathrm{c})}\text{·}\mathrm{LN}({\mathrm{x}}^{\mathrm{A}}\left)\right)$

13: Stage 3: Selective scan (ZOH discretization inside) 14: y ← SelectiveScan(x̃M, Δ, A, B, C, D)

15: Stage 4: Output-level Zeta residual $16\text{:}\mathrm{yM}\text{←}\mathrm{y}+\mathsf{\zeta }\text{̂}\text{·}\tan \mathrm{h}({\mathrm{W}}_{\mathrm{out}}\text{·}\mathrm{LN}({\mathrm{x}}^{\mathrm{A}}\left)\right)$

17: Module-level residual

3.4. Adaptive Modulation Strength

If the five learnable scalars α, β, γ, η, and ζ are used without constraints, their magnitudes may continue to increase during training and cause the cross-modal pathways to overwhelm the primary-modality stream. This risk is particularly relevant during early training, when stable correspondence between primary- and auxiliary-modality representations has not yet formed. CM-S6 therefore applies bounded reparameterization, combining a tanh mapping with interval clipping, as defined in (12):

$$\mathsf{\rho }=\mathrm{c}\mathrm{l}\mathrm{i}\mathrm{p}\left(\tanh (\widehat{\mathsf{\rho }}),0,{\mathsf{\rho }}_{\mathrm{m}\mathrm{a}\mathrm{x}}\right),\hspace{1em}\mathsf{\rho }\in \mathsf{\alpha },\mathsf{\beta },\mathsf{\gamma },\mathsf{\eta },\mathsf{\zeta }$$

(12)

Here, ρ̂ denotes the underlying learnable real-valued variable. The tanh mapping provides a smooth bounded response over its active range, while explicit clipping enforces the prescribed upper limit. The coefficient bounds are set to α_max = 0.50, β_max = 0.70, γ_max = 0.70, η_max = 0.50, and ζ_max = 0.30 according to the numerical sensitivity and intended role of each path. Because Δ directly affects the exponential discretization term, α_max = 0.50 keeps the multiplicative factor Δperturb within [0.5, 1.5], providing a positive relative adjustment around the primary-modality baseline. B and C enter the state equations through additive linear terms and are assigned the less restrictive bounds β_max = γ_max = 0.70. CII acts before the SSM, so η_max = 0.50 limits the extent to which input injection can alter the primary representation. The output calibration path is auxiliary rather than the principal information route, and ζ_max is therefore set to 0.30. These bounds prevent any single path from dominating the primary stream. Section 4.5 analyzes the resulting coefficient distributions and stability.

3.5. Model Variants

We construct full CM-S6 and the lightweight CM-S6(L) from the same CM-S6 module and extend the topology to the HSI, SAR, and DSM inputs of Augsburg. All configurations preserve the five-coefficient mechanism defined in Section 3.3; they differ mainly in scan direction, the number of CM-S6 blocks, and the topology of modality interaction. The tri-modal topology expands the number of auxiliary modalities and is not treated as a separate capacity level. Full CM-S6 uses bidirectional scanning. For the dual-modal Houston 2013 and MUUFL tasks, HSI and LiDAR are first mapped into a D-dimensional feature space by separate convolutional embeddings and flattened into sequences with the same token order. Within each layer, the HSI branch is first updated with HSI as the primary modality and LiDAR as the auxiliary modality. The updated HSI representation then modulates the LiDAR branch. The backward branch applies the same chain interaction to reversed token sequences and restores the original order after scanning. This process yields four outputs: forward HSI, backward HSI, forward LiDAR, and backward LiDAR. Learnable weights $w\in R^4$are normalized by softmax to combine the four sequences, after which global average pooling and a classification head produce the prediction. The chain structure places each modality in both a primary update path and an auxiliary modulation path, yielding bidirectional, alternating cross-modal state updates. Augsburg contains HSI, SAR, and DSM inputs. Each modality serves as the primary stream of its own CM-S6 stack. The HSI stream receives a unified auxiliary signal obtained by attention-gated aggregation of SAR and DSM, while the SAR and DSM streams each receive the modulated HSI representation as their auxiliary signal. The aggregation is defined in (13):

$${\mathrm{q}}_{\mathrm{t}}=\mathrm{LN}\left({\stackrel{\mathrm{̄}}{\mathrm{x}}}^{\mathrm{HSI}}\right),{\mathrm{k}}_{\mathrm{t}}^{\left(\mathrm{i}\right)}=\mathrm{LN}\left({\stackrel{\mathrm{̄}}{\mathrm{x}}}^{\left(\mathrm{i}\right)}\right),{\mathrm{g}}^{\left(\mathrm{i}\right)}=\mathrm{softmaxi}\left({\mathrm{q}}_{\mathrm{t}}\text{⋅}{\mathrm{k}}_{\mathrm{t}}^{\left(\mathrm{i}\right)}\right),{\mathrm{x}}_{\mathrm{t}}^{\mathrm{A}}=\sum \mathrm{i}\text{∈}\mathrm{SAR},\mathrm{DSM}{\mathrm{g}}^{\left(\mathrm{i}\right)}\text{⋅}{\stackrel{\mathrm{~}}{\mathrm{x}}}_{\mathrm{t}}^{\left(\mathrm{i}\right)}$$

(13)

Here, x̄ denotes global mean pooling over the token dimension for each modality sequence, and x̃ denotes an auxiliary token sequence after lightweight compression projection. The aggregated auxiliary signal $x_t^A$ enters the five-coefficient modulation paths in (6)–(11). The forward and backward outputs of the three modality streams form six sequences. They are combined using softmax-normalized learnable weights $w\in R^6$ and then passed through global average pooling and the classification head. CM-S6(L) retains one forward CM-S6 block and removes both backward scanning and the independent auxiliary-modality output branch. HSI is used as the primary modality, while LiDAR for Houston 2013 and MUUFL, or SAR and DSM for Augsburg, provides the auxiliary input. When multiple auxiliary modalities are available, they are aggregated using (13). The CM-S6 output is passed directly to global average pooling and the classification head. This simplification changes only the scan branches and number of CM-S6 blocks; it preserves the CII, Δ/B/C, and ζ modulation paths.

Figure 4 presents the overall architectures of the full dual- and tri-modal models. CM-S6(L) uses the same modality embeddings and auxiliary-modality aggregation but simplifies the scan structure as described above and is therefore not drawn separately. Appendix A further discusses the conceptual distinction between parameter-generation-pathway modulation and data-level fusion from a function-space perspective.

4. Experiments and Discussion

4.1. Datasets and Implementation Details

We evaluate CM-S6 on three public multisource remote sensing datasets: Houston 2013, Augsburg, and MUUFL Gulfport. These datasets represent two multimodal classification settings: dual-modal HSI–LiDAR fusion and tri-modal HSI–SAR–DSM fusion. Their different modality combinations, class distributions, and training-set sizes allow CM-S6 to be evaluated under several limited-label conditions.

Houston 2013 was released for the 2013 IEEE GRSS Data Fusion Contest [33]. It contains HSI and single-channel LiDAR elevation data covering 349 × 1905 pixels and 15 land-cover classes. The original HSI contains 144 bands. After removing four noisy bands from each end of the spectrum, the remaining 136 bands are normalized using band-wise min–max scaling and reduced to 20 principal components. The LiDAR data are independently normalized using min–max scaling. The available label maps are combined, after which 20 labeled center pixels per class are randomly selected for training, resulting in 300 training samples; all remaining labeled centers are used for testing. The split is generated using a fixed random seed of 42. An 11 × 11 neighborhood centered on each labeled pixel is extracted as the input patch.

The Augsburg dataset contains HSI, SAR, and DSM data with a spatial size of 332 x 485 pixels and seven land-cover classes [34]. The 180-band HSI is normalized band-wise and reduced to 20 principal components. The SAR data contain four polarization channels, whereas the DSM is single-channel. To reduce the influence of extreme scattering values, each SAR channel is clipped at its 1st and 99th percentiles and subsequently normalized. The DSM is normalized using min–max scaling. We use the training and test label masks supplied with the dataset, which contain 761 and 77,533 labeled pixels, respectively. The input patch size is 11 × 11.

MUUFL Gulfport comprises HSI and two-channel LiDAR data with a spatial size of 325 x 220 pixels, 11 land-cover classes, and 53,687 labeled pixels [35,36]. The original HSI contains 72 bands. After removing eight noisy bands, the remaining 64 bands are normalized band-wise and reduced to 20 principal components. The two LiDAR channels are independently normalized using min–max scaling. The available training and test label maps are combined, after which 20 labeled center pixels per class are randomly selected for training using a fixed seed of 42. This produces 220 training samples, while the remaining 53,467 labeled centers are used for testing. The input patch size is 11 × 11.

For all three datasets, normalization parameters, SAR percentile thresholds, and PCA transformations are estimated once from all spatial pixels in the complete co-registered scene before patch extraction. No class labels are used in estimating these preprocessing parameters. Accordingly, the experiments follow the full-scene transductive preprocessing convention commonly used in hyperspectral image classification. Training and test center pixels are mutually exclusive under the adopted splits.

All three datasets follow a common training pipeline. We use AdamW with an initial learning rate of 5 × 10⁻⁴ and a weight decay of 1 × 10⁻⁴ [37]. The batch sizes are 64, 32, and 64 for Houston 2013, Augsburg, and MUUFL, respectively, and all models are trained for 100 epochs. During each run, the test split was excluded from gradient-based optimization and loss computation. For the benchmark results, however, the model was evaluated on the test split after each epoch, and the checkpoint with the highest test OA was retained for reporting. This epoch-wise checkpoint-selection rule was applied identically to CM-S6, CM-S6(L), and all comparison methods. The objective function is class-weighted cross-entropy with label smoothing of 0.1. Following a 10-epoch linear warm-up, the learning rate is reduced to 1 × 10⁻⁶ using cosine annealing, and the gradient norm is clipped at 1.0. For CM-S6 and CM-S6(L), the learning rate assigned to the five learnable modulation scalars is ten times the base learning rate, whereas all other parameters use the base rate. Random horizontal and vertical flips are applied synchronously across modalities, and automatic mixed-precision training is used.

Performance is evaluated using overall accuracy (OA), average accuracy (AA), and the Kappa coefficient. Based on their competitiveness and architectural relevance, MTMixer, MCAMamba, and PICNet are selected for detailed repeated-run comparisons with CM-S6 and CM-S6(L). These five methods are evaluated over ten repeated training runs using aligned random seeds 42–51. This protocol provides run-level observations for estimating the mean, standard deviation, performance variability, and paired statistical differences. The remaining comparison methods are reported using five-run results. The progressive, single-path-removal, and pathway ablations are each repeated five times using seeds 42–46. Unless otherwise stated, repeated-run results are reported as the mean ± standard deviation. All experiments are conducted on a single NVIDIA GeForce RTX 4090 GPU.

4.2. Comparison with Representative Baselines

To systematically evaluate the classification performance of CM-S6, we compare it with eight representative public methods: the single-modal Mamba methods MambaHSI and S²-Mamba [6,7]; the single-modal Transformer method DSFormer [38]; the multimodal Transformer methods MFT and PICNet [13,14]; the Transformer–Mamba hybrid method MTMixer [18]; and the multimodal Mamba methods HLMamba and MCAMamba [8,9]. To ensure a controlled comparison, all methods are trained and evaluated under the same data protocol as CM-S6. Specifically, they use identical training and test partitions, scene-level preprocessing, modality registration, 11 × 11 patch extraction, data augmentation, and evaluation procedures; no method is assigned an additional or method-specific data split. For single-modal baselines, only the HSI component of the corresponding patch is used, whereas multimodal baselines receive the registered auxiliary modalities required by their respective architectures. Therefore, the compared methods use the same sample centers and labels, and the performance differences primarily reflect their model designs. Results on the three datasets are reported in Table 1, Table 2 and Table 3, respectively.

4.2.1. Classification on Houston 2013

The quantitative results on Houston 2013 are reported in Table 1. Over ten independent runs, CM-S6 achieves OA, AA, and Kappa values of 96.16% ± 0.18%, 96.53% ± 0.18%, and 95.85% ± 0.20%, respectively. Its OA differs from that of MCAMamba, 96.21% ± 0.19%, by only 0.052 percentage points, and the paired tests detect no significant difference in OA or Kappa. CM-S6 achieves an AA that is 0.290 percentage points higher than that of MCAMamba, with a statistically significant difference (p = 0.0175), indicating a more balanced class-wise performance. Relative to MTMixer and PICNet, CM-S6 improves OA by 0.929 and 3.447 percentage points, respectively. The complete performance differences, 95% confidence intervals, and paired-test p-values are listed in Table 4.

CM-S6 maintains a small model size while achieving OA and Kappa values for which no statistically significant difference from MCAMamba is detected. Its parameter count is 0.6628M, corresponding to only 2.89% of the 22.9293M parameters of MCAMamba, while the two methods obtain closely matched OA and Kappa values. CM-S6(L) further reduces the parameter count to 0.1879M, a 71.65% reduction relative to the full model, while retaining an OA of 95.60±0.25%. This value is higher than the corresponding OA values of MTMixer, MFT, HLMamba, and PICNet, but is 0.555 percentage points lower than that of full CM-S6, showing the accuracy-model-size trade-off of the lightweight structure. FLOPs, inference latency, and throughput are analyzed jointly in the computational-efficiency analysis below. The class-wise results show that CM-S6 achieves the highest accuracies for Healthy Grass and Residential, at 97.57% and 99.93%, respectively, and reaches 100% on Soil, Tennis Court, and Running Track. CM-S6 does not win every class (e.g., Commercial and Parking Lot 1 are lower than MCAMamba); its gain lies in more balanced class-wise performance, consistent with the significantly higher AA. Class names are listed in Appendix B. Figure 5 presents representative classification results. Compared with several baselines, CM-S6 produces more spatially continuous predictions over grass, residential areas, and small-scale objects, although local confusion remains around commercial areas and parking-lot boundaries. This qualitative pattern is consistent with the class-wise quantitative results.

4.2.2. Classification on Augsburg

Augsburg is the most complex of the three experimental datasets in terms of modality composition and has the lowest proportion of training samples. HSI, SAR, and DSM exhibit clear representational differences, and the training-to-test sample ratio is approximately 1:102. These characteristics impose stringent requirements on few-shot learning and the stability of multimodal fusion. The classification results are reported in Table 2. Over ten independent runs, CM-S6 achieves OA, AA, and Kappa values of 91.91±0.49%, 68.35±1.75%, and 88.40±0.71%, respectively. The corresponding results of MTMixer are 91.93±0.44%, 70.17±2.28%, and 88.44±0.64%, exceeding CM-S6 by 0.026, 1.819, and 0.041 percentage points, respectively. The paired t-test yields p-values of 0.914, 0.110, and 0.904 for OA, AA, and Kappa, respectively. None reaches the 0.05 significance level; therefore, no statistically significant difference between CM-S6 and MTMixer is detected on Augsburg. CM-S6 nevertheless improves OA over MCAMamba and PICNet by 0.748 and 1.406 percentage points, respectively. The complete performance differences, 95% confidence intervals, and paired-test p-values are reported in Table 4.

In terms of model size, CM-S6 contains only 0.9823M parameters, approximately 2.74% of the parameter count of MTMixer. It therefore reduces the parameter count by approximately 97.26%, while no statistically significant difference in OA or Kappa is detected between the two methods. CM-S6(L) further reduces the parameter count to 0.1909M, an 80.57% reduction relative to the full model. Its OA is 89.65±0.51%, higher than those of DSFormer and MFT but 2.256 percentage points lower than that of CM-S6. Thus, CM-S6(L) provides more pronounced compression in the tri-modal setting with a corresponding accuracy cost. The class-wise results also reveal substantial variation among categories in Augsburg. CM-S6 achieves the highest accuracy on Low Plants, at 95.90±0.81%, and reaches 97.44±0.59% on Residential Area. Its accuracy on Water is 39.27±7.41%, lower than those of MCAMamba, PICNet, and MTMixer, while the results for Industrial Area and Commercial Area also show considerable variation. Class-wise gains vary with training-sample size, class separability, and modality complementarity; no single modulation path can be isolated as the sole driver for a given class. Class names are in Appendix B. Figure 6 presents the classification results. CM-S6 preserves spatial continuity in major land-cover regions, but local confusion remains around industrial areas, commercial areas, and water boundaries, consistent with the class-wise quantitative results.

4.2.3. Classification on MUUFL Gulfport

MUUFL consists of HSI and two-channel LiDAR data. Only 20 labeled pixels per class are selected for training, resulting in 220 training samples, whereas the test set contains 53,467 samples. The large test set and limited training set increase the difficulty of modeling within-class variation and exploiting cross-modal information. The quantitative results are reported in Table 3. Over ten independent runs, CM-S6 achieves OA, AA, and Kappa values of 82.18±0.46%, 80.98±0.89%, and 77.15±0.58%, respectively. Its OA and Kappa are the highest mean values in the table. The corresponding results of MTMixer are 81.71±1.49%, 82.34±3.62%, and 76.73±1.76%. CM-S6 exceeds MTMixer by 0.464 and 0.417 percentage points in OA and Kappa, respectively, but has an AA that is 1.360 percentage points lower. The paired t-test yields p-values of 0.362, 0.283, and 0.482 for OA, AA, and Kappa, respectively. None reaches the 0.05 significance level; therefore, no statistically significant difference between CM-S6 and MTMixer is detected on MUUFL. Relative to MCAMamba and PICNet, CM-S6 improves OA by 2.111 and 3.076 percentage points, respectively. The complete performance differences, 95% confidence intervals, and paired-test p-values are reported in Table 4.

CM-S6 contains 0.6632M parameters, only 1.84% of the parameter count of MTMixer. It therefore achieves higher mean OA and Kappa values while reducing the parameter count by approximately 98.16%. CM-S6(L) further reduces the parameter count to 0.1884M, a 71.59% reduction relative to the full model. Its OA is 80.29±0.54%, which is 0.223 and 1.187 percentage points higher than those of MCAMamba and PICNet, respectively, but 1.425 and 1.888 percentage points lower than those of MTMixer and full CM-S6. These results further illustrate the trade-off between parameter size and classification performance in the lightweight model. The class-wise results show that CM-S6 achieves the highest accuracy on Trees, at 86.03±1.59%, and reaches 94.92±1.16% on Building and 99.98±0.07% on Water. Its accuracies on Mostly Grass, Sidewalk, and Yellow Curb are 60.28±4.93%, 50.81±4.27%, and 66.99±7.68%, respectively, and remain below those of some baselines. The overall accuracy of CM-S6 is primarily supported by its strong recognition of Trees, Building, and Water, whereas classes with similar spectral responses, ambiguous spatial boundaries, or complex land-cover composition remain challenging. The detailed correspondence between class indices and class names is provided in Appendix B. Figure 7 presents the MUUFL classification results. The prediction maps of PICNet, MTMixer, and MCAMamba contain varying degrees of stripe-like noise or locally scattered misclassification, whereas CM-S6 and CM-S6(L) produce more continuous spatial distributions over major land-cover regions such as trees, buildings, and roads. Local confusion remains around the boundaries of grass, mixed ground surface, and sidewalks, consistent with the class-wise accuracy results.

4.2.4. Statistical Significance Analysis

To assess performance differences quantitatively, we perform two-sided paired t-tests between CM-S6 and PICNet, MTMixer, and MCAMamba using the ten independent runs aligned by random seeds 42-51. Because three model comparisons are conducted for each dataset, p-values are adjusted using the Holm procedure to control the accumulation of Type I error associated with multiple testing. Table 4 reports OA differences, 95% confidence intervals, and significance-test results for CM-S6 relative to each comparison method; a positive difference indicates higher OA for CM-S6.

On Houston 2013, the OA difference between CM-S6 and MCAMamba is -0.052 percentage points. Its 95% confidence interval crosses zero, and the Holm-adjusted p-value is 0.4876; thus, the test detects no significant difference. CM-S6 improves OA over MTMixer and PICNet by 0.929 and 3.447 percentage points, respectively, with adjusted p-values below 0.001, indicating statistically significant differences. On Augsburg, the OA difference between CM-S6 and MTMixer is only -0.026 percentage points and is not significant. CM-S6 improves OA over MCAMamba and PICNet by 0.748 and 1.406 percentage points, respectively, and both differences remain significant after Holm correction. On MUUFL, CM-S6 improves OA over MTMixer by 0.464 percentage points, but the confidence interval crosses zero and the difference is not significant. Its OA improvements over MCAMamba and PICNet are 2.111 and 3.076 percentage points, respectively, and both differences are statistically significant. For the strongest high-capacity comparison method on each dataset, the paired OA test does not detect a statistically significant difference from CM-S6. CM-S6 also shows statistically significant OA advantages over the other strong baselines considered above. Combined with the parameter counts in Table 1, Table 2 and Table 3, these results indicate that the competitive classification performance of CM-S6 is achieved without expanding model capacity.

4.3. Ablation Study

To identify the sources of the performance gains of CM-S6, we construct an A0-A14 ablation sequence under a unified capacity setting of D = 96 and n_layers = 1. A0 and A1 represent Single-S6 and CatFuse-S6, respectively. A2-A6 progressively introduce η, α, β, γ, and ζ. A7-A11 remove one modulation path at a time from the full model, A12 and A13 retain only CII+ζ and Δ/B/C, respectively, and A14 is the lightweight variant. A0, A1, A6, and A14 are repeated ten times with seeds 42-51, whereas the remaining configurations are repeated five times with seeds 42-46. The complete results are reported in Table 5(a)-5(c), and Figure 8 summarizes the progressive and pathway ablations A0-A13. A14 is analyzed separately as a capacity-accuracy trade-off.

The overall results show that introducing multimodal information can improve classification, but the fusion location affects how effectively complementarity is used. CatFuse-S6 outperforms Single-S6 on all three datasets, indicating that the auxiliary modality can provide useful information for primary-modality discrimination. Full CM-S6 further improves OA over CatFuse-S6 by 0.724, 1.306, and 2.429 percentage points on Houston 2013, Augsburg, and MUUFL, respectively, and all paired comparisons remain significant after Holm correction. Input-level concatenation changes the joint representation entering the backbone, whereas CM-S6 additionally allows the auxiliary modality to participate in state updating, input writing, and state readout. The larger gains on Augsburg and MUUFL are consistent with the more limited training samples and stronger modality differences in these datasets.

As visualized in Figure 8(a)-8(c), the progressive ablation results show that η-controlled input injection provides the first gain in the progressive cross-modal sequence. Relative to A0, A2 improves OA by 1.361, 3.092, and 2.450 percentage points on Houston 2013, Augsburg, and MUUFL, respectively, indicating that establishing a primary-auxiliary relationship before state-space recurrence can improve subsequent sequence modeling. Adding α, β, and γ is non-monotonic: some intermediate configurations dip before recovering. This indicates that Δ/B/C are coupled pathways—the marginal effect of each depends on the presence of the others—rather than independent additive modules. Figure 8(d)-8(f) plots the OA difference between full A6 and each A7-A13 variant, such that a positive bar indicates higher OA for the full model. The three datasets exhibit different sensitivities to single-path removal. On Houston 2013, the OA values of all removal configurations fall within 96.06%-96.17%, and none of the differences is significant. This pattern is consistent with its strong single-modal baseline: HSI already provides substantial spectral discrimination, leaving limited measurable marginal contributions for LiDAR near the accuracy ceiling. On Augsburg, removing η or γ produces clearer mean decreases, but the effects remain trends after correction for multiple comparisons. This pattern may reflect the more complex complementarity among HSI, SAR, and DSM, for which other paths can partly compensate for the loss of one path. MUUFL is more sensitive to η: removing this path decreases OA by 1.152 percentage points, and the difference remains significant after Holm correction (p = 0.0053). This result indicates that input-side injection makes a measurable contribution to relating spectral information to spatial-structural information in the HSI-LiDAR setting of MUUFL.

A12 (CII+ζ only) and A13 (Δ/B/C only) isolate the two pathway groups. Critically, A13 outperforms A1 (CatFuse-S6, feature-level concatenation) by 0.66, 0.88, and 1.95 percentage points on Houston 2013, Augsburg, and MUUFL, respectively, and outperforms A0 (Single-S6) by 1.29–3.10 points. This demonstrates that parameter-level modulation alone—without any input/output injection—can independently exploit auxiliary information and surpass conventional feature-level fusion. Both A12 and A13 exceed the single-modal baseline on all datasets, confirming that the two groups are independently effective. On Houston 2013, neither differs significantly from the full model (high-accuracy ceiling effect). On MUUFL, A13 (81.70%) is closer to the full model (82.18%) than A12 (81.22%), consistent with a stronger role for internal parameter generation when spectral–structural complementarity is high. Across datasets, neither pathway group dominates universally; their relative contributions depend on sensor composition and class structure, and the full model achieves the best or near-best balance on all three datasets.

The lightweight variant further reveals the trade-off between model capacity and cross-modal discriminative ability. CM-S6(L) reduces the parameter count by 71.59%-80.57% across the three datasets, while its OA decreases by 0.555-2.256 percentage points; all corresponding differences are statistically significant. The accuracy decrease is smaller on Houston 2013 and more pronounced on Augsburg and MUUFL, indicating that complex modality combinations and limited training samples place greater demands on representation capacity. CM-S6(L) should therefore be viewed as an accuracy-efficiency trade-off for resource-constrained scenarios rather than a lossless replacement for the full model. The associated computational costs are discussed further in the following efficiency analysis. A0, A1, A6, and A14 use aligned-seed paired tests (10 runs); A6 vs. five-run ablation configurations use Welch approximation with Holm correction. Because the ablation configurations use only five runs, pathway-level conclusions should be interpreted as trends rather than definitive independent-effect estimates.

4.4. Computational Efficiency and Model Size

To evaluate the trade-off between classification accuracy and resource cost, we compare eight baseline methods with CM-S6 and CM-S6(L) in terms of parameter count, FLOPs, inference latency, throughput, and peak memory under a unified environment. All measurements are obtained on an NVIDIA GeForce RTX 4090 using PyTorch 2.1.2, CUDA 12.1, and FP32. Latency is measured with a batch size of 1 after 50 warm-up iterations and over 200 timed repetitions, whereas throughput is measured with a batch size of 16 after 30 warm-up iterations and over 100 repetitions. All three datasets use 11 x 11 input patches. FLOPs are approximated as twice the number of multiply-accumulate operations. Because custom selective-scan operators can introduce bias into theoretical operation counts, FLOPs and measured latency are analyzed separately. To ensure that accuracy and cost refer to the same model instance, Table 6 and Figure 9 use the OA of the checkpoint selected for efficiency evaluation. The multi-run means in Table 1, Table 2 and Table 3 remain the basis for classification-performance and significance conclusions.

Table 6(a) shows that full CM-S6 contains fewer than 1M parameters on all three datasets and achieves the highest selected-checkpoint OA under this efficiency-evaluation protocol. On Houston 2013, its parameter count is only 2.89% of that of MCAMamba, and its OA is 0.319 percentage points higher, although its FLOPs and latency are 18.0% and 20.8% higher, respectively. On Augsburg, the parameter count and FLOPs of CM-S6 are 2.77% and 16.74% of those of MTMixer, respectively, and its OA is 1.438 percentage points higher; however, its latency is approximately 10.05 times that of MTMixer. Together with Table 6(b), these results show that parameter count, FLOPs, and measured speed are not monotonically related. MFT has the lowest FLOPs and latency and the highest throughput on all three datasets, but its accuracy is lower than that of the CM-S6 variants. The main advantage of full CM-S6 is therefore parameter efficiency at high accuracy rather than uniformly superior runtime performance.

Table 6(c) shows that CM-S6(L) exchanges a small decrease in selected-checkpoint OA for substantial parameter compression and runtime acceleration. Its parameter count remains approximately 0.19M across the three datasets. On the Augsburg and MUUFL efficiency-evaluation checkpoints, it achieves higher OA than all external baselines, supporting its intended use in resource-constrained deployment. The reduction in peak memory is smaller than the reductions in parameter count and FLOPs, indicating that memory consumption for the current 11 x 11 inputs is also affected by intermediate activations and framework overhead.

Figure 9 consolidates six accuracy-efficiency views. In Figure 9(a)-9(c), FLOPs are plotted on the horizontal axis and OA on the vertical axis, with bubble area representing parameter count. The corresponding Pareto frontiers comprise MFT, CM-S6(L), MCAMamba, and CM-S6 on Houston 2013, and MFT, CM-S6(L), and CM-S6 on both Augsburg and MUUFL. In Figure 9(d)-9(f), measured latency is plotted against OA, with bubble area representing peak memory. The corresponding frontiers comprise MFT, CM-S6(L), MCAMamba, and CM-S6 on Houston 2013; MFT, MTMixer, CM-S6(L), and CM-S6 on Augsburg; and MFT, DSFormer, CM-S6(L), and CM-S6 on MUUFL. CM-S6(L) lies on the frontier in all six views, whereas full CM-S6 forms the highest-accuracy endpoint, consistent with their resource-trade-off and accuracy-oriented roles, respectively. Each Pareto frontier is computed using only the horizontal and vertical metrics; the third metric encoded by bubble area is provided for auxiliary comparison and does not enter the two-dimensional nondominance calculation.

4.5. Analysis of Learnable Modulation Coefficients

We now examine the five modulation coefficients to characterize how CM-S6 allocates cross-modal control. Recall that η/ζ govern input/output calibration and α/β/γ govern Δ/B/C generation; all are global scalars whose token- and channel-specific variation comes from the auxiliary projections. We note that these coefficients reflect pathway-level activation, not per-token sensor reliability or physical sensor quality.

We track full CM-S6 and CM-S6(L) for 100 epochs on each dataset using run 1 and seed 42, with a coefficient learning rate ten times the base rate. Full-model curves average the effective coefficients over four blocks on Houston 2013 and MUUFL and six on Augsburg; CM-S6(L) has one block. Table 7 reports final values and late-stage stability, Figure 10 shows the trajectories, and Figure 11 isolates the final Full-Lite reallocation. Results are from one run per setting (seed 42) and are descriptive.

Figure 10 shows that all coefficients enter stable nonzero regimes; no pathway remains at its initialization value or collectively decays toward zero. The maximum standard deviation over the final 20 epochs ranges from 2.81x10^-4 to 7.82x10^-4. The stabilization order is dataset-dependent: ζ stabilizes relatively early for full CM-S6 on Houston 2013 and Augsburg, whereas η, β, γ, and ζ stabilize later on MUUFL. In CM-S6(L), α reaches 0.50 early while the other pathways continue to adjust. Thus, the trajectories do not support a universal input-before-output convergence order.

Full CM-S6 exhibits a consistent relative allocation: α is largest at 0.4345-0.4910, η/β/γ lie at 0.2984-0.3675, and ζ remains at 0.0938-0.1139. This pattern is consistent with greater relative use of input injection and internal SSM parameter generation than output calibration. CM-S6(L) reallocates these strengths. On every dataset, α reaches its prescribed bound of 0.5000. Relative to full CM-S6, ζ increases by 0.1277, 0.1404, and 0.1038 on Houston 2013, Augsburg, and MUUFL, while β and γ also increase in most cases. Augsburg shows the clearest redistribution, with η, β, and γ reaching 0.3666, 0.4184, and 0.3677. Figure 11 pairs the final Full and Lite values and reports Lite minus Full. The pattern suggests stronger use of existing pathways under reduced capacity.

The coefficient trajectories complement the ablations: both A12 (CII+ζ) and A13 (Δ/B/C) exceed the single-modal baseline, and A13 is closer to the full model on MUUFL. All coefficients remain in stable nonzero regimes with capacity-dependent reallocation, indicating that the modulation paths are active rather than degenerate.

5. Conclusions

This work addresses the indirect role of auxiliary information in existing multimodal Mamba models, where cross-modal interaction is generally completed before or outside selective scanning. We proposed CM-S6, which conditions the Δ/B/C parameter-generation pathways of selective SSMs on auxiliary-modality signals via lightweight projections, complemented by CII and output calibration and coordinated by five bounded coefficients. CM-S6(L) retains all paths with simplified scan branches for efficiency.

On Houston 2013, no statistically significant difference in OA or Kappa is detected between full CM-S6 and MCAMamba, while CM-S6 achieves a higher AA. On Augsburg, no statistically significant difference in OA or Kappa is detected between CM-S6 and MTMixer. On MUUFL, CM-S6 obtains the highest mean OA and Kappa, although the differences from MTMixer are not statistically significant. The full models contain 0.6628M, 0.9823M, and 0.6632M parameters on the three datasets, respectively, indicating competitive classification performance with a small model size. CM-S6(L) reduces the parameter count to approximately 0.19M and increases throughput by 3.86–5.82 times, with a dataset-dependent accuracy–efficiency trade-off. The efficiency measurements further show that parameter count and FLOPs alone do not fully predict measured inference speed under the tested implementation, leaving room for future optimization of the full CM-S6 selective-scan operators.

The ablation results show that input injection/output calibration and internal Δ/B/C modulation can each exploit auxiliary information when retained separately and act complementarily in the full model, with their relative effects varying across datasets. All coefficients remain in nonzero regimes with capacity-dependent allocation. Overall, full CM-S6 and CM-S6(L) provide accuracy- and efficiency-oriented configurations of the same parameter-level fusion framework.

This study assumes co-registered modalities, fixed-size image patches, and an S6 backbone. Future work will optimize selective-scan and cross-modal projection kernels for hardware execution, investigate parameter-level modulation in state-space architectures such as Mamba-2/SSD [39], and evaluate robustness to missing modalities, noise, misregistration, and inconsistent spatial resolution. Multi-seed tracking and controlled interventions should provide more precise evidence about the contributions of individual modulation paths. Extending CM-S6 to multimodal medical imaging and multi-view video understanding is another relevant direction.