Inductive Algebraic Resonance Memory for Attention-Free Language Modelling

Self-attention couples every pair of positions in a sequence, incurring a cost that grows quadratically with se-quence length and a key–value state that grows without bound during generation. Here we introduce the inductive algebraicresonance memory (IARM), an attention-free sequence-mixing mechanism that replaces pairwise attention with two ingre-dients: a softmax-gated mixture of learned low-rank operators that modulates a positive feature map of the query, and acoordinatewise causal memory that reads each feature channel as a normalized cumulative average of the value stream.The resulting layer runs in time linear in sequence length and maintains a constant-size recurrent state per channel, so au-toregressive decoding requires no growing cache. We instantiate IARM as a 118 003 200-parameter decoder-only languagemodel, pretrain it on a 10-billion-token educational web corpus, and fine-tune it for two epochs on 207 865 UltraChat con-versations, reducing the assistant-token perplexity from 20.4 to 6.90. Diagnostics over all 528 learned operators show thatthey converge to near rank-1 maps, that the four operators in each head are engaged in balance, and that the effectivememory window sharpens monotonically with network depth. A custom groupwise 4-bit export reproduces the weightsto within a mean relative error of 9.3 % while compressing the model 3.67-fold, and is loaded by an accompanying Apple-silicon (MLX) runtime that ports the mechanism one-to-one. IARM is offered as a transparent, fully reproducible testbedfor attention-free causal modelling rather than as a benchmarked production model.