QF2: Quick Firing Model Weight Updates

1. Introduction

Modern artificial intelligence has predominantly relied on gradient backpropagation, an algorithm that, despite its effectiveness, fundamentally diverges from biological learning mechanisms observed in the human brain [1]. Neuroscientific evidence reveals no biological counterpart to explicitly defined loss functions or the long-range chain-rule gradient computations found in artificial neural networks. The human brain does not operate with residual connections throughout its circuitry [2], nor do its neurons utilize unbounded activations such as ReLU [3]. While these engineering designs have been remarkably effective for scaling deep learning systems, they may actually hinder the development of AI systems that more closely resemble human intelligence. In contrast, biological neurons communicate through simple firing-based signaling and local synaptic wiring changes triggered by neuronal firing, where each neuron’s output is fundamentally binary, corresponding to discrete “firing” (1) or “silent” (0) states. Human learning, moreover, is inherently incremental and hierarchical: knowledge is acquired gradually, progressing from simple to complex concepts, rather than through massive, unordered statistical learning as practiced in modern AI [4]. These observations highlight the fundamental differences between current deep learning paradigms and true biological intelligence.

Recently, the Quick Feedforward (QF) learning framework [5] was introduced to address many of the shortcomings of gradient-based backpropagation. QF fundamentally changes the route of information entry into the model: instead of learning via error signals from loss functions, QF injects explicit knowledge through instructions, aligning neural activations with the desired output. The knowledge is then consolidated into model weights through direct matrix inversion-based updates. While this approach is mathematically elegant, matrix inversion remains biologically implausible, neurons are unlikely to perform such complex linear algebra or derivative-based chain computations. In contrast, biological learning is driven by spike-based firing and synaptic modifications, indicating that effective learning can be achieved without either gradient descent or matrix inversion, though the underlying mechanisms remain largely mysterious.

Building on these insights, we propose Quick Firing (QF2) learning, a biologically inspired paradigm that aligns more closely with neural firing and synaptic plasticity. The contributions of this work are threefold: (1) we introduce the Quick Firing framework, replacing both gradient-based and matrix-inversion-based updates with direct, spike-driven synaptic weight changes; (2) we provide mathematical proof of the validity and efficiency of the QF2 learning rule; and (3) we present open-source implementations and validation of QF2 learning on large language models.

2. Theory

2.1. Overview

Figure 1 illustrates the theoretical foundation and workflow of QF2 learning, a biologically inspired mechanism for rapid knowledge consolidation in neural networks. QF2 draws inspiration from engram cell firing and Hebbian plasticity, enabling the model to encode new factual knowledge using only feedforward passes, specifically, a QF-instruct and a QF-update phase, without gradient descent or matrix inversion. As illustrated, QF2 learning process is divided into three phases:

QF-instruct: Both the instruction input (X*, e.g., “Oxi has 1 people”) and the query (“how many”) are fed to the model. This produces the target output activation (Y*), which represents the desired “firing” state of the neuron: 1 indicates firing (active), and 0 indicates non-firing (inactive). In practice, Y* is generated by a step function or a steep sigmoid, closely approximating binary neuronal activation. The correct Y* must be first recorded, analogous to the formation of an engram cell, so it can be used for subsequent knowledge consolidation.

QF-update: Only the query (X) is provided. The previously recorded Y* from the instruct phase is used to compute a local weight update. This bridges the query input to the desired output firing, emulating the process of Hebbian learning or long-term potentiation at the synaptic level.

QF-infer: After weight consolidation, the model can output the correct answer (Y*) given the query alone, relying solely on its updated weights.

2.2. QF2 Weight Update

Unlike quick feedforward (QF) learning, which relies on global matrix inversion or pseudo-inverse computations to inject new knowledge while minimizing changes to the weight matrix, quick firing QF2 learning adopts a simple, generalizable, and biologically plausible update rule:

$$\begin{array}{c}∆W_{ij}=x_j\left(y_i^{\mathrm{*}}-0.5\right)\#\end{array}$$

(1)

Here, x_j​ is the j^th component of the query token input, and $y_i^{*}$ is the target token output ("firing" state: nearly 1 for firing, 0 for silence) determined by a step function or steep sigmoid during the instruct phase. This update requires only the current input x_j and the pre-recorded target activation $y_i^{*}$, making it straightforward to apply in any neural system.

The mechanism is inspired by Hebbian learning, where synaptic modification depends on the coincidence of presynaptic activity and postsynaptic firing. However, QF2 introduces a key innovation: effective synaptic change requires that the desired firing state $y_i^{*}$ be determined and recorded in advance, analogous to the formation of engram cells. For learning to occur, the target “firing” $y_i^{*}$ must precede (in the instruct phase) and mark the knowledge to be learned.

Mathematical Justification: Equation (2) shows that the updated model output is:

$$\begin{array}{c}{y}_i=\sigma \left(\sum _j\left(W_{ij}+∆W_{ij}\right)x_j\right)=\sigma \left(\sum _j\left(W_{ij}+x_j\left(y_i^{\mathrm{*}}-0.5\right)\right)x_j\right)\\ =\sigma \left(\sum _j\left(W_{ij}{x}_j+x_j^2\left(y_i^{\mathrm{*}}-0.5\right)\right)\right)\#\end{array}$$

(2)

Where σ is a steep sigmoid (e.g., σ(x) = sigmoid(βx), β≫1), closely approximating a step function.

The term $x_j^2\left(y_i^{*}-0.5\right)$ introduces a positive or negative bias depending on whether the target is firing ($y_i^{*}$​=1) or not ($y_i^{*}$​=0) and always acts in the direction that reinforces the desired response. Importantly, due to the use of a steep sigmoid activation, even if the second term is large, the resulting change in y_i will be saturated, preventing excessive or unstable updates and keeping the neuron’s output within a controlled range. Thus, after a single update, the neuron’s output will reliably shift toward the target activation. Since ${\mathit{Y}}^{\mathit{*}}$is generated by a nearly binary step function, QF2 effectively implements a “one-shot” memory binding between $\mathit{X}$ and ${\mathit{Y}}^{\mathit{*}}$.

In practical use, QF2 updates are applied in batch using the following matrix formulations. Here, $ni$ ​and $no$ are the numbers of input and output neurons (or hidden dimensions), T is the number of desired answer tokens, and lr is the learning rate, which can be adjusted according to token importance:

$$\begin{array}{c}∆{\mathit{W}}_{ni\times no}=lr*{\left({\mathit{y}}^{\mathbf{*}}-0.5\right)}_{ni\times T}{\mathit{x}}_{T\times no}^{⊺}\#\end{array}$$

(3)

$$\begin{array}{c}{\mathit{y}}_{ni\times T}=\sigma \left({\mathit{W}}_{ni\times no}{\mathit{x}}_{no\times T}+lr\mathrm{*}{\left({\mathit{y}}^{\mathbf{*}}-0.5\right)}_{ni\times T}{\mathit{x}}_{T\times no}^{⊺}{\mathit{x}}_{\mathit{n}o\times T}\right)\#\end{array}$$

(4)

Equation (3) updates all weights in parallel using the outer product between the desired targets and input activations. Equation (4) shows the resulting output after applying the QF2 update. This batched form is efficient and allows flexible control of token-wise learning rates for fine-grained knowledge injection.

2.3. Quick Framework Transfer

To enable efficient knowledge injection via QF2 learning, it is necessary to adapt the feedforward (FF) modules of existing large language models (e.g., Qwen2) into a structure compatible with the QF2 update rule. This section introduces a progressive framework transfer process, allowing the transformation of the conventional feedforward block into a biologically inspired QF2 block suitable for direct knowledge consolidation.

As illustrated in Figure 2, the transition is governed by a blending factor α, which gradually shifts the feedforward computation from the original Qwen2 module to the QF2-compatible design:

$$\begin{array}{c}{\mathit{h}}^{\mathrm{\text{'}}}=QFt\left(\mathit{h};\alpha \right)=\alpha {\mathit{F}\mathit{F}}_{\mathit{q}\mathit{w}\mathit{e}\mathit{n}2}\left(\mathit{h}\right)+(1-\alpha ){\mathit{F}\mathit{F}}_{\mathit{q}\mathit{f}2}\left(\mathit{h}\right)\end{array}$$

(5)

where ${\mathit{F}\mathit{F}}_{\mathit{q}\mathit{w}\mathit{e}\mathit{n}2}$ is the standard feedforward output, and ${\mathit{F}\mathit{F}}_{\mathit{q}\mathit{f}2}$ denotes the QF2-modified output.

Structural changes are as follows:

Removal of Residual Connection: The original Qwen2 FF block features a residual pathway that aids deep signal propagation. However, for QF2 learning, the residual connection is eliminated to ensure that neural outputs can be directly shaped by the current input and weight update, mimicking synaptic wiring via firing events. This architectural change is critical, as residual connections can bypass key information, preventing the model from correctly establishing new synaptic links during “firing” and thereby undermining the desired one-shot connection mechanism.

Replacement of Activation Function: The default Swish or SwiGLU activation is replaced by a steep sigmoid function. While Swish-based activations can accelerate convergence and improve expressivity, the steep sigmoid enforces near-binary output activations, effectively emulating neuronal “firing” (1 for active, 0 for silent). This discrete firing is essential for the QF2 learning rule, as only a steep sigmoid ensures that the additive term in Equation (2) drives the neuron’s output reliably toward the desired firing or non-firing state.

Progressive Blending: The transition from the original Qwen2 FF module (α=1) to the QF2 FF module (α=0) is controlled by an annealing schedule, e.g., α=0.99^step, such that the model smoothly morphs its structure during training. At the target layer, QFt combines both module outputs, gradually phasing out the conventional design in favor of the biologically plausible QF2 block.

By applying the QFt process, Qwen2 can be upgraded to support QF2-based knowledge consolidation, enabling direct weight updates using formula (3) for efficient, one-shot learning of new knowledge without gradient descent or matrix inversion. Moreover, QFt can be used for architecture conversion in other models as well

Figure 3. Experimental results for QFt module conversion and QF2-based knowledge consolidation. Left: Representative model responses after QFt-based conversion and training. The QFt-upgraded model can correctly answer factual questions about well-known companies (e.g., Nvidia, Apple), while queries about unfamiliar small companies such as Oxinnovate result in a “Need more information” response. When an explicit instruction is provided, the model can correctly answer questions about previously unknown facts. Right: Stepwise evaluation of the eight-stage QF2 learning process. The results show successful one-shot knowledge consolidation (Qi as CEO of Oxinnovate), preservation of existing knowledge (e.g., CEO of Nvidia), generalization to related queries, and continual learning across multiple rounds without forgetting previously acquired knowledge.

Figure 3. Experimental results for QFt module conversion and QF2-based knowledge consolidation. Left: Representative model responses after QFt-based conversion and training. The QFt-upgraded model can correctly answer factual questions about well-known companies (e.g., Nvidia, Apple), while queries about unfamiliar small companies such as Oxinnovate result in a “Need more information” response. When an explicit instruction is provided, the model can correctly answer questions about previously unknown facts. Right: Stepwise evaluation of the eight-stage QF2 learning process. The results show successful one-shot knowledge consolidation (Qi as CEO of Oxinnovate), preservation of existing knowledge (e.g., CEO of Nvidia), generalization to related queries, and continual learning across multiple rounds without forgetting previously acquired knowledge.