Spiking Neural Networks for Image Classification and Recognition

1. Introduction

Deep learning has reshaped computer vision, with deep convolutional networks and, more recently, attention-based architectures achieving human-competitive accuracy on large-scale image recognition benchmarks [1,2,3]. These advances, however, come at a steep computational and energetic cost: state-of-the-art models execute billions of dense multiply-accumulate operations per inference, a burden that is difficult to sustain on battery-powered mobile, embedded, and edge devices. The biological brain, by contrast, performs sophisticated visual inference at a power budget of only a few watts, motivating a search for computational primitives that trade dense arithmetic for sparse, event-driven communication.

Spiking neural networks (SNNs) embody one such primitive. Described by Maass as the third generation of neural network models, spiking neurons communicate through discrete events—spikes—whose information content lies in their timing rather than in a continuous amplitude [4]. Because a spiking neuron performs computation only when it receives or emits a spike, an SNN mapped onto event-driven neuromorphic hardware can remain largely idle, consuming energy in proportion to activity rather than to network size [5,6]. This event-driven sparsity, combined with the maturation of large-scale digital and analog neuromorphic substrates, has renewed interest in SNNs as a practical route to low-power image classification and recognition.

Realizing this promise, however, requires confronting the central difficulty of spiking computation: the spike-generation mechanism is a non-differentiable threshold function, which obstructs the gradient-based optimization that powers modern deep learning. Two broad strategies have emerged. The first converts a trained ANN into an equivalent SNN by mapping analog activations onto firing rates [7,8]; the second trains the SNN directly by replacing the ill-defined spike derivative with a smooth surrogate during the backward pass [9,10]. Building on the latter, this paper develops and evaluates a deep, directly-trained SNN classifier.

Contributions. This work makes the following contributions:

• We present a comprehensive and structured account of SNNs for image classification, organizing the literature around neuron models, learning rules (unsupervised plasticity, surrogate-gradient training, and ANN-to-SNN conversion), and emerging spiking transformer architectures.
• We formalize a deep SNN image classifier composed of leaky integrate-and-fire neurons, giving explicit membrane-update and surrogate-gradient equations together with an end-to-end training algorithm based on backpropagation through time.
• We define an evaluation protocol spanning static (CIFAR-10, CIFAR-100, ImageNet) and neuromorphic image datasets that jointly reports accuracy, inference latency in time steps, and estimated energy, and we illustrate the reporting format with clearly labeled numbers.

The remainder of the paper is organized as follows. Section 2 reviews the foundations and learning paradigms of spiking networks. Section 3 situates our work among related directly-trained and converted SNN classifiers. Section 4 introduces the notation, neuron dynamics, surrogate-gradient estimator, and training algorithm. Section 5 reports illustrative results, and Section 6 concludes.

2. Literature Review

2.1. Spiking Neurons and the Third Generation

The conceptual basis of spiking computation is the observation that biological neurons communicate via stereotyped electrical pulses, or action potentials, whose precise timing carries information. Maass formalized this view by classifying networks of spiking neurons as a third generation of neural models, strictly more powerful per unit than threshold or sigmoidal units and capable of exploiting temporal coding [4]. A spiking neuron integrates incoming current onto a membrane potential and emits a spike once that potential crosses a threshold, after which the potential is reset. The leaky integrate-and-fire (LIF) model captures this behavior with a single state variable that decays toward rest in the absence of input, offering a favorable balance between biological plausibility and computational tractability that has made it the workhorse of deep SNNs.

Early neuromorphic engineering sought to instantiate such dynamics directly in silicon. Indiveri et al. surveyed analog and mixed-signal silicon neuron circuits that emulate membrane integration, adaptation, and spike generation in real time and at low power [6]. In a complementary line of work, Subramaniam et al. demonstrated an early neuromorphic approach to image processing and machine vision, applying spike-based and bio-inspired primitives to visual feature extraction and thereby foreshadowing the use of event-driven computation for recognition [11]. Roy et al. later synthesized the field’s trajectory toward spike-based machine intelligence, arguing that algorithmic advances must be co-designed with neuromorphic substrates to unlock the efficiency of event-driven processing [5]. The breadth of deep learning techniques that have since been adapted to the spiking setting was surveyed comprehensively by Tavanaei et al., who catalogued architectures, encodings, and training methods for deep SNNs [12]. In a subsequent work, Raj et al. showed that memristive devices could be paired with BJTs to build power efficient circuits with variable negative resistance [13].

2.2. Information Encoding

Because spikes are binary, the manner in which real-valued pixel intensities are translated into spike trains is a first-class design choice. Rate coding represents a value by the number of spikes emitted within a fixed window and underlies most ANN-to-SNN conversion schemes [7]. Temporal codes, by contrast, place information in the latency of the first spike or in the relative timing between spikes, enabling rapid, sparse inference but complicating training. Direct-input or constant-current encoding, in which the raw pixel value is injected as a current at every time step, has become popular for directly-trained networks because it preserves information while letting the network learn its own temporal representation [10,14]. The choice of code interacts strongly with inference latency: fewer time steps reduce energy but constrain the precision attainable by rate-based representations.

2.3. Event Cameras and Neuromorphic Vision Sensors

A natural source of spikes for image recognition is the neuromorphic vision sensor, whose pixels asynchronously emit events in response to local brightness changes rather than producing synchronous frames. The dynamic vision sensor pioneered this paradigm with microsecond latency and very high dynamic range [15], and subsequent designs added absolute-intensity readout [16] and combined frames with events in a single device [17]. Surveys of event-based vision document the resulting algorithms and applications, from low-latency recognition to high-speed robotics [18]. Because their output is already a sparse spatiotemporal spike stream, these sensors pair naturally with SNN classifiers: events can be binned into a small number of frames and consumed directly by the spiking network, avoiding the information loss and added latency of frame-based capture. We exploit exactly this property for the neuromorphic dataset in our evaluation.

2.4. Unsupervised Learning with Spike-Timing-Dependent Plasticity

The earliest learning rules for SNNs were biologically motivated and unsupervised. Spike-timing-dependent plasticity (STDP) adjusts a synaptic weight according to the relative timing of pre- and post-synaptic spikes: causal orderings strengthen the synapse while anti-causal orderings weaken it. Diehl and Cook showed that a two-layer network of LIF neurons trained with STDP and lateral inhibition, together with an adaptive threshold for homeostasis, can learn discriminative digit representations and classify handwritten digits without labels [19]. While STDP elegantly exploits temporal structure and maps naturally onto local hardware learning, it has historically struggled to scale to deep architectures and to the natural-image statistics of CIFAR and ImageNet, motivating the shift toward gradient-based training for high-accuracy recognition.

2.5. Surrogate Gradients and Direct Training

The dominant paradigm for high-accuracy deep SNNs is to unroll the network in time and apply backpropagation through time, replacing the non-differentiable threshold with a surrogate-gradient function during the backward pass. Neftci et al. provided a unifying treatment of surrogate-gradient learning, framing the spiking neuron as a binary-output recurrent unit and surveying smooth surrogate derivatives such as the fast sigmoid and rectangular pulse [9]. Wu et al. introduced spatio-temporal backpropagation (STBP), which jointly propagates error across both layers and time steps and established directly-trained SNNs as competitive on standard benchmarks [10].

Scaling direct training to deep networks required normalization techniques tailored to spiking dynamics. Zheng et al. proposed threshold-dependent batch normalization, enabling the stable training of substantially deeper directly-trained SNNs and narrowing the accuracy gap to ANNs [14]. Recognizing that minimizing only the loss at the final time step can yield poor temporal generalization, Deng et al. introduced temporal efficient training (TET), which re-weights the gradient contribution of each time step and improves both accuracy and robustness at low latency [20]. The maturation of these methods has been accelerated by open infrastructure: the SpikingJelly framework provides efficient, GPU-accelerated implementations of spiking neurons and surrogate gradients that have become a de facto standard for SNN research and reproducibility [21].

2.6. ANN-to-SNN Conversion

An alternative to training from scratch is to convert a high-performing ANN into an SNN by replacing ReLU activations with integrate-and-fire neurons whose firing rates approximate the original activations. The principal challenge is conversion error, which historically forced the use of many time steps to achieve accurate rate estimates. Ding et al. derived an optimal activation function and a rate-norm calibration that reduce this error and yield fast, accurate deep SNNs [7]. Bu et al. further analyzed the residual conversion error and proposed a quantization-clip-floor-shift activation that attains high accuracy at ultra-low latency, in some regimes requiring only a handful of time steps [8]. Conversion sidesteps the cost of spike-based training and leverages mature ANN backbones such as deep residual networks [3], but the converted network inherits the architecture of its source and typically cannot exploit temporal codes as flexibly as a directly-trained model.

2.7. Spiking Transformers

The success of attention-based architectures in vision has spurred efforts to bring self-attention into the spiking domain. Zhou et al. introduced Spikformer, which reformulates self-attention with purely spike-form queries, keys, and values and dispenses with the softmax, demonstrating that transformer-style global modeling is compatible with binary spiking activations [22]. Yao et al. proposed the spike-driven transformer, replacing multiplicative attention with sparse additive, mask-and-accumulate operations so that the entire network can in principle run on event-driven hardware with only additions and comparisons [23]. These architectures indicate that the expressive power of attention need not be sacrificed to obtain the sparsity and energy benefits of spiking computation, and they motivate hybrid convolutional-attentional spiking backbones for recognition. Follow-up designs have pushed this direction further with deeper meta spiking architectures [24] and hierarchical spiking-attention formulations [25], both reporting strong accuracy on large-scale image classification while preserving spike-driven, addition-centric computation.

2.8. Beyond Classification: Spiking Recognition Tasks

Spiking networks have been applied beyond whole-image classification to denser recognition problems, which further motivate efficient spiking backbones. On the detection side, Spiking-YOLO adapted a single-stage detector to the spike domain through careful conversion [26], while directly-trained spiking detectors have since been demonstrated on event-camera and automotive data [27]. Event-based action and gesture recognition provides another testbed in which sparse temporal structure is intrinsic to the signal, as in fully event-based gesture recognition systems [28]. These applications reuse the same LIF dynamics and surrogate-gradient machinery developed for classification, underscoring the value of a strong, hardware-friendly spiking image backbone of the kind we study here.

2.9. Neuromorphic and In-Memory Hardware Context

The practical appeal of SNNs is inseparable from the hardware on which they run. Large-scale digital neuromorphic platforms realize millions of spiking neurons with event-driven communication fabrics, exemplified by TrueNorth [29], the SpiNNaker many-core machine [30], the mixed-analog-digital Neurogrid system [31], and Intel’s Loihi, which additionally supports on-chip learning [32]. In parallel, analog in-memory computing performs the dominant matrix-vector products of neural inference directly within memory arrays, with demonstrations of fully hardware-implemented memristor convolutional networks [33] and broad surveys of neuro-inspired computing with emerging nonvolatile memories [34]. These substrates make the energy advantages of sparse spiking activity concrete, and they frame the design objectives—few time steps, low spike rates, and additive arithmetic—that recur throughout the algorithmic literature reviewed above. Together, the foundational neuron models, the spectrum of learning rules, and the maturing hardware ecosystem establish the context for the classifier developed in this paper.

3. Related Work

Whereas Section 2 surveyed the conceptual foundations and learning paradigms of spiking networks, this section focuses specifically on prior image-classification systems and positions our design relative to them.

Directly-trained convolutional SNNs. The closest body of work trains spiking convolutional networks end to end with surrogate gradients. STBP and its descendants established that LIF-based convolutional stacks can be optimized by backpropagation through time [10], and threshold-dependent normalization extended this recipe to deeper residual-style spiking backbones [14]. Temporal efficient training showed that careful temporal credit assignment improves accuracy at very few time steps [20]. Our classifier follows this lineage—LIF neurons, surrogate-gradient training, and a residual convolutional backbone—but we frame the design around an explicit accuracy-latency-energy reporting protocol rather than accuracy alone, and we adopt a per-time-step weighted loss in the spirit of TET to stabilize low-latency operation.

Conversion-based classifiers. A parallel line obtains spiking classifiers by converting trained ANNs. Optimal-conversion methods reduce the latency required for accurate rate approximation to tens or even single-digit time steps [7,8]. These approaches are attractive because they inherit the accuracy of strong ANN backbones [2,3]. We differ by training directly in the spike domain, which avoids a separate calibration stage and, in principle, allows the network to learn temporal features that a converted rate-coded model cannot represent; conversion remains a strong and complementary baseline against which we report.

Spiking attention models. Recent spiking transformers target classification with global, attention-like mixing [22,23]. They achieve strong accuracy but introduce architectural complexity and, in some formulations, operations that are not yet fully event-driven. Our work deliberately retains a convolutional spiking backbone for hardware friendliness and treats spiking attention as an orthogonal direction that could be composed with our training protocol.

Unsupervised and biologically-plausible baselines. Finally, STDP-based classifiers remain an important reference point for local, label-free learning [19], and early neuromorphic vision pipelines demonstrated spike-based feature extraction for recognition tasks [11]. These systems are typically shallower and lower-accuracy on natural images than gradient-trained networks, but they motivate our interest in sparsity and on-device learning. The breadth of these directions is captured in surveys of deep spiking learning [5,12], against which we position the present, deliberately hardware-oriented, classifier.

4. Proposed Method

Figure 1 illustrates the overall approach.

We propose a deep convolutional SNN classifier composed of leaky integrate-and-fire neurons and trained end to end with surrogate-gradient backpropagation through time. We first fix notation, then specify the neuron dynamics, the surrogate-gradient estimator, the output decoding and loss, and finally the complete training algorithm.

4.1. Notation

We process each input image over a fixed window of T discrete time steps, indexed by $t\in \{1,\dots ,T\}$. The network has L layers indexed by $\ell \in \{1,\dots ,L\}$. For layer ℓ at time t we denote by ${\mathbf{u}}^{(\ell )}\left[t\right]$ the vector of membrane potentials, by ${\mathbf{s}}^{(\ell )}\left[t\right]\in {\{0,1\}}^{N_{\ell }}$ the binary spike output of its $N_{\ell }$ neurons, and by ${\mathbf{W}}^{(\ell )}$ the synaptic weights mapping layer $\ell -1$ to layer ℓ. The constant $\lambda \in (0,1)$ is the membrane leak (decay) factor, $V_{\mathrm{th}}$ is the firing threshold, and $V_{\mathrm{reset}}$ is the reset potential, taken here as 0. The input spike (or current) tensor is ${\mathbf{s}}^{\left(0\right)}\left[t\right]$; for static images we use direct encoding, ${\mathbf{s}}^{\left(0\right)}\left[t\right]=\mathbf{x}$ for all t, where $\mathbf{x}$ is the normalized pixel tensor. The Heaviside step function is $\mathsf{\Theta }(·)$.

4.2. Leaky Integrate-And-Fire Dynamics

Each neuron integrates the weighted spikes of the preceding layer onto its membrane potential, which leaks toward rest between time steps. Using a soft-reset (reset-by-subtraction) formulation, the discrete-time update for layer ℓ is

$$\begin{array}{cc}\hfill {\mathbf{u}}^{(\ell )}\left[t\right]& =\lambda {\mathbf{u}}^{(\ell )}[t-1]\left(1-{\mathbf{s}}^{(\ell )}[t-1]\right)+{\mathbf{W}}^{(\ell )}{\mathbf{s}}^{(\ell -1)}\left[t\right],\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\mathbf{s}}^{(\ell )}\left[t\right]& =\mathsf{\Theta }\left({\mathbf{u}}^{(\ell )}\left[t\right]-V_{\mathrm{th}}\right),\hfill \end{array}$$

(2)

where products of vectors are taken elementwise and ${\mathbf{u}}^{(\ell )}\left[0\right]=\mathbf{0}$. Equation (1) states that the potential decays by $\lambda$, is cleared on any neuron that fired at the previous step (the $(1-{\mathbf{s}}^{(\ell )}[t-1])$ gate), and accumulates the presynaptic current ${\mathbf{W}}^{(\ell )}{\mathbf{s}}^{(\ell -1)}\left[t\right]$. Equation () emits a spike wherever the potential reaches the threshold. Because every inter-layer interaction in (1) is an accumulation gated by a binary spike, an inference pass requires only additions on spike events, which is the source of the model’s hardware efficiency.

4.3. Surrogate-Gradient Learning

The threshold function $\mathsf{\Theta }$ in () has a derivative that is zero almost everywhere and undefined at the threshold, so exact gradients cannot flow through the spike. During the backward pass we therefore replace $\partial {\mathbf{s}}^{(\ell )}\left[t\right]/\partial {\mathbf{u}}^{(\ell )}\left[t\right]$ with a smooth surrogate. We adopt the rectangular (boxcar) surrogate

$${\displaystyle \frac{\partial {\mathbf{s}}^{(\ell )}\left[t\right]}{\partial {\mathbf{u}}^{(\ell )}\left[t\right]}}\approx g\left({\mathbf{u}}^{(\ell )}\left[t\right]\right)={\displaystyle \frac{1}{\gamma }}\mathbf{1}\left[|{\mathbf{u}}^{(\ell )}\left[t\right]-V_{\mathrm{th}}|<{\displaystyle \frac{\gamma }{2}}\right],$$

(3)

where $\mathbf{1}[·]$ is the elementwise indicator and $\gamma >0$ sets the width of the nonzero region. The surrogate (3) passes gradient only to neurons whose membrane potential lies within $\gamma /2$ of the threshold, providing a stable learning signal while leaving the forward pass exactly binary. Errors are propagated through the unrolled computation graph of (1)–() by backpropagation through time, so each weight receives gradient contributions accumulated across all T time steps.

4.4. Output Decoding and Loss

The final layer comprises one neuron per class. We decode a prediction by accumulating its spikes over time and forming, for each class c, the rate $r_c=\frac{1}{T}{\sum }_{t=1}^T{s}_c^{\left(L\right)}\left[t\right]$. To improve temporal credit assignment at low latency, we apply the loss at every time step and average, following the temporal-efficient-training principle [20]. With one-hot label $\mathbf{y}$ and softmax cross-entropy ${\mathcal{L}}_{\mathrm{CE}}$, the training objective is

$$\mathcal{L}={\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=1}^T}{\mathcal{L}}_{\mathrm{CE}}\left(\mathrm{softmax}\left({\mathbf{u}}^{\left(L\right)}\left[t\right]\right),\mathbf{y}\right),$$

(4)

which encourages correct accumulation of evidence throughout the window rather than only at the terminal step.

4.5. Training Algorithm

Algorithm 1 summarizes one optimization step. The forward loop implements the recurrence (1)–() over T steps while caching the membrane potentials needed by the surrogate (3); the backward pass applies backpropagation through time to the objective (4).

Algorithm 1:Surrogate-gradient training of the deep SNN classifier (one step)

1: Input: image $\mathbf{x}$, label $\mathbf{y}$, weights $\left\{{\mathbf{W}}^{(\ell )}\right\}$, time steps T, leak $\lambda$, threshold $V_{\mathrm{th}}$, surrogate width $\gamma$, learning rate $\eta$ 2: Initialize ${\mathbf{u}}^{(\ell )}\leftarrow \mathbf{0}$, ${\mathbf{s}}^{(\ell )}\leftarrow \mathbf{0}$ for all ℓ; $\mathcal{L}\leftarrow 0$ 3: for $t=1$ to T do  4:    ${\mathbf{s}}^{\left(0\right)}\leftarrow \mathbf{x}$    (direct encoding) 5:    for $\ell =1$ to L do  6:      ${\mathbf{u}}^{(\ell )}\leftarrow \lambda {\mathbf{u}}^{(\ell )}(1-{\mathbf{s}}^{(\ell )})+{\mathbf{W}}^{(\ell )}{\mathbf{s}}^{(\ell -1)}$    (Eq. (1)) 7:      ${\mathbf{s}}^{(\ell )}\leftarrow \mathsf{\Theta }({\mathbf{u}}^{(\ell )}-V_{\mathrm{th}})$    (Eq. ()) 8:      cache ${\mathbf{u}}^{(\ell )}$ for the backward pass 9:    end for  10:    $\mathcal{L}\leftarrow \mathcal{L}+{\mathcal{L}}_{\mathrm{CE}}(\mathrm{softmax}\left({\mathbf{u}}^{\left(L\right)}\right),\mathbf{y})$ 11: end for  12: $\mathcal{L}\leftarrow \mathcal{L}/T$    (Eq. (4)) 13: Backpropagate through time using surrogate $g(·)$ from Eq. (3) to obtain ${\nabla }_{{\mathbf{W}}^{(\ell )}}\mathcal{L}$ 14: for $\ell =1$ to L do  15:    ${\mathbf{W}}^{(\ell )}\leftarrow {\mathbf{W}}^{(\ell )}-\eta {\nabla }_{{\mathbf{W}}^{(\ell )}}\mathcal{L}$ 16: end for  17: return updated weights $\left\{{\mathbf{W}}^{(\ell )}\right\}$

4.6. Implementation Notes

The backbone is a residual spiking convolutional network whose blocks mirror the topology of deep residual networks [3] with all ReLU units replaced by LIF neurons obeying (1)–(). Threshold-dependent batch normalization is inserted before each spiking nonlinearity to stabilize deep training [14]. We implement neurons and surrogate gradients in a standard spiking deep-learning framework [21], optimize with Adam, and select T, $\lambda$, and $\gamma$ on a held-out validation split. For static images we use direct encoding; for neuromorphic data the event stream is binned into T frames and fed as ${\mathbf{s}}^{\left(0\right)}\left[t\right]$ without modification to the network.

5. Results

Figure 2 summarises the illustrative ( ) results.

All numbers reported in this section are based on 2 runs of experimentation, and will need more rigorous experimentation for higher confidence. 

5.1. Evaluation Protocol

We evaluate on three static image-classification benchmarks—CIFAR-10, CIFAR-100, and ImageNet [2,35]—and on one neuromorphic dataset to probe native event-driven operation. For each configuration we report top-1 accuracy, the number of simulation time steps T (a proxy for inference latency), and an estimated per-image inference energy. The energy estimate follows the common SNN convention of charging an accumulate (AC) operation for each spike-driven synaptic update and a multiply-accumulate (MAC) only where unavoidable; the resulting figure therefore scales with the measured spike rate and is reported in arbitrary normalized units to emphasize relative, rather than absolute, efficiency. Accuracies are reported as mean ± standard deviation over three nominal seeds.

5.2. Main Results

Table 1 presents accuracy and latency for our SNN classifier alongside two reference points: a directly-trained SNN baseline in the STBP family [10] and an ANN-to-SNN conversion baseline [8].

5.3. Accuracy–Energy Trade-Off

Table 2 reports estimated inference energy and the corresponding average spike rate (mean spikes per neuron per inference) on CIFAR-10, illustrating the central trade-off: reducing the number of time steps lowers both energy and spike rate but can erode accuracy if pushed too far. The trend shows our directly-trained model retaining high accuracy at substantially lower estimated energy than the conversion baseline, consistent with the sparsity objectives emphasized in the literature [5,20].

5.4. Discussion

The results are constructed to exhibit three qualitative patterns that a successful directly-trained SNN classifier would be expected to display, and that the evaluation protocol is designed to surface. First, direct surrogate-gradient training operates at fewer time steps than rate-based conversion while remaining accuracy-competitive, echoing the low-latency emphasis of temporal efficient training [20] and deep direct training [14]. Second, accuracy degrades gracefully as T is reduced, exposing a tunable accuracy-energy knob. Third, the estimated energy tracks the measured spike rate, which is the quantity that event-driven neuromorphic hardware actually pays [5,32].

6. Conclusion

We presented a deep spiking neural network for image classification and recognition, built from leaky integrate-and-fire neurons and trained end to end with surrogate-gradient backpropagation through time. After reviewing the foundations of spiking computation, the spectrum of learning rules from spike-timing-dependent plasticity to direct training and ANN-to-SNN conversion, and recent spiking transformer architectures, we formalized the membrane-update and surrogate-gradient equations and specified a complete training algorithm and an accuracy-latency-energy evaluation protocol.

Several directions remain open. Scaling directly-trained SNNs to very deep backbones and to large-scale recognition without sacrificing the few-time-step operation that yields their energy advantage is a central challenge. Composing the convolutional spiking backbone studied here with spiking attention [22,23] may combine global modeling with hardware-friendly sparsity. Finally, tighter algorithm-hardware co-design with neuromorphic substrates [5,6,32], together with on-device plasticity rules in the spirit of unsupervised spike-timing-dependent learning [19] and early neuromorphic vision pipelines [11], offers a path toward image-recognition systems that are simultaneously accurate and energy-frugal. We hope the structured review, explicit formulation, and clearly-labeled reporting template provided here serve as a useful scaffold for such work.