A Dendritic-Inspired Network Science Generative Model for Topological Initialization of Connectivity in Sparse Artificial Neural Networks

1. Introduction

Artificial neural networks (ANNs) have demonstrated remarkable potential in various fields; however, their size, often comprising billions of parameters, poses challenges for both economic viability and environmental sustainability. Biological neural networks, in contrast, can efficiently process information using ultra-sparse structures [15,47]. This efficiency arises from the brain’s highly structured and evolutionarily optimized network topology. A central component of this architecture is the dendritic tree, the primary receptive surface of the neuron [14]. Conventional ANNs omit a crucial component of the brain’s efficiency since they traditionally depict neurons as simple point-like integrators, mainly ignoring the computing power inherent in the intricate structure of dendrites.

Research has revealed that dendrites are not passive conductors but active computational units capable of performing sophisticated, nonlinear operations [26,33]. This insight has motivated theoretical frameworks that model a single neuron as a multi-layer network, where dendritic branches act as nonlinear subunits that feed a final integrator at the cell body [28]. As clarified by recent work on dendritic artificial neural networks [12], the dendritic tree’s ability to sample restricted parts of the input space mirrors the operation of convolutional layers. In this paradigm, distinct dendritic branches process specific, localized receptive fields without sharing weights, allowing for precise, location-specific feature integration. Additional efforts to translate these principles into artificial systems have confirmed that dendritic morphology has a significant impact on performance [2,20,21]. These approaches, however, are often limited to fixed structures that mimic the computational non-linearity or direct morphology of biological neurons, often overlooking the broader rules of connectivity.

Our understanding of these topological constraints has been revolutionized by the advent of large-scale functional connectomics [1]. These studies reveal that biological neural networks are not randomly wired; rather, they exhibit precise, non-random connectivity patterns characterized by specific "like-to-like" wiring rules and distinct structural motifs across cortical layers. The function of every neuron is shaped by a rigorous circuit architecture and widespread specificity in connectivity. Translating these high-level connectomic principles—such as modularity, hierarchy, and non-random receptive field organization—into scalable, generative frameworks for artificial networks remains an open challenge.

To address the gap for a flexible, principled framework for generating and testing dendritic topologies, we introduce a dendritic-inspired network science generative model for sparse topology design of neural networks: the Dendritic Network Model (DNM). The novelty of our model lies in the topological organization of the receptive fields on the input layers. Rather than directly simulating the non-linear dendritic computation or the precise biological connectomes, we introduce a network science topological initialization model that abstracts these principles. The DNM is a flexible generative model for creating sparse, biologically-inspired network architectures, constructing a network topology grounded in connectivity principles rather than direct morphological imitation. The model’s parametric approach enables the systematic exploration of the relationship between network structure and computational function. The DNM provides a principled method for generating sparse network initializations that can be integrated into modern deep learning frameworks. We demonstrate that this approach can improve performance over standard sparse initialization techniques and offers a powerful platform for exploring how structural constraints, inspired by biology, can lead to more efficient and capable artificial neural networks.

Our approach can be contrasted with other dendritic-inspired methodologies in the field. For instance, [31] experimentally demonstrated a fully integrated hardware network using memristor devices, where artificial dendrites provided non-linear integration and filtering to achieve highly efficient physical networks. Subsequently, [36] utilized bio-realistic spiking neural networks to show how active dendrites combined with synaptic turnover can optimize learning in binary classification scenarios. These works pave the way for recent advancements like the work of [12], which presents a brain-emulating model that reproduces the morphological organization and integrative functions of dendrites. In their framework, a dendrite is mapped to a node within a tree-like subnetwork, creating a powerful computational component for larger networks. Contrary to these methods, our model is a brain-inspired (there is no interest in emulating the dendrite function) network science model and a compact methodology to initialize the topology between two layers in a way that is dendritic-inspired (Figure 1). We assume that each node in the forward node layer is a soma and the connections that enter the soma layer are distributed according to some organization principles that are dendritic-inspired. Our dendrites are virtual: each dendrite is a group of synaptic connections, and they emerge by topological segregation: groups of synaptic connections that are separated from other groups of synaptic connections. We aim to show that if we organize the connectivity between two layers according to an organization principle that is dendritic-inspired, we can gain performance with respect to other topological initialization methods.

In this article, we describe the Dendritic Network Model in detail and analyse its topology and geometric characterization. We evaluate its effectiveness with extensive experiments across multiple architectures and tasks. To assess its basic functionality, we use it to initialize several static and dynamic sparse training (DST) methods on MLPs for image classification on the MNIST [29], EMNIST [13], Fashion MNIST [49], and CIFAR-10 [25] datasets. The results show that DNM clearly outperforms other sparse initialization methods over all training models tested at 99% sparsity. Next, we extend the tests on Transformers [45] for Machine Translation on the Multi30k en-de [16], IWSLT14 en-de [11], and WMT17 en-de [7] benchmarks. On this architecture, DNM outperforms all topological initialization methods at high sparsity levels. These findings underscore the potential of DNM in enabling highly efficient and effective network initialization for large-scale sparse neural network training. By analyzing the best-performing DNM topologies, we can also gain insights into the relationship between network geometry, data structure, and model performance.

2. Related Works

2.1. Sparse Topological Initialization Methods

Dynamic sparse training (DST) trains a neural network with a sparse topology that evolves throughout the learning process. The initial arrangement of the connections is a critical aspect of this framework. This starting structure determines the initial pathways for information flow and acts as the foundational scaffold upon which the network learns and evolves. A well-designed initial topology can significantly improve a model’s final performance and training efficiency, whereas a poor starting point can severely hinder its ability to learn effectively. The principal topological initialization approaches for dynamic sparse training are grounded in network science theory, where three basic generative models for monopartite sparse artificial complex networks are the Erdős-Rényi (ER) model [17], the Watts-Strogatz (WS) model [48], and the Barabási-Albert (BA) model [4]. Since the standard WS and BA models are not directly designed for bipartite networks, they were recently extended into their bipartite counterparts and termed as Bipartite Small-World (BSW) and Bipartite Scale-Free (BSF) [53], respectively. BSW generally outperforms BSF for dynamic sparse training [53]. The Correlated Sparse Topological Initialization (CSTI) [52] is a feature-informed topological initialization method that considers the links with the strongest Pearson correlations between nodes and features in the input layer. SNIP [30] is a data-informed pruning method that identifies important connections based on their saliency scores, calculated using the gradients of the loss function with respect to the weights. Ramanujan graphs [22] are a class of sparse graphs that exhibit optimal spectral properties, making them suitable for initializing neural networks with desirable connectivity patterns. The Bipartite Receptive Field (BRF) network model [54] generates networks with brain-like receptive field connectivity. This is the first attempt to mimic the structure of brain connections in a sparse network initialization model. Radix-Nets [23] offer a deterministic approach to "de novo" sparsity, utilizing mixed-radix numeral systems and the Kronecker product to construct topologies that ensure path-connectedness and symmetry while facilitating asymptotic sparsity. Finally, dendritic Artificial Neural Networks (dANNs) [12] introduce a bio-inspired architecture that mimics the structured connectivity and restricted input sampling of biological dendrites (e.g., using Local Receptive Fields). Unlike traditional approaches that strive for class specificity, this architecture fosters mixed-selective neuronal responses.

While the methods discussed above primarily focus on initializing layered or bipartite structures, a parallel line of research investigates Artificial Neural Networks (ANNs) with general complex topologies, unconstrained by multipartite restrictions. [39] demonstrated that hybrid topologies combining scale-free and small-world properties, inspired by the C. elegans connectome, can significantly improve learning curves. Moving beyond manual architecture design, [50] utilize random graph models (ER, BA, WS) to generate “randomly wired" networks that achieve competitive performance in image recognition. To facilitate the translation between arbitrary graph structures and neural models, [44] introduced the deepstruct framework. More recently, [6] provided a systematic comparison of these architectures, revealing that complex, non-layered topologies can outperform traditional Multilayer Perceptrons (MLPs) in high-difficulty tasks by potentially exploiting compositional sparsity.

3. The Dendritic Network Model

3.1. Biological Inspiration and Principles

The architecture of the Dendritic Network Model (DNM) is inspired by the structure of biological neurons. In the nervous system, neurons process information through complex, branching extensions called dendrites, which act as the primary receivers of synaptic signals. Inspired by this phenomenon, the DNM imposes a structured, dendrite-like organization on how output neurons connect to the preceding layer’s inputs (Figure 1). In this work, we refer to the set of connections between two adjacent layers as a sandwich layer, a term already used in prior literature to denote the bipartite subnetwork of edges that lies between one layer of neurons and the next. We retain this terminology because it captures the specific object our model generates, the pattern of connections, rather than the neurons themselves. By contrast, the term hidden layer refers to the neurons in intermediate layers. Since the DNM defines how neurons connect, but does not generate or modify the neurons themselves, “hidden layer" would not accurately describe the structural entity under consideration. Within each sandwich layer, each output neuron forms multiple dendritic branches, where each branch connects the neuron to a contiguous block of input neurons. These blocks are separated by inactive spaces, segments of the input layer to which the neuron does not connect. All branches belonging to a given output neuron must lie within a predefined local receptive window, resulting in a structured, compartmentalized connectivity pattern. This design moves away from unstructured random sparsity and instead emulates the localized, clustered organization characteristic of biological dendrites.

3.2. The DNM Generative Algorithm

To translate these biological principles into a computational structure, the DNM produces the sparse connectivity matrix of sandwich layers via a generative algorithm. The process builds connections iteratively for each output neuron j through the following steps: (1) Degree determination: first, determine the total degree for the output neuron based on a specific degree distribution strategy; (2) Receptive field definition: define the receptive field for the output neuron by topologically mapping the output neuron’s position to a central point on the input layer and establishing a receptive window around this center; (3) Dendritic allocation: determine the number of dendritic branches used to connect the output neuron to the input layer; (4) Dendritic Placement: place evenly spread-out dendritic brances within the neuron’s receptive window defined in step 2; (5) Synaptic distribution: distribute the output neuron’s total degree across the dendrites. Appendix F describes the algorithm in depth.

3.3. Parametric Specification

By parametrizing the core biological features of DNM, like the number of dendrites for each output neuron, the size of the receptive windows, the distribution of synapses across dendrites, and the degree distribution across output neurons, the DNM provides a flexible framework for generating network topologies that are sparse, structured, and biologically plausible. Appendix C shows how the connectivity of an MLP is shaped by the DNM. To apply biological spatial principles to non-spatial MLPs, we index neurons $i\in \{1,\dots ,N\}$, and their physical location $x_i$ is defined linearly such that the distance between adjacent indices is minimized. This allows us to define "spatial" distributions where connectivity probabilities depend on the relative distance $|x_i-x_j|$ between neurons in adjacent layers.

Sparsity (s)

The sparsity parameter (s) defines the percentage of potential connections between the input and output layers that are absent, controlling the trade-off between computational cost and representational power.

Dendritic Distribution

The dendritic distribution governs the number of branches that connect each output neuron to the input layer, which can be seen as the number of distinct input regions a neuron integrates information from. The central parameter for this is M, which defines the mean number of dendrites per output neuron. This distribution can be implemented in one of three ways. The simplest is a fixed distribution, where every output neuron has exactly M dendrites. Alternatively, a non-spatial distribution introduces stochasticity by sampling the number of dendrites for each output neuron from a probability distribution (e.g., Gaussian or uniform) with a mean of M. Finally, a spatial distribution makes the number of dendrites for each neuron dependent on its position within the layer. Using a Gaussian or inverted Gaussian profile, this configuration implies that some neurons integrate signals from many distinct input regions (a high dendrite count), while others connect to fewer, more focused regions (a low dendrite count).

Receptive Field Width Distribution

The receptive field of an output neuron j is defined as the contiguous subset of input neurons to which jpotentially connects. We define a mapping function $\varphi \left(j\right)$ that projects the index of output neuron j to a center coordinate on the input layer. The receptive field is then the interval $[\varphi \left(j\right)-{\displaystyle \frac{W_j}{2}},\varphi \left(j\right)+{\displaystyle \frac{W_j}{2}}]$, where $W_j$ is the receptive field width determined by the parameter $\alpha$. The receptive field mirrors the concept of receptive fields in biology. This process is governed by a mean parameter, $\alpha$, which specifies the average percentage of consecutive input neurons on the input layer from which an output neuron can sample connections. This distribution can be configured in several ways: a fixed distribution assigns an identical window size $\alpha$ to all output neurons; a non-spatial distribution introduces variability by drawing each neuron’s window size from a probability distribution (e.g., Gaussian or uniform) centered on $\alpha$; and a spatial distribution links the window size to the neuron’s position in its layer, allowing for configurations where receptive windows are, for instance, wider at the center and narrower at the edges.

Degree Distribution

The degree distribution samples the number of incoming connections for each output neuron. This can be configured using a fixed distribution, where every output neuron is allocated the same degree. To introduce heterogeneity, a non-spatial distribution can be used to sample the degree for each neuron from a probability distribution. Finally, a spatial distribution allows the degree to vary based on the neuron’s position, for instance, by creating highly connected, hub-like neurons at the center of the layer. The mean degree is set by the layer size and target sparsity.

Synaptic Distribution

Once an output neuron’s total degree is determined, the synaptic distribution allocates these connections among its various dendritic branches. The allocation can be fixed, where each dendrite receives an equal number of synapses. Alternatively, a non-spatial distribution can introduce random variability in synapse counts per dendrite. A spatial distribution can also be applied, making the number of synapses dependent on a dendrite’s topological location, for example by assigning more connections to central branches versus outer ones. This distribution has a mean of ${\displaystyle \frac{N_{in}·(1-s)}{M}}$, where $N_{in}$ is the size of the input layer. While the degree distribution determines the total connectivity $k_j$ of an output neuron j, the synaptic distribution governs the partition of $k_j$ across the neuron’s $M_j$ dendritic branches. Formally, if $s_{j,b}$ is the number of synapses on the b-th dendritic branch of neuron j, the distribution ensures ${\sum }_{b=1}^{M_j}{s}_{j,b}=k_j$. This allocation can be uniform (fixed), stochastic (non-spatial), or topology-dependent (spatial), allowing specific branches, such as those in the center of the receptive field, to be more densely connected than distal branches.

Layer Border Wiring Pattern

The DNM includes a setting to control how connections are handled at the boundaries of the input layer. The default behavior is a wrap-around topology, where the input layer is treated as a ring. This means a receptive window for a neuron near one edge can wrap around to connect to neurons on the opposite edge, ensuring all neurons have a similarly structured receptive field. Alternatively, a bounded pattern can be enforced. In this mode, receptive windows are strictly confined within the layer’s physical boundaries. If a receptive field extends beyond the first or last input neuron, it is clamped to the edge. This enforces a more stringent locality, which we analyze further in Appendix I.

3.4. Network Topology and Geometric Characterization

A central hypothesis of this work is that specific topological features, such as modularity and hierarchy, confer distinct inductive biases that facilitate learning in ANNs. To test this hypothesis, it is essential to demonstrate that the DNM is not limited to a single structural configuration but rather functions as a flexible generative framework capable of accessing a diverse landscape of topologies.

In this section, we systematically vary the hyperparameters defined in Section 3.3 to characterize this landscape. Our goal is to show that by tuning the DNM’s parameters, we can controllably transition the network architecture across three distinct regimes: from unstructured random graphs, to highly modular networks, and finally to hierarchical, scale-free [4] topologies. 1.

Figure 2 illustrates this topological diversity by comparing a baseline random network with several DNM configurations in a 3-layered MLP of dimensions $98\times 196\times 196$, with $90\%$ sparsity. Each panel displays the network’s coalescent embedding [8] in hyperbolic space, its adjacency matrix, and network science metrics: characteristic path length (L), modularity (Q), structural consistency (${\sigma }_c$), and the power-law exponent of the degree distribution ($\gamma$).

To visualize the network’s latent geometry, the coalescent embedding maps nodes onto a 2D hyperbolic disk. In this representation, angular coordinates are computed via non-linear dimensionality reduction (Laplacian Eigenmaps) to cluster structurally similar nodes, while radial coordinates are derived from node popularity (degree). Consequently, this visualization reveals two key structural properties: hierarchy (nodes near the center act as hubs, while peripheral nodes are leaf-like) and modularity (angular grouping indicates community structure). The full algorithmic details are provided in Appendix A.

The baseline random network (Figure 2a) lacks structure ($Q=0.14$, ${\sigma }_c=0.04$). Figure 2b represents a DNM network with $M=3$, $\alpha =1$, and all distributions fixed, yielding high modularity ($Q=0.64$) and structural consistency (${\sigma }_c=0.76$). A key finding is that by setting a spatial Gaussian degree distribution (Figure 2c), the DNM generates a hierarchical topology that exhibits scale-free [4] properties. Specifically, the resulting degree distribution follows a power law $P\left(k\right)\sim k^{-\gamma }$ with an exponent $\gamma =2.30$. Since typical scale-free networks exhibit $2<\gamma <3$[4], this confirms that DNM can synthesize architectures with hub-like characteristics and hierarchical organization purely through parametric initialization. Similar measures are found when setting a spatial Gaussian synaptic distribution (Figure 2d, $Q=0.54$, ${\sigma }_c=0.74$), because this configuration does not alter the structure of the network much, as highlighted by the adjacency matrix depicted.

This analysis shows that DNM is a highly flexible framework that can produce a wide spectrum of network architectures. This ability to controllably generate diverse network geometries is fundamental for analyzing the relationship between network structure and computational function in ANNs.

4. Experiments

4.1. Experimental Setup

To evaluate the structural advantage of DNM over other initialization methods, we conduct experiments over two regimes: Static Sparse Training and Dynamic Sparse Training (DST).

Static Sparse Training

We first evaluate DNM on Multilayer Perceptrons (MLPs) for image classification tasks on the MNIST [29], Fashion MNIST [49], EMNIST [13], and CIFAR10 [25] datasets. In this regime, the topology remains fixed after initialization to isolate the performance of initial sparse network.

Dynamic Sparse Training (DST)

To validate the robustness of DNM as an initialization strategy for evolving topologies, we integrate it into state-of-the art DST frameworks. We select three DST methods that represent different landscapes of topological evolution mechanisms: SET [38] utilises random link regrowth; RigL [18] adopts gradient-based link regrowth; CHTs [54] uses a network science-based Hebbian-inspired gradient-free link regrowth. A detailed description of these models is provided in Appendix H.2. By testing DNM on these fundamentally different regrowth strategies, we aim to prove that the benefits of the initialization are not limited to a specfic training paradigm. Finally, we apply DNM to Transformer [46] models for machine translation tasks on the Multi30k en-de [16], IWSLT14 en-de [10], and WMT17 [7] datasets.

Implementation details

For MLP training, we sparsify all layers except the final layer to prevent disconnected output neurons, noting that the final layer has relatively fewer connections compared to previous layers. Comprehensive parameter settings are detailed in Appendix D, and sensitivity tests on the hyperparameters of the DNM are provided in Appendix E.

Baseline methods

We compare the performance of the DNM initialization against baseline topologies found in the literature. On static sparse training, we compare DNM with a randomly initialized network, the Bipartite Small World (BSW) [53], the Bipartite Receptive Field (BRF) [54], the Ramanujan graph [22] initialization techniques, the RadiX-Nets [23] and dANN-R [12] 2, which proved to be the best-performing dANN variant over our tests. We also include CSTI [52] and SNIP [30] as the baseline models, noting that their comparison is inherently unfair due to their data-informed nature. For dynamic sparse training, we compare DNM with a random initialization, BRF, BSW, Ramanujan graph, RadiX-Nets, dANN, SNIP and CSTI. Finally, for tests on Transformer models, we compare DNM with the BRF initialization, which was proven to be the state-of-the-art sparse initialization method in previous studies [54].

4.2. MLP for Image Classification

Static sparse training

As an initial evaluation of DNM’s performance, we compare it to other topological initialization methods for static sparse training for image classification tasks. On all benchmarks, DNM outperforms the baseline models, as shown in Table 1. Analyzing the best-performing DNM networks is crucial to understanding the relationship between network topology and task performance. This aspect is assessed in Section 5.

Dynamic sparse training

We first test DNM on the baseline dynamic sparse training methods, SET and RigL. The results are shown and discussed in Appendix B, proving that DNM outperforms the other sparse initialization methods of MLPs (99% sparsity) over the datasets tested.Table 2 shows the result of the same tests on the state-of-the-art DST method, CHTs. Not only does DNM exhibit high performance for this task, but it can also surpass the input-informed CSTI method.

4.3. Transformer for Machine Translation

We assess the Transformer’s performance on a machine translation task across three datasets. We take the best performance of the model on the validation set and report the BLEU score on the test set. Beam search, with a beam size of 2, is employed to optimize the evaluation process. On the Multi30k and IWSLT datasets, we conduct a thorough hyperparameter search to find the best settings for our DNM model. For the WMT dataset, in contrast, we simply use the best settings found in the previous tests. This approach helps to verify that DNM performs well even without extensive, dataset-specific tuning. DNM markedly improves the performance of the CHTs algorithm (Table 3 and Table 4).

5. Results Analysis

To understand which network structures are inherently best suited for specific tasks, we analyze the topologies of the top-performing models from our static sparse training experiments. Static sparse training is ideal for this analysis because its fixed topology allows us to link network structure to task performance directly, isolating it as a variable in a way that is impossible with dynamic methods. Figure 3 shows the adjacency matrices of these models, their direct bipartite graph representations, and their key metrics in network science. For image classification on Fashion MNIST and EMNIST, the optimal network’s topology is identical, and very similar to that on MNIST. These networks are scale-free ($\gamma \le 3$) [4] and exhibit a small characteristic path. Finally, we obtain contrasting results when assessing the network adopted for CIFAR-10 classification. Its higher $\gamma$ parameter indicates that this network lacks hub nodes, possibly hinting that for more complex datasets like CIFAR-10, a more distributed and less hierarchical connectivity pattern is advantageous. Such a topology might promote the parallel processing of localized features across the input space, which is critical for natural image recognition, where object location and context vary significantly. In Appendix N, we expand our analysis by examining the topologies of the best-performing and worst-performing models on each of the dataset, and Appendix E gives a more detailed analysis of the best parameter combinations for each of the tests performed.

Overall, this analysis reveals a compelling relationship between task complexity and optimal network topology. While simpler, more structured datasets like MNIST and EMNIST benefit from scale-free, hierarchical architectures that can efficiently integrate global features through hub neurons, the more complex CIFAR-10 dataset favors a flatter, more distributed architecture. This underscores the potential of the DNM: its parametric flexibility allows it to generate these distinct, task-optimized topologies, moving beyond a one-size-fits-all approach to sparse initialization.

6. Conclusion

In this work, we introduced the Dendritic Network Model (DNM), a novel generative framework for initializing sparse neural networks inspired by the structure of biological dendrites. We have shown that the DNM is a highly flexible tool capable of producing a wide spectrum of network architectures, from modular to hierarchical and scale-free, by systematically adjusting its core parameters.

Our extensive experiments across multiple architectures demonstrate the effectiveness of our approach. At extreme sparsity levels, DNM consistently outperforms alternative topological initialization methods in both static and dynamic sparse training regimes, sometimes exceeding the performance of the data-informed CSTI and SNIP.

Crucially, our studies reveal a compelling link between network topology and task complexity. While simpler datasets benefit from hierarchical, scale-free structures, more complex visual data favors distributed, non-hierarchical connectivity. This finding underscores that the optimal sparse topology is task-dependent and highlights the power of DNM as a principled platform for exploring the relationship between network geometry and function.

Scale-Free Network 

A Scale-Free Network [4] is characterized by a highly uneven distribution of degrees amongst the nodes. A small number of nodes, called hubs, have a very high degree, and a large number of nodes have very few connections. The degree distribution of scale-free networks follows a power law trend $P\left(k\right)\sim k^{\gamma }$, where $\gamma$ is a constant smaller than 3. In contrast, nodes in random networks are distributed following a Binomial distribution.

Watts-Strogatz Model and Small-World Network 

A Small-World Network [48] is characterized by a small average path length. This property implies that any two nodes can communicate through a short chain of connections. The Watts-Strogatz [48] model is well known for its high clustering and short path lengths. This network is modelled by a parameter $\beta$ between 0 and 1 that can determine its level of clustering. When $\beta$ takes low values ($\beta \approx 0$), the WS network is a highly clustered lattice. On the other hand, when $\beta$ approaches 1, the network becomes a random small-world graph. Intermediate values of $\beta$ can generate a clustered network that maintains small-world connectivity.

Formally, a network is small-world when the path of length L between two randomly chosen nodes grows proportionally to the logarithm of the number of nodes (N) in the network, that is:

$$L\propto logN.$$

(A1)

Structural Consistency 

Structural consistency [34] is an index based on the first-order matrix perturbation of the adjacency matrix, which represents the predictability of the network structure. A perturbation set $\Delta E$ is randomly sampled from the original link set E. Identifying as $E^L$ the links ranked as the top L according to the structural perturbation method [34], with $L=|\Delta E|$, the structural consistency ${\sigma }_c$ is calculated as:

$${\sigma }_c={\displaystyle \frac{|E^L\cap \Delta E|}{\Delta E}}.$$

(A2)

Modularity 

Modularity [43] quantifies the tendency of nodes in the network to form distinct communities (or clusters). This measure ranges from -1 to 1. A high modularity score (close to 1) hints at the presence of dense connections between nodes within communities, but sparse connections between nodes belonging to different communities. A modularity score close to 0, in contrast, suggests that the network lacks any community organization and the interaction between nodes is essentially uniform. When modularity approaches -1, the network exhibits an anticommunity structure. This means that nodes are strongly connected across the network, and there is little differentiation into separate groups. In other words, a negative modularity represents a cohesive network. The formula to compute the modularity (Q) is:

$$Q={\displaystyle \frac{1}{2m}}\sum _{ij}\left[A_{ij}-{\displaystyle \frac{k_i{k}_j}{2m}}\right]\delta (c_i,c_j),$$

(A3)

where A represents the network’s adjacency matrix, and $k_i$ and $k_j$ are the degrees of nodes i and j, respectively. $\delta (c_i,c_j)$ is the Kronecker delta function, which equals one if i and j are in the same community, else it equals 0.

Characteristic path length 

The characteristic path length is computed as the average node-pairs length in the network; it is a measure associated with the network’s small-worldness [9]. The characteristic path length (L) is derived by:

$$L={\displaystyle \frac{1}{n(n-1)}}·\sum _{i,j}d(i,j),$$

(A4)

where n is the number of nodes in the network, and d(i,j) is the shortest path length between node i and node j.

Coalescent Embedding 

Coalescent embedding [40] is a class of machine learning algorithms used for unsupervised dimensionality reduction and embedding complex networks in a geometric space, often hyperbolic. This method maps high-dimensional information on a low-dimensional embedding while maintaining the essential topological features of the network. This embedding reveals latent structures of the system, like hierarchical and scale-free structures. In this article, coalescent embedding maps the networks that have latent hyperbolic geometry onto the two-dimensional hyperbolic space. The approach involves 4 steps: 1) links are pre-weighted with topological rules that approximate the underlying network geometry [9]; 2) non-linear dimensionality reduction; 3) generation of angular coordinates; 4) generation of radial coordinates.

This process is illustrated in Figure 2, which showcases the results of applying a specific coalescent embedding pipeline to four different synthetic networks. The embeddings shown were generated without any initial link pre-weighting (step 1). For the non-linear dimensionality reduction (step 2), the Isomap [3] algorithm was used. Finally, the angular coordinates (step 3) were determined using Equidistant Adjustment (EA), a process that preserves the relative order of the nodes while arranging them at perfectly uniform angular intervals.

Appendix D.1. MLP for Image Classification

Models are trained for 100 epochs using Stochastic Gradient Descent (SGD) with a learning rate of 0.025, a batch size of 32, and a weight decay of $5\times {10}^{-4}$ Table A3.

Table A3. Hyperparameters of MLP on Image Classification Tasks

Hyper-parameter	MLP
Hidden Dimension	1568 (3072 for CIFAR10)
# Hidden layers	3
Batch Size	32
Training Epochs	100
LR Decay Method	Linear
Start Learning Rate	0.025
End Learning Rate	$2.5e^{-4}$
$\zeta$ (fraction of removal)	0.3
Update interval (for DST)	1 epoch
Momentum	0.9
Weight decay	$5e^{-4}$

Table A3. Hyperparameters of MLP on Image Classification Tasks

Hyper-parameter	MLP
Hidden Dimension	1568 (3072 for CIFAR10)
# Hidden layers	3
Batch Size	32
Training Epochs	100
LR Decay Method	Linear
Start Learning Rate	0.025
End Learning Rate	$2.5e^{-4}$
$\zeta$ (fraction of removal)	0.3
Update interval (for DST)	1 epoch
Momentum	0.9
Weight decay	$5e^{-4}$

All sparse models are trained at a 99% sparsity level. For dynamic methods, we used SET, RigL, and CHTs. The regrowth strategy for CHTs is CH2_L3n [41]. For our DNM, we conduct a grid search over its key hyperparameters. We tested a mean dendrite count (M) of 3. For the dendritic, degree, receptive field width, and synaptic distributions, we searched across fixed, spatial Gaussian, and spatial inverted Gaussian options. The mean receptive field width ($\alpha$) was fixed at 1.0. For the BSW baseline, the rewiring probability is searched in the set {0.0, 0.2, 0.4, 0.6, 0.8, 1.0}. For the BRF baseline, we searched the randomness parameter r over the same set of values, and also tested both fixed and uniform degree distributions.

Appendix D.2. Transformer for Machine Translation

We use a standard 6-layer Transformer architecture with 8 attention heads and a model dimension of 512. The dimension of the feed-forward network is set to 1024 for Multi30k and 2048 for IWSLT14 and WMT17. All models are trained using the Adam optimizer [24] with the noam learning rate schedule. Dataset-specific training parameters are as follows:

• Multi30k: Trained for 5,000 steps with a learning rate of 0.25, 1000 warmup steps, and a batch size of 1024.
• IWSLT14: Trained for 20,000 steps with a learning rate of 2.0, 6000 warmup steps, and a batch size of 10240.
• WMT17: Trained for 80,000 steps with a learning rate of 2.0, 8000 warmup steps, and a batch size of 12000.

We evaluated models at final sparsity levels of 90% and 95%. For Multi30k and IWSLT14, we performed a comprehensive hyperparameter search. The search for IWSLT14 included a mean dendrite count $M\in \{3,7,21\}$ and various combinations of fixed and spatial distributions for all DNM parameters. For WMT17, to assess generalization, we did not perform a new search. Instead, we directly applied the best-performing DNM configuration identified from the IWSLT14 experiments. This configuration used M=7 with a fixed dendritic distribution, alongside spatial Gaussian or inverse-Gaussian distributions for degree, receptive field width, and synapses.

Dynamic Sparse Training for Image Classification 

We analyse the sensitivity of DNM parameters for the initialization of CHTs on MLPs for image classification at 99% sparsity. The analysis is performed over MNIST, Fashion MNIST, EMNIST, and CIFAR-10. The sensitivity analysis, summarized in Table A4, evaluates the impact of DNM initialization parameters for CHTs at 99% sparsity. Across all benchmarks, the Degree Distribution consistently emerges as the most critical parameter, highlighting the paramount importance of initial network connectivity. Following in descending order of influence are the Receptive Field Width, Dendritic, and Synaptic distributions. Analyzing the top 5% best-performing configurations, we observe similar trends across various datasets (Figure A3). The most relevant findings is that a spatial gaussian receptive field width distribution is constantly preferred.

Figure A3. Sensitivity analysis of the DNM parameters for CHTs initialization over MLPs for image classification on CIFAR-10 at 99% sparsity.

Figure A3. Sensitivity analysis of the DNM parameters for CHTs initialization over MLPs for image classification on CIFAR-10 at 99% sparsity.

Table A4. Sensitivity Analysis Results for CHTs on MNIST, EMNIST, Fashion MNIST, CIFAR-10 for Image Classification. The table presents the average coefficient of variation (CV) for each DNM parameter across different configurations at 99% sparsity level. A higher CV indicates greater sensitivity of the model’s performance to changes in that parameter.

	MNIST	EMNIST	Fashion MNIST	CIFAR-10
Degree Dist	0.001538	0.001419	0.003379	0.017639
Rec Field Width Dist	0.000506	0.001585	0.001205	0.005381
Dendritic Dist	0.000525	0.001265	0.001136	0.004983
Synaptic Dist	0.000464	0.000989	0.000964	0.004306

Table A4. Sensitivity Analysis Results for CHTs on MNIST, EMNIST, Fashion MNIST, CIFAR-10 for Image Classification. The table presents the average coefficient of variation (CV) for each DNM parameter across different configurations at 99% sparsity level. A higher CV indicates greater sensitivity of the model’s performance to changes in that parameter.

	MNIST	EMNIST	Fashion MNIST	CIFAR-10
Degree Dist	0.001538	0.001419	0.003379	0.017639
Rec Field Width Dist	0.000506	0.001585	0.001205	0.005381
Dendritic Dist	0.000525	0.001265	0.001136	0.004983
Synaptic Dist	0.000464	0.000989	0.000964	0.004306

Dynamic Sparse Training for Machine Translation 

We focus on the analysis of DNM for the initialization of the CHTs model on Multi-30k for machine translation. Both at 90% and 95% sparsity, the calculated coefficients of variation do not surpass 0.01. This indicates that the model’s performance is relatively stable across different settings of each parameter, suggesting low sensitivity (Table A5). At both sparsity levels, the dendritic distribution appears to be the most sensitive, whereas M and $\alpha$ are the most stable. Analyzing the top 5% best-performing configurations, we observe that spatial distributions generally outperform fixed distributions (Figure A4).

Table A5. Sensitivity Analysis Results for CHTs on Multi30k for Machine Translation. The table presents the average coefficient of variation (CV) for each DNM parameter across different configurations at 90% and 95% sparsity levels. A higher CV indicates greater sensitivity of the model’s performance to changes in that parameter.

Parameter	Average Coefficient of Variation (CV)
	90%	95%
Dendritic distribution	0.008665	0.010520
Degree distribution	0.010910	0.014196
Receptive Field Width Distribution	0.008511	0.009607
Synaptic distribution	0.009444	0.011526
M	0.008821	0.009938
$\alpha$	0.007576	0.009607
Note: Higher CV indicates greater impact on performance.

Table A5. Sensitivity Analysis Results for CHTs on Multi30k for Machine Translation. The table presents the average coefficient of variation (CV) for each DNM parameter across different configurations at 90% and 95% sparsity levels. A higher CV indicates greater sensitivity of the model’s performance to changes in that parameter.

Parameter	Average Coefficient of Variation (CV)
	90%	95%
Dendritic distribution	0.008665	0.010520
Degree distribution	0.010910	0.014196
Receptive Field Width Distribution	0.008511	0.009607
Synaptic distribution	0.009444	0.011526
M	0.008821	0.009938
$\alpha$	0.007576	0.009607
Note: Higher CV indicates greater impact on performance.

Figure A4. Sensitivity analysis of the DNM parameters for CHTs initialization over Transformer models for machine translation on Multi30k at 90% sparsity.

Figure A4. Sensitivity analysis of the DNM parameters for CHTs initialization over Transformer models for machine translation on Multi30k at 90% sparsity.

Appendix H.1. Sparse Network Initialization

Bipartite Scale-Free (BSF)

The Bipartite Scale-Free network [53] is an extension of the Barabási-Albert (BA) model [4] to bipartite networks. We detail the steps to generate the network. 1) Generate a BA monopartite network consisting of $m+n$ nodes, where m and n are the numbers of nodes of the first and second layer of the bipartite network, respectively. 2) Randomly select m and n nodes to assign to the two layers. 3) Count the number of connections between nodes within the same layer (frustrations). If the two layers have an equal number of frustrations, match each node in layer 1 with a frustration to a node in layer 2 with a frustration, randomly. Apply a single rewiring step using the Maslov-Sneppen randomization (MS) procedure for every matched pair. If the first layer counts more frustrations, randomly sample a subset of layer 1 with the same number of frustrations, and repeat step 1. For each remaining frustration in layer 1, sequentially rewire the connections to the opposite layer using the preferential attachment method from step 1. If the second layer has more frustrations than the first, apply the opposite procedure. The resulting network will be bipartite and exhibit a power-law distribution with exponent $\gamma =2.76$.

Bipartite Small-World (BSW)

The Bipartite Small-World network [53] is an extension of the Watts-Strogatz model to bipartite networks. It is modelled as follows: 1) Build a regular ring lattice with a number of nodes $N=\#L_1+\#L_2$, with $L_1>L_2$, where $L_1$ and $L_2$ represent the nodes in the first and second layers of the bipartite network, respectively. 2) Label the N nodes in a way such that for every $L_1$ node positioned in the network, $\#L_1/\#L_2$ nodes from $L_2$ are placed at each step. Then, at each step, establish a connection between an $L_1$ node and the $K/2$ closest $L_2$ neighbours in the ring lattice. 3) For every node, take every edge connecting it to its $K/2$ rightmost neighbors, and rewire it with a probability $\beta$, avoiding self-loops and link duplication. When $\beta =1$, the generated network corresponds to a random graph.

Correlated Sparse Topological Initialization (CSTI)

The Correlated Sparse Topological Initialization (CSTI) [53] initializes the topology of the layers that interact directly with the input features. The construction of CSTI follows four steps. 1) Vectorization: Denoting as n the number of randomly sampled input data from the training set and as M the number of valid features with variance different from zero among these samples, we build an $n\times M$ matrix. 2) Feature selection: We perform feature selection by calculating the Pearson Correlation for each feature. Hence, we construct a correlation matrix. 3) Connectivity selection: Next, we construct a sparse adjacency matrix, with entries "1" corresponding to the top-k% values from the correlation matrix (where the value of k depends on the desired sparsity level). A scaling factor × determines the dimension of the hidden layer. 4) Assembling topologically hubbed network blocks: Finally, the selected adjacency matrix masks the network to form the initialized topology for each sandwich layer.

SNIP

SNIP [30] is a static sparse initialization method that prunes connections based on their sensitivity to the loss function. The sensitivity of a connection is defined as the absolute value of the product of its weight and the gradient of the loss with respect to that weight, evaluated on a small batch of training data. Connections with the lowest sensitivity are pruned until the desired sparsity level is reached. This method allows for the identification of important connections before training begins, enabling the training of sparse networks from scratch.

Ramanujan Graphs

Ramanujan Graphs [35] are a class of optimal expander graphs that exhibit excellent connectivity properties. They are characterized by their spectral gap, which is the difference between the largest and second-largest eigenvalues of their adjacency matrix. A larger spectral gap indicates better expansion properties, meaning that the graph is highly connected and has a small diameter. Ramanujan graphs are constructed using deep mathematical principles from number theory and algebraic geometry. They are known for their optimal expansion properties, making them ideal for applications in network design, error-correcting codes, and computer science. In this article, we use bipartite Ramanujan graphs as a sparse initialization method for neural networks.

In our experiments, we built bipartite Ramanujan graphs as a theoretically-grounded initialization method, following the core principles outlined by [22]. Drawing inspiration from the findings of [37], which prove the existence of bipartite Ramanujan graphs for all degrees and sizes, we constructed these graphs as follows:

• Generate d random permutation matrices, where d is the desired degree of the graph. Each permutation matrix represents a perfect matching, which is a set of edges that connects each node in one layer to exactly one node in the other layer without any overlaps.
• Iteratively combine these matchings. In each step, deterministically decide whether to add or subtract the successive matching to the current adjacency matrix. This decision is made by minimizing a barrier function that ensures that the eigenvalues remain within the Ramanujan bounds.

Bipartite Receptive Field (BRF)

The Bipartite Receptive Field (BRF) [54] is the first sparse topological initialization model that generates brain-network-like receptive field connectivity. The BRF directly generates sparse adjacency matrices with a customized level of spatial-dependent randomness according to a parameter $r\in [0,1]$. A low value of r leads to less clustered topologies. As r increases towards 1, the connectivity patterns tend to be generated uniformly at random. Specifically, when r tends to 0, BRF builds adjacency matrices with links near the diagonal (adjacent nodes from the two layers are linked), whereas when r increases 1, this structure tends to break.

Mathematically, consider an $N\times M$ bipartite adjacency matrix $M_{i,j_{i=1,...,M,j=1,...,N}},$ where M represents the input size and n represents the output size. Each entry $m_{i,j}$ of the matrix is set to 1 if node i from the input layer connects to node j of the output layer, and 0 otherwise. Define the scoring function

$$S_{i,j}=d_{ij}^{{\displaystyle \frac{1}{1-r}}},$$

(A5)

where

$$d_{ij}=min\left\{\right|i-j|,|(i-M)-j|,|i-(j-N)\left|\right\}$$

(A6)

is the distance between the input and output neurons. $S_{ij}$ represents the distance of an entry of the adjacency matrix from the diagonal, raised to the power of ${\displaystyle \frac{1-r}{r}}$. When the parameter r tends to zero, the scoring function becomes more deterministic; when it tends to 1, all scores $S_{i,j}$ become more uniform, leading to a more random adjacency matrix.

The model is enriched by the introduction of the degree distribution parameter. The Bipartite Receptive Field with fixed sampling (BRFf) sets the degree of all output neurons to be fixed to a constant value. The Bipartite Receptive Field with uniform sampling, on the other hand, samples the degrees of output nodes from a uniform distribution.

RadiX-Nets

RadiX-Nets [23] represent a family of sparse deep neural network topologies initialized "de novo," meaning they are constructed with a sparse structure from the outset rather than being pruned from a dense parent network. This approach addresses the limitations of hardware capacity by deterministically generating topologies that are much more diverse than previous sparse implementations (such as X-Nets) while preserving critical graph-theoretic properties. The construction relies on mixed-radix numeral systems and the Kronecker product to achieve connectivity.Formally, a mixed-radix topology is defined by an ordered set of integers

$$\mathcal{N}=(N_1,N_2,\dots ,N_L),$$

(A7)

which implicitly defines a numeral system representing integers in the range 0 to $N^{\prime }-1$, where $N^{\prime }={\prod }_{i=1}^L{N}_i$. he final RadiX-Net topology is constructed via the Kronecker product of adjacency submatrices from these mixed-radix systems and dense adjacency submatrices. This construction guarantees two essential properties:

• Path-connectedness: There exists at least one path between any input and output node, ensuring that information can flow through the network.
• Symmetry: Each input node has the same number of paths to each output node, promoting uniformity in information distribution.

For a RadiX-Net defined by a mean radi $\mu$ and d radices (with $d\sim lo{g}_{\mu }\left(N^{\prime }\right)$), the density of the graph scales asymptotically as ${\displaystyle \frac{1}{{\mu }^{d-1}}}$, allowing for significant sparsity while maintaining expressive power.

dANNs

Dendritic Artificial Neural Networks (dANNs) [12] incorporate the structural connectivity and restricted sampling properties of biological dendrites to improve parameter efficiency and robustness9. Unlike the point-neuron model of standard ANNs, a dANN splits the computational unit into two layers: a dendritic layer and a somatic layer. Input data is first fed into the dendritic layer via sparse connections (synaptic weights), where each dendrite performs a linear weighted sum followed by a nonlinearity. These dendritic activations are then weighted (cable weights) and summed at the soma before passing through a second nonlinearity.A defining feature of dANNs is their restricted input sampling, inspired by the receptive fields (RFs) of the visual cortex12. The authors propose three specific sampling strategies:

• Random Sampling (dANN-R): Input features are selected randomly for each dendrite.
• Local Receptive Fields (dANN-LRF): Inputs are sampled from a spatially restricted neighborhood (e.g., a $4\times 4$ pixel grid), preserving spatial locality.
• Global Receptive Fields (dANN-GRF): Sampling is restricted per soma, but dendrites belonging to that soma sample from distributed locations around a central point.

The dANN models exhibit mixed selectivity, where nodes respond to multiple classes. This property results in networks that are more robust to noise and overfitting, matching or outperforming dense networks while using orders of magnitude fewer trainable parameters.

Appendix H.2. Dynamic Sparse Training (DST)

SET [38]

At each training step, SET removes connections based on weight magnitude and randomly regrows new links.

RigL [18]

At each training step, RigL removes connections based on weight magnitude and regrows new links based on gradient information.

CHTs [54]

Cannistraci-Hebb Training (CHT)[42] is a brain-inspired gradient-free link regrowth method. It predicts the existence and the likelihood of each nonobserved link in a network. The rationale is that in complex networks that have a local-community structure, nodes within the same community tend to activate simultaneously ("fire together"). This co-activation encourages them to form new connections among themselves ("wire together") because they are topologically isolated. This isolation, caused by minimizing links to outside the community, creates a barrier that reinforces internal signaling. This strengthened signaling, in turn, promotes the creation of new internal links, facilitating learning and plasticity within the community. CHTs enhances this gradient-free regrowth method by incorporating a soft sampling rule and a node-based link-prediction mechanism.

Experimental Design 

We identified the best-performing DNM hyperparameter configuration from the static sparse training experiments for each of the four datasets: MNIST, Fashion MNIST, EMNIST, and CIFAR-10. We then performed a cross-transfer analysis where the optimal configuration for a source dataset was used to initialize MLP models for the other three target datasets. We compared these transferred topologies against baseline initialization methods and against the DNM models specifically tuned for the target task ("DNM (Fine-tuned)").

Results 

The results, summarized in Table A10, reveal two distinct trends in topological transferability. First, we observe high transferability among the simpler grayscale datasets. Regardless of whether the parameters are transferred from MNIST, EMNIST, or Fashion MNIST, the resulting DNM models consistently outperform the baseline initialization methods on the other grayscale targets. In these cases, the transferred performance is often comparable to the task-specific fine-tuned models.

However, a significant performance gap emerges when transferring topologies from simpler tasks to the more complex CIFAR-10 dataset. As shown in Table A10, configurations optimized for MNIST, EMNIST, or Fashion MNIST yield poor performance when applied to CIFAR-10 (approximately 47-48% accuracy compared to 58.71% for the fine-tuned model). This suggests that the structural priors learned from simpler inputs are insufficient for the complexity of natural images. Consequently, for practical scenarios involving image classification with MLPs, we recommend utilizing the parametric configuration derived from CIFAR-10. This configuration acts as a more robust baseline for complex tasks. The specific parameters for this recommended configuration are detailed in Table A11.

Table A10. Performance of transferred DNM topologies on static sparse training at 99% sparsity. We evaluate the transferability of the single best hyperparameter configuration found on each source dataset applied to the other targets. Scores represent accuracy averaged over 3 seeds ± standard error. Bold values denote the best performance among transfer and baseline methods.

	Static Sparse Training
	MNIST	Fashion MNIST	EMNIST	CIFAR10
FC	$98.80\pm 0.00$	$90.87\pm 0.02$	$87.08\pm 0.04$	$62.35\pm 0.13$
CSTI	$98.11\pm 0.03$	$88.55\pm 0.18$	$84.74\pm 0.06$	$52.60\pm 0.25$
SNIP	$98.03\pm 0.03$	$88.65\pm 0.07$	$85.19\pm 0.04$	$61.89\pm 0.48$
Random	$95.58\pm 0.03$	$86.76\pm 0.05$	$78.42\pm 0.26$	$54.75\pm 0.15$
BSW	$97.27\pm 0.05$	$87.87\pm 0.10$	$82.92\pm 0.05$	$56.26\pm 0.04$
BRF	$97.28\pm 0.03$	$87.78\pm 0.14$	$82.88\pm 0.02$	$54.86\pm 0.08$
Ramanujan	$96.39\pm 0.10$	$86.44\pm 0.14$	$81.78\pm 0.08$	$54.61\pm 0.32$
RadiX-Nets	$97.06\pm 0.12$	$88.02\pm 0.05$	$82.65\pm 0.11$	$50.90\pm 0.23$
dANN-R	$96.10\pm 0.11$	$86.52\pm 0.01$	$80.64\pm 0.11$	$51.57\pm 0.23$
DNM (Fine-tuned)	$98.07\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$58.71\pm 0.28$
Transferred from MNIST	$98.07\pm 0.09$	$88.92\pm 0.09$	$85.80\pm 0.04$	$48.24\pm 0.25$
Transferred from Fashion MNIST	$98.05\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$47.52\pm 0.39$
Transferred from EMNIST	$98.05\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$47.52\pm 0.39$
Transferred from CIFAR-10	$97.48\pm 0.38$	$83.48\pm 0.02$	$88.62\pm 0.11$	$58.71\pm 0.28$

Table A10. Performance of transferred DNM topologies on static sparse training at 99% sparsity. We evaluate the transferability of the single best hyperparameter configuration found on each source dataset applied to the other targets. Scores represent accuracy averaged over 3 seeds ± standard error. Bold values denote the best performance among transfer and baseline methods.

	Static Sparse Training
	MNIST	Fashion MNIST	EMNIST	CIFAR10
FC	$98.80\pm 0.00$	$90.87\pm 0.02$	$87.08\pm 0.04$	$62.35\pm 0.13$
CSTI	$98.11\pm 0.03$	$88.55\pm 0.18$	$84.74\pm 0.06$	$52.60\pm 0.25$
SNIP	$98.03\pm 0.03$	$88.65\pm 0.07$	$85.19\pm 0.04$	$61.89\pm 0.48$
Random	$95.58\pm 0.03$	$86.76\pm 0.05$	$78.42\pm 0.26$	$54.75\pm 0.15$
BSW	$97.27\pm 0.05$	$87.87\pm 0.10$	$82.92\pm 0.05$	$56.26\pm 0.04$
BRF	$97.28\pm 0.03$	$87.78\pm 0.14$	$82.88\pm 0.02$	$54.86\pm 0.08$
Ramanujan	$96.39\pm 0.10$	$86.44\pm 0.14$	$81.78\pm 0.08$	$54.61\pm 0.32$
RadiX-Nets	$97.06\pm 0.12$	$88.02\pm 0.05$	$82.65\pm 0.11$	$50.90\pm 0.23$
dANN-R	$96.10\pm 0.11$	$86.52\pm 0.01$	$80.64\pm 0.11$	$51.57\pm 0.23$
DNM (Fine-tuned)	$98.07\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$58.71\pm 0.28$
Transferred from MNIST	$98.07\pm 0.09$	$88.92\pm 0.09$	$85.80\pm 0.04$	$48.24\pm 0.25$
Transferred from Fashion MNIST	$98.05\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$47.52\pm 0.39$
Transferred from EMNIST	$98.05\pm 0.09$	$88.86\pm 0.21$	$85.63\pm 0.10$	$47.52\pm 0.39$
Transferred from CIFAR-10	$97.48\pm 0.38$	$83.48\pm 0.02$	$88.62\pm 0.11$	$58.71\pm 0.28$

Discussion 

This experiment strongly suggests that the structural principles identified by DNM as optimal for MNIST serve as a powerful and generalizable prior for other image classification tasks. The ability to transfer a high-performing topology with minimal performance loss has significant practical implications, as it can drastically reduce the computational cost associated with architecture search for new applications. This finding reinforces the idea that bio-inspired, structured initialization is not merely a task-specific trick but a robust strategy for building efficient sparse networks.

Appendix M.1. MLP for Image Classification

For Multi-Layer Perceptrons (MLPs) applied to image classification tasks, particularly those involving complex visual features, we recommend the configuration detailed in Table A11. This setup was found to be the most effective for the CIFAR-10 dataset, and proved to stably outperform other initialization baselines on diverse datasets (Appendix K).

Table A11. Recommended DNM parameter configuration for practical image classification scenarios. This configuration corresponds to the optimal settings identified for CIFAR-10.

Parameter	Value / Distribution
Sparsity (s)	0.99
Mean Dendrites	3
Dendritic Distribution	Fixed
Receptive window width	1.0
Receptive window width distribution	Spatial Inverse-Gaussian
Degree Distribution	Spatial Inverse-Gaussian
Synaptic Distribution	Spatial Inverse-Gaussian
Layer Border Wiring Pattern	Wrap-around

Table A11. Recommended DNM parameter configuration for practical image classification scenarios. This configuration corresponds to the optimal settings identified for CIFAR-10.

Parameter	Value / Distribution
Sparsity (s)	0.99
Mean Dendrites	3
Dendritic Distribution	Fixed
Receptive window width	1.0
Receptive window width distribution	Spatial Inverse-Gaussian
Degree Distribution	Spatial Inverse-Gaussian
Synaptic Distribution	Spatial Inverse-Gaussian
Layer Border Wiring Pattern	Wrap-around

Appendix M.2. Transformers for Machine Translation

For Transformer architectures applied to machine translation, we recommend the configuration that successfully transferred from IWSLT14 to the large-scale WMT17 dataset (Section 4.3). As shown in Table A12, this configuration uses a higher dendrite count ($M=7$) compared to MLPs and employs spatial distributions to organize connectivity.

Table A12. Recommended DNM configuration for Transformers on Machine Translation tasks (derived from IWSLT14 to WMT17 transfer).

Parameter	Value / Setting
Sparsity (s)	0.90
Mean Dendrites	7
Dendritic Distribution	Fixed
Receptive window width	1.0
Receptive window width distribution	Spatial Gaussian
Degree Distribution	Spatial Inverse-Gaussian
Synaptic Distribution	Spatial Inverse-Gaussian
Layer Border Wiring Pattern	Wrap-around

Table A12. Recommended DNM configuration for Transformers on Machine Translation tasks (derived from IWSLT14 to WMT17 transfer).

Parameter	Value / Setting
Sparsity (s)	0.90
Mean Dendrites	7
Dendritic Distribution	Fixed
Receptive window width	1.0
Receptive window width distribution	Spatial Gaussian
Degree Distribution	Spatial Inverse-Gaussian
Synaptic Distribution	Spatial Inverse-Gaussian
Layer Border Wiring Pattern	Wrap-around

Powerlaw Gamma 

Regarding the degree distribution exponent ($\gamma$), we see Fashion-MNIST aligning with CIFAR-10. Both datasets exhibit a positive correlation between accuracy and $\gamma$ ($r=0.31$ for Fashion-MNIST, $r=0.45$ for CIFAR-10). Since a higher $\gamma$ implies a steeper decay in the degree distribution (fewer extreme hubs), this indicates that complex tasks prefer a more distributed connectivity where information processing is shared among many nodes rather than concentrated in a few central hubs. Conversely, MNIST and EMNIST show negative correlations ($r=-0.16$ and $r=-0.27$), suggesting they benefit from the strong, centralized integration provided by heavy-tailed, hub-dominated topologies.

Structural Consistency 

Across all four datasets, we observe a consistent negative correlation between Structural Consistency (${\sigma }_c$) and accuracy. Fashion-MNIST exhibits the strongest negative correlation ($r=-0.81$), reinforcing the finding that strict structural predictability is detrimental to initialization performance, regardless of the dataset complexity.

Modularity 

We observe that higher modularity (Q) correlates positively with performance across CIFAR-10. The best-performing topologies exhibit high modularity, suggesting that complex natural image recognition benefits from distinct, specialized communities of neurons that process local features independently before integration. Conversely, for the simpler MNIST and EMNIST datasets, lower modularity appears advantageous, indicating that a more integrated network structure suffices for less complex tasks.

Characteristic Path 

Like CIFAR-10, Fashion-MNIST models show a positive correlation ($r=0.39$), indicating a benefit from longer path lengths that preserve local processing. This stands in contrast to MNIST and EMNIST ($r<0$), which favor "small-world" architectures with short global integration paths.

Motivation 

At 99% sparsity, a sparse network retains only 1% of the parameters of the original FC model. A critical question arises: Does the sparse topology provide an inductive bias that facilitates learning, or could a dense network with the same tiny parameter budget perform equally well? The Dense Equivalent model is constructed by reducing the hidden dimension size H of the MLP such that the total number of trainable parameters equals the non-zero parameter count of the sparse DNM model. If DNM outperforms the DE baseline, it confirms that the arrangement of connections (the dendritic topology) is superior to a fully connected structure of equal capacity. Conversely, if the DE model performs comparably, it would suggest that the performance is governed solely by the parameter count, negating the need for complex sparse initialization.

Results 

Table A13 presents the comparison between the Dense Equivalent baseline and the sparse initializations at 99% sparsity for image classification on the CIFAR-10 dataset. To match the extreme parameter reduction of 99% sparsity, the layer widths of the dense network must be drastically reduced. This severely limits the width of the representation that can be propagated through the network. In contrast, other sparse initialization methods maintain the original wide layer dimensions but sparsify the connections. As shown in the results, DNM significantly outperforms the Dense Equivalent baseline across all datasets.

Table A13. Comparison of Static Sparse Training vs. Dense Equivalent (DE) architectures on the CIFAR-10 dataset. Models utilize the same number of trainable parameters (approx. 1% of the original FC network). The DE model uses reduced hidden dimensions to match the parameter count of the 99% sparse DNM. Results are averaged over 3 seeds ± standard error.

Static Sparse Training
	CIFAR10
FC	62.35±0.13
CSTI	52.60±0.25
SNIP	61.89±0.48
DE	56.37±0.31
Random	54.75±0.15
BSW	56.26±0.04
BRF	54.86±0.08
Ramanujan	54.61±0.32
RadiX-Nets	50.90±0.23
dANN-R	51.57±0.23
DNM	58.71±0.28

Table A13. Comparison of Static Sparse Training vs. Dense Equivalent (DE) architectures on the CIFAR-10 dataset. Models utilize the same number of trainable parameters (approx. 1% of the original FC network). The DE model uses reduced hidden dimensions to match the parameter count of the 99% sparse DNM. Results are averaged over 3 seeds ± standard error.

Static Sparse Training
	CIFAR10
FC	62.35±0.13
CSTI	52.60±0.25
SNIP	61.89±0.48
DE	56.37±0.31
Random	54.75±0.15
BSW	56.26±0.04
BRF	54.86±0.08
Ramanujan	54.61±0.32
RadiX-Nets	50.90±0.23
dANN-R	51.57±0.23
DNM	58.71±0.28