Lightweight Autonomous Navigation on Nano-UAVs: A Stem-Optimized Depthwise Separable CNN with 540K MACs

1. Introduction

Nano-UAVs are small enough to fly through corridors and scan warehouse interiors without GPS. Platforms like the Crazyflie 2.1 weigh under 50 g and relay sensor data in real time [1,2]. The problem is what they carry for computation. A GAP8 processor has roughly 320 kB of usable SRAM; its compute budget sits in the single-digit milliwatt range [3,4]. Fitting any useful neural network into those constraints turns out to be surprisingly difficult [5,6].

PULP-Dronet v2 [7] was an early proof that end-to-end visual navigation could work on the GAP8: 19 frames per second, 320 kB footprint. Lamberti et al. then released PULP-Dronet v3 [8] with a 66,000-image dataset carrying joint collision and steering labels. They tested three block types—residual (RB), depthwise separable (D+P), and inverted residual linear bottleneck (IRLB)—at four channel-width factors ($\gamma \in \{1,2,4,8\}$). This produced models from 2.9k parameters (1.1M MACs) up to 51k parameters (12M MACs). The smallest, Tiny-PULP-Dronet v3 at $\gamma =/8$, runs at 139 frames/s. It also loses 6 percentage points of collision accuracy.

When we profiled the $\gamma =/4$ D+P variant layer by layer, one problem was immediately apparent. The $5\times 5$ first convolution alone consumed approximately 2 million MACs, or about 83% of the model’s entire 2.4M total. The three D+P feature-extraction blocks that follow it together account for barely 300K. That ratio seems difficult to justify, so we asked whether it could be fixed without redesigning the rest of the network.

Our answer is a simple factorization: replace the $5\times 5$ standard convolution stem with a $3\times 3$ depthwise convolution and a $1\times 1$ pointwise convolution in sequence. On its own this cuts stem MACs by roughly $12\times$ at $\gamma =/4$. We also replaced two training choices that seemed poorly matched to a network this small: the fixed learning rate (shifted to cosine annealing) and the offline-only augmentation (shifted to per-epoch online color jitter). Together, the three modifications push the $\gamma =/4$ model to 83.97% collision accuracy at 540K MACs, $4.4\times$ below the original, and within 0.03 percentage points of the full 12M-MAC model.

Our contributions are threefold: (i) a $3\times 3$ depthwise + $1\times 1$ pointwise factorization of the stem that cuts first-layer MACs by over $5\times$ without touching any downstream layer; (ii) cosine-annealing scheduling and per-epoch color-jitter augmentation—neither requiring architectural changes—that together produce consistent accuracy gains in the PULP-Dronet v3 pipeline; (iii) a 6409-parameter model that reaches 83.97% collision accuracy and 0.372 RMSE at 540K MACs, a +2.97 pp improvement over the same-$\gamma$ baseline at $4.4\times$ lower compute.

2. Related Work

2.1. Autonomous Navigation for Nano-UAVs

DroNet [9] showed early on that a residual CNN trained partly on car and bicycle footage could fly a quadrotor down corridors using monocular vision. The idea was promising but the model—1.6M parameters, 320 kB on disk—was too big for anything under 50 g. PULP-Dronet [10] shrank it to fit the GAP8 on a 27 g Crazyflie 2.1. Then PULP-Dronet v2 [7] added neural architecture search and deployment optimization to push throughput to 19 frames/s. PULP-Dronet v3 [8] is the current iteration. It introduced depthwise separable and inverted residual blocks, published a 66k-image dataset with joint collision/steering labels, and used the $\gamma$ factor to trade accuracy against model size. At $\gamma =/8$ (2.9k parameters, 139 frames/s) you get speed but lose 6 pp of accuracy.

2.2. Efficient CNN Architectures

MobileNet v1 [11] showed that depthwise separable convolutions offer roughly an order of magnitude fewer MACs than standard convolutions, at usually-tolerable accuracy cost. MobileNet v2 [12] added inverted residual blocks and linear bottlenecks. EfficientNet [13] demonstrated that scaling depth, width, and resolution together along a compound coefficient beats scaling any one dimension alone. ShuffleNet [14] went a different route, using channel shuffling to cut the cost of group convolutions.

Until recently, very little work focused on the stem layer itself. RegNet [15] and ConvNeXt [16] both found that changing supposedly minor design choices in the first convolution can shift the accuracy–efficiency curve more than expected. We borrow that insight, but our hardware target has about 1000× less memory than a GPU workstation, which changes the calculus considerably.

2.3. Knowledge Distillation and Model Compression for Drones

Distillation [17] lets a small student network mimic a larger teacher’s outputs, and the $\gamma$ parameter in PULP-Dronet v3 plays a loosely similar role—the full-sized model is the implicit reference. INT8 quantization [18] is already available in that pipeline and incurs only minor accuracy loss. Pruning [19] removes redundant filters. NAS [3,7] can discover good architectures automatically, though the search itself is expensive. What we propose is orthogonal to all of these: the stem change is architectural, and the training modifications are independent of any post-hoc compression.

2.4. Training Strategies for Small Models

Training compact models is harder than it might seem [20]. A learning rate that converges cleanly with 51k parameters can oscillate or stall at 6.4k. Cosine annealing [21] helps here because its smooth decay avoids the instability that fixed rates cause late in training. Online augmentation—brightness and contrast jitter applied fresh each epoch rather than baked in during preprocessing—has shown clear benefits for visual navigation tasks [22,23]. The PULP-Dronet v3 baseline uses neither: learning rate is fixed at ${10}^{-3}$, and augmentation runs once before training. We found that correcting both defaults matters at $\gamma =/4$.

3. Methodology

3.1. Baseline Architecture: PULP-Dronet v3

We build on the D+P variant of PULP-Dronet v3 [8]. The pipeline has five stages: a stem convolution, max-pooling, three depthwise separable blocks with progressively wider channels, dropout, and a fully connected head that outputs both steering angle and collision probability. Each D+P block applies $3\times 3$ depthwise then $1\times 1$ pointwise convolution twice in sequence, with batch normalization and ReLU6 after each pair. Where the channel count changes between blocks, a $1\times 1$ bypass branch handles the residual connection.

The stem is a standard $5\times 5$ convolution with stride 2 that maps the grayscale input image ($1\times 200\times 200$) to $C=32/\gamma$ channels. The output spatial size is $100\times 100$. At $\gamma =/4$ ($C=8$), the MAC count for this single layer is $25·8·100·100\approx 2.0\mathrm{M}$, which is around 83% of the model’s total 2.4M MACs. The feature extraction blocks that follow it share the remaining 17%.

3.2. Proposed Architecture: Stem-Optimized D+P CNN

The change is simple: swap the $5\times 5$ stem for a $3\times 3$ depthwise convolution (stride 2) on the single input channel, followed by a $1\times 1$ pointwise convolution that expands from 1 to C channels. The standard stem costs $25·C$ MACs per output position; the factorized one costs $9+C$. At $C=8$ that drops from 200 to 17 per position. Both new layers use batch normalization and ReLU6, same as everywhere else in the network.

Everything after the stem stays the same. Max-pooling ($2\times 2$, stride 2), then three D+P blocks at channel widths $8\to 8\to 16\to 32$. The FC layer takes the flattened $32\times 7\times 7=1,568$-dimensional vector and maps it to 2 outputs. Total parameter count: 6409. Total MACs: about 540K.

Figure 1 shows the full architecture. Top panel: data flow from input through the factorized stem, the three D+P blocks, and the dual-output head. Bottom panel: inside one D+P block.

3.3. Training Configuration

We train on the official PULP-Dronet v3 dataset [8]: 136,510 training images, 19,591 for validation, 3071 for testing. The images come from both indoor and outdoor scenes. Steering labels are yaw rates divided by 90 to get values in $[-1,+1]$. Collision labels are binary. All images are center-cropped to $200\times 200$ and converted to grayscale.

Table 1 lists our hyperparameters alongside the baseline. The differences worth explaining are the learning rate schedule, the augmentation strategy, and the dropout configuration.

Cosine annealing. With 51k parameters, a fixed ${10}^{-3}$ rate is a defensible choice. At 6.4k parameters, we found it consistently overshot in the second half of training. Cosine decay from ${10}^{-3}$ to ${10}^{-6}$ over 100 epochs [21] gives the optimizer room to move early on and tightens appropriately as training progresses—which matters more for small models than large ones.

Online augmentation. In the baseline, augmentation runs once before training starts and every epoch sees the same transformed images. Applying brightness and contrast jitter (magnitude 0.3) inside the data loader means each pass produces a slightly different photometric view of the same image. The CPU overhead per image is trivial; the regularization benefit for a model this size is not.

Dropout placement and rate. Our preliminary runs with $p=0.5$ before the flatten showed clear underfitting—the validation curve plateaued well below training. At 6409 parameters, discarding half the spatial features before flattening removes too much signal. Shifting to $p=0.3$ after the flatten concentrates regularization on the FC layer, which accounts for 3136 of those 6409 weights and is where overfitting on a fixed dataset tends to originate.

3.4. Loss Function

We keep the same loss as the baseline, an unweighted sum of MSE for steering and BCE for collision:

$$\begin{array}{cc}\hfill \mathcal{L}& ={\mathcal{L}}_{\mathrm{MSE}}+{\mathcal{L}}_{\mathrm{BCE}}\hfill \\ \hfill & ={\displaystyle \frac{1}{N}}\sum _{i=1}^N{({\widehat{y}}_i^{\mathrm{steer}}-y_i^{\mathrm{steer}})}^2\hfill \\ \hfill & +{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[-y_i^{\mathrm{coll}}log\left({\widehat{y}}_i^{\mathrm{coll}}\right)-(1-y_i^{\mathrm{coll}})log(1-{\widehat{y}}_i^{\mathrm{coll}})\right]\hfill \end{array}$$

(1)

Here $y^{\mathrm{steer}}\in [-1,+1]$ is the normalized steering angle, $y^{\mathrm{coll}}\in \{0,1\}$ is the binary collision label, and N is the batch size. We used the exact same custom_loss_v3 function from the official repository so that every comparison in the paper is on equal footing.

4. Results

4.1. Main Results

Table 2 puts our model next to every PULP-Dronet v3 configuration on the same 3071-image test split. The headline number: 83.97% collision accuracy at 540K MACs, versus 81% and 2.4M MACs for the same-$\gamma$ baseline.

Look at the comparison with $\gamma =/1$: 83.97% vs. 84% despite a $22\times$ difference in MACs. Against the $\gamma =/8$ Tiny variant, we are 5.97 pp more accurate at roughly half the compute. Steering RMSE (0.372 vs. 0.373 for the baseline) barely changes, so the accuracy gain is not coming at the expense of steering quality.

4.2. Training Dynamics

Figure 2 shows the loss and accuracy curves over 100 epochs. Training loss fell from 0.755 to 0.287 without any plateau or instability. Validation loss stabilized near 0.39 after epoch 60. On the accuracy side: 80% by epoch 5, 85% by epoch 22, 87% by epoch 36, peak of 87.78% at epoch 87. We used that checkpoint for test evaluation.

The cosine schedule’s effect is visible in the curve shape. Epochs 1–30 show fast gains as the learning rate is still large. Between epochs 30 and 70, the rate has shrunk enough for careful refinement without bouncing the loss around. By the final 30 epochs the near-zero rate effectively pins the weights. Late-epoch spikes in validation loss—a recurring issue with fixed rates on networks this small—did not appear here.

4.3. Efficiency Analysis

Table 3 gives the per-component MAC breakdown. Changing the stem alone takes the total from 2.4M down to 540K. In the original network, 83% of compute goes to the first layer—before any spatial features have been extracted. After factorization that share drops to about 31%, and the D+P blocks hold the majority of the remaining budget.

4.4. Ablation Study

Table 4 puts the original configuration against our full method on the same data and evaluation splits.

The +2.97 pp gain repeated consistently across seeds (standard deviation below 0.3 pp). We did not run a full factorial isolation of each modification individually, and we treat that as a gap in the current study rather than a solved question.

5. Discussion

The $5\times 5$ stem convolution in PULP-Dronet v3 was designed for $C=32$ output channels, where it is a modest contributor to total MACs. The $\gamma$ parameter scales the downstream blocks but not the stem, so as the rest of the network shrinks the stem’s share of compute grows. At $\gamma =/4$ it has reached 83%. Factorization changes the per-position cost from $25C$ to $9+C$; the key point is that the latter grows far less steeply as C decreases, directly addressing the imbalance.

On the GAP8, MACs and latency scale roughly linearly [8]. The $\gamma =/4$ baseline runs at around 102 frames/s; cutting $4.4\times$ of the MACs should push that past 200. A nano-drone moving at 0.5–1.5 m/s does not need 200-Hz perception. The extra cycles can go to lowering the clock instead, and on a 250 mAh battery that translates to flight time, which is almost always what matters most.

Something we did not fully anticipate: three very different models—our 540K-MAC version, the $\gamma =/1$ baseline at 12M MACs, and the $\gamma =/2$ variant at 5.2M—all land near 84% accuracy. A $22\times$ compute gap producing a 0.03 pp accuracy difference is unusual. We think the explanation is the dataset, not the models. The benchmark images cover a variety of indoor and outdoor scenes but the set is finite. At some point label noise and scene coverage stop being things a bigger model can fix. Getting past 84% would likely need harder data.

The dropout story is worth calling out. With $p=0.5$ before the flatten, our early runs underfit badly—validation plateaued while training loss kept dropping. In a 6409-parameter model, zeroing half the feature maps before they reach the FC layer throws away too much signal. We moved dropout after the flatten and cut it to $p=0.3$. The FC layer holds 3136 of the total 6409 parameters, so that is where regularization does the most good.

We should be upfront about what this study does not cover. No closed-loop flights were attempted; every number comes from a static benchmark. At $\gamma =/4$ our model is around twice the size of the $\gamma =/8$ Tiny variant (6.4k vs. 2.9k parameters, 25 kB vs. 12 kB), so if flash storage is the constraint rather than inference speed, the contribution does not help. We also did not isolate each modification in a factorial ablation, leaving open the question of individual effect sizes.

Further compression is straightforward to add. INT8 quantization [18] would roughly halve the storage. Distilling from the $\gamma =/1$ teacher [17] might pick up another point of accuracy. The D+P blocks could be pruned [19]. None of that conflicts with anything we changed.

6. Conclusions

Profiling the $\gamma =/4$ variant of PULP-Dronet v3 showed that its first convolution alone was consuming 83% of the total MAC budget. Factorizing that $5\times 5$ layer into a $3\times 3$ depthwise + $1\times 1$ pointwise pair cuts its cost by roughly $12\times$. Cosine-annealing scheduling and online color jitter were added to the training loop at no inference cost. The final model reaches 83.97% collision accuracy, 0.372 steering RMSE, 540K MACs, and 6409 parameters—+2.97 pp over the baseline at the same $\gamma$, and essentially identical accuracy to the full 12M-MAC model.

What remains to be done: quantize to INT8 and fly the model on a real GAP8-based drone. We also want to try the same stem swap at $\gamma =/8$—if it works there, the result would be smaller and more accurate than Tiny-PULP-Dronet v3. Whether distillation from a larger teacher can crack the 84% ceiling is a separate question, but one that seems worth exploring.