Runtime Adaptive FPGA Architecture for Dynamic Language Model Inference in Intelligent Vehicles

1. Introduction

Transformer- and vision-language-based models are increasingly investigated for intelligent cockpits, scene understanding, navigation, planning, and decision support in intelligent vehicles [1,2]. LLM-based agents have also been explored for automated traffic-scenario generation and simulation [3]. Unlike cloud servers, an onboard platform must operate under restricted power, thermal, and memory-bandwidth budgets while processing workloads whose prompt lengths vary substantially with driving context. A routine command may contain only a few tokens, whereas a multimodal scene description or accumulated conversational context can extend to thousands of tokens. Moreover, prefill and autoregressive decoding stress different resources: prefill exposes large matrix operations, while decoding is increasingly constrained by weight and KV-cache traffic.

Algorithmic compression and post-training quantization reduce model size and arithmetic cost [4], but they do not eliminate utilization loss caused by a fixed hardware mapping. Prefill attention contains a quadratic term in sequence length, whereas projection and feed-forward operations scale linearly. A resource partition selected for short prompts can therefore become suboptimal as context grows, while permanent provisioning for the longest context wastes resources during routine operation.

Hardware-aware design establishes general edge-intelligence constraints [5]. FPGA accelerators provide a suitable substrate for parameterized language-model and sequence-model inference [6,7,8]. More importantly for runtime adaptation, modern AMD FPGA devices support Dynamic Function eXchange (DFX), allowing selected hardware regions to be reconfigured while the static system remains active [9]. The unresolved question is not whether reconfiguration is possible, but when its latency and energy savings exceed the transition cost.

In-vehicle language interaction differs from conventional datacenter serving. Requests may be short spoken commands, but the model can also receive accumulated dialogue, retrieved navigation information, driver-monitoring summaries, and serialized scene descriptions. Public automotive benchmarks such as IntelliCockpitBench and STRIDE-QA illustrate this multimodal and temporally grounded setting [2,10]. The resulting context distribution is not stationary, so a single fixed compute partition can be efficient for one regime and wasteful for another.

The deployment constraint is also broader than raw throughput. Automotive systems must manage bounded latency, thermal headroom, memory capacity, and predictable behavior under changing sensor and cockpit workloads. Quantization reduces storage and arithmetic demand, yet KV-cache reads and attention-score traffic continue to grow with context. Grouped-query attention reduces the KV footprint, but it does not remove the shift from predominantly linear FFN work at short context to increasingly important attention work at long context [11,12].

DFX offers a middle ground between a permanently overprovisioned fabric and software-only scheduling. A static shell can retain memory access, safety monitoring, and control while a reconfigurable partition changes operator replication. The relevant systems question is therefore not simply whether a partial bitstream can be loaded, but whether its total transition cost is amortized over the expected residency of the next workload regime [9,13].

Quantization and algorithm–hardware co-design reduce the cost of edge Transformer inference. WAPS-Quant scales weight–activation products for low-bit post-training quantization [4]. CLAT reduces text-generation latency through clustered attention [6], FlightLLM provides a complete FPGA mapping flow for LLM inference [7], and LightMamba combines quantization with a customized FPGA architecture [8]. Memory-centric FPGA studies further show that decoding can become bandwidth dominated even when arithmetic throughput is ample [14], while MEADOW emphasizes dataflow and data-packing choices for low-power edge LLMs [15]. These systems motivate a model that separates compute allocation from memory-derived lower bounds.

Dynamic FPGA reconfiguration changes hardware allocation as the workload or inference phase changes. Its principal limitation is transition overhead, including bitstream transfer, configuration-port occupancy, pipeline drainage, and state restoration [9]. Recent phase-aware work such as PD-Swap also treats prefill and decoding as distinct operating regimes [16]. However, reporting only the latency of the selected configuration can hide a negative end-to-end result. A meaningful comparison must include the no-switch compute benefit, transition penalty, expected residency, and an explicit stay-or-switch decision.

1.1. Quantization and Compact Open Models

Post-training quantization is a practical route to edge deployment because it avoids full retraining. SmoothQuant migrates activation outliers into weights to enable accurate W8A8 execution [17], while WAPS-Quant explicitly scales weight–activation products [4]. Weight-only methods such as AWQ reduce model storage and bandwidth but do not directly provide the balanced W8A8 datapath assumed here [18]. SmolLM2-135M-Instruct is selected for this revision because its Apache-2.0 checkpoint is public, compact, and executable on a CPU for reproducible quality screening [11].

1.2. FPGA and Memory Centric Language Model Acceleration

FlightLLM demonstrates an end-to-end mapping flow on an Alveo U280 [7]. CLAT specializes attention for low-latency generation [6], while memory-centric embedded work on KV260 emphasizes that decoder throughput can be bounded by off-chip traffic even when arithmetic resources remain available [14]. Hummingbird extends this systems view across multiple resource-constrained embedded FPGA platforms [19]. These studies motivate the separation of compute, weight traffic, activation traffic, and KV-cache events retained in the present simulator.

1.3. Dynamic Reconfiguration and Research Gap

AMD UG909 and UG947 define the DFX implementation model for Zynq UltraScale+ devices, including static logic, reconfigurable partitions, reconfigurable modules, and partial bitstreams [9,13]. PD-Swap is the closest recent comparison because it specializes hardware by inference phase [16]. The present contribution addresses a different axis: context-dependent attention-versus-FFN allocation within a fixed arithmetic budget, controlled by a stateful break-even policy with dwell, hysteresis, and bandwidth awareness. The resulting method complements phase-level specialization and is evaluated with reproducible software and controller experiments. Table 1 summarizes the distinction between the proposed method and representative FPGA and adaptive-inference studies.

1.4. Automotive Inference Workload Characteristics

An intelligent-vehicle language service is a pipeline rather than an isolated text generator. Speech recognition, navigation retrieval, driver monitoring, cabin perception, and vehicle-state interfaces can all contribute context before the first model token is produced. The user-visible metric is therefore often time to first token, while continued dialogue quality depends on steady decode rate and on avoiding long-tail stalls. A hardware mode that minimizes prefill may not minimize the complete interaction when model weights and KV state compete for the same DDR interface.

Public benchmarks expose different parts of this pipeline. IntelliCockpitBench evaluates multimodal cockpit understanding across thousands of image-query pairs [2]. STRIDE-QA associates questions with temporally grounded driving scenes and provides a large dialogue corpus [10]. Neither benchmark by itself yields a ready-to-run FPGA trace: images require a vision encoder, dialogue history changes context length, and deployment prompts add system and safety instructions. The present work therefore uses actual benchmark text to anchor request length and synthetic traces only for controlled controller stress.

Context growth changes both arithmetic and memory behavior. Projection and FFN MAC counts grow linearly with L, whereas attention score/value work grows quadratically during prefill. During decode, grouped-query attention reduces the number of stored KV heads, yet each new token still reads a cache whose size grows with previous context [12]. This asymmetry motivates reallocating arithmetic resources across modes while separately tracking a memory-bandwidth floor.

Quantization changes the balance but does not eliminate it. W8A8 reduces weight and activation traffic and maps naturally to packed DSP arithmetic, while outlier-sensitive activations can degrade quality without appropriate scaling [17]. Weight-only methods such as AWQ can further reduce model traffic [18], but would require a different activation datapath and are not silently substituted for the W8A8 design evaluated here.

Predictability is as important as mean throughput in an onboard system. Reconfiguration blocks or redirects part of the fabric, and transition timing can vary with payload, configuration path, software stack, and traffic contention. For that reason, the controller uses a break-even test and residency state rather than a context threshold alone. Deadlines can override energy-oriented choices, but deadline guarantees remain outside scope until worst-case implementation timing is measured.

Taken together, prior work establishes that quantization, specialized FPGA datapaths, memory-aware scheduling, and partial reconfiguration are individually useful, but it does not fully resolve how an embedded accelerator should redistribute a fixed arithmetic budget as context length changes. Existing board-validated systems primarily optimize one mapping, one inference phase, or one memory regime. This gap motivates a finite mode library and a controller that makes each transition conditional on predicted amortized benefit rather than prompt length alone.

This study develops the runtime-adaptive architecture in six directions. First, it targets the documented ZCU102/XCZU9EG platform and a fixed 2,048-DSP arithmetic budget. Second, it aligns the workload model with the open SmolLM2-135M-Instruct checkpoint. Third, it combines checkpoint-level W8A8 screening with benchmark-derived driving-domain request statistics. Fourth, it converts the continuous allocation optimum into a four-mode integer library suitable for DFX-oriented design. Fifth, it evaluates a stateful break-even controller against static, greedy, threshold, and offline-oracle baselines over repeated stochastic traces. Sixth, it adds transition-cost, bandwidth-estimation, and controller-overhead experiments to quantify robustness rather than relying on a single nominal operating point.

2. Runtime Adaptive Architecture

The accelerator is organized as a closed control loop comprising workload perception, adaptive execution control, and a reconfigurable compute fabric. The perception layer obtains prompt length and inference phase, estimates compute demand and KV-cache traffic, and exposes latency or deadline constraints. The controller maps this state to one of a finite set of hardware modes; the fabric executes attention and feed-forward kernels with the selected DSP partition. The application-level organization, workload families, and closed-loop adaptation path are summarized in Figure 1.

2.1. Workload Perception and Computational Model

Following the standard Transformer attention and feed-forward structure [20], for sequence length L, hidden dimension d, intermediate width dff, and N layers, the prefill workload is decomposed into attention-related and feed-forward MAC counts. Attention includes Q/K/V and output projections together with score and value products; the SmolLM2 gated feed-forward block uses three linear projections.

W_att(L) = N [4Ld² + 2L²d]

(1)

W_ffn(L) = 3NLdd_ff

(2)

W(L) = W_att(L) + W_ffn(L) = αL² + βL

(3)

equations (1)–(3) make the original coefficients explicit: α=2Nd and β=N(4d²+3ddff). Prompt length is available before prefill, so no additional performance counter is required; controller and bitstream-selection delay must nevertheless be included in a physical implementation.

The perception layer deliberately uses low-cost observables. Prompt length and requested output budget are known before execution, inference phase is explicit in the runtime, and bandwidth can be estimated from counters over a finite window. The design avoids assuming exact future requests or oracle knowledge. This makes the controller implementable in a static processor or lightweight hardware state machine and allows each decision to be logged and audited.

The workload model separates terms because their scaling determines the preferred allocation. Projection and FFN work are approximately linear in L, while attention score and value products introduce the quadratic component during prefill. During decoding, arithmetic per token is smaller but KV-cache traffic grows with context. A single scalar operation count would hide these regime changes and could select a mode that is compute-efficient but memory-inefficient.

2.2. Adaptive Execution Control

The controller divides the finite DSP budget between attention and feed-forward clusters. C0 is the resident short-context mode. A candidate is accepted only when predicted saving over its expected residency exceeds transition cost. Formally, T_policy=min[T_resident, T_candidate+T_PR/K]. This rule is augmented by minimum dwell, hysteresis, phase awareness, and bandwidth smoothing so that small estimate changes do not cause rapid mode reversal.

The controller evaluates both the candidate latency and the consequence of leaving the current mode. A switch is rejected when the estimated benefit is smaller than transition cost, when minimum dwell has not expired, or when the improvement lies inside the hysteresis band. A deadline override selects the fastest predicted feasible mode, while the same state variables can be extended with implementation-specific worst-case bounds for hard real-time deployment.

Bandwidth smoothing is stateful because instantaneous counters can fluctuate with unrelated traffic. The estimator uses a recent window, while the residency model captures persistence in the workload regime. These two mechanisms address different failure modes: smoothing prevents noisy mode reversals, and residency prevents a short favorable interval from triggering a transition that cannot be amortized.

2.3. Reconfigurable Compute Fabric

The fabric contains parameterizable Q/K/V and output-projection GEMM clusters, score-product units, SoftMax, value-product units, gated feed-forward blocks, and a configurable on-chip memory fabric. Partial reconfiguration changes cluster replication and routing within a designated region while the static region retains control and memory access. A small library of routable integer configurations is used instead of a continuously varying allocation, allowing the runtime policy to make discrete, auditable mode transitions. Figure 2 shows the resulting static-shell, policy-control, and reconfigurable-compute organization.

The attention and FFN engines share a common numerical format and stream interface so that mode changes alter replication rather than operator semantics. The architecture retains SoftMax and normalization outside the packed INT8 MAC assumption, which avoids overstating DSP throughput. Local buffering is sized as a design target for reuse and burst absorption; it is not assumed to hold the complete model or KV cache.

The memory path is organized around explicit DMA transactions and a shared on-chip buffer. Weight, activation, and KV traffic are accounted for separately because they have different reuse distances and transfer sizes. This separation also provides a practical calibration path: sustained bandwidth can be measured for each traffic class and substituted into the event model without changing the controller interface.

2.4. Static Shell Reconfigurable Partition and State Continuity

The proposed floorplan separates a stable shell from one reconfigurable compute partition. The shell contains the processing-system interface, AXI interconnect, DMA descriptors, DDR access control, clocks and resets, performance counters, watchdog logic, and the policy controller. Keeping these functions static prevents a mode transition from disconnecting host control or corrupting outstanding memory transactions. The reconfigurable partition contains the replicated attention and FFN arrays plus mode-specific local buffering and routing.

Mode compatibility requires more than equal DSP totals. Every reconfigurable module must expose identical partition interfaces, clocking assumptions, reset behavior, and state-transfer semantics. Buffer contents that survive a transition must either reside in the static region or be drained to DDR before reconfiguration. The controller must therefore account for pipeline drain, DMA completion, partition isolation, bitstream transfer, reset, and warm-up—not only the nominal ICAP or PCAP transfer time.

The 2.0–3.2 MiB buffer targets are conservative relative to the device’s 40.9 Mb total on-chip RAM and leave capacity for the static shell. BRAM and URAM are treated separately in implementation planning because their port widths, aspect ratios, clocking, and physical columns affect mapping efficiency. This conservative sizing reduces memory pressure while preserving enough buffering for tile reuse and burst absorption across all four modes.

The controller observes only quantities that can plausibly be available at run time: prompt/context length, inference phase, requested output budget, smoothed bandwidth counters, current mode, and residence time. It does not assume foreknowledge of exact future requests. Expected residency is estimated from recent persistence and can be capped to avoid overconfidence. A physical implementation should log each prediction, action, transition, and observed latency so that policy errors can be separated from datapath errors.

The mode library is intentionally small. A large library may approximate the continuous optimum more closely, but it increases implementation effort, bitstream storage, validation combinations, and controller uncertainty. Four modes are sufficient to represent short, middle, long, and maximum-context regimes while keeping pairwise transition testing manageable. C0 and C3 are the minimum useful implementation subset because they exercise the widest allocation difference and bound the expected benefit.

The resulting architecture is intentionally modular. The perception layer defines observable state, the controller converts that state into a mode decision, the static shell preserves system continuity, and the reconfigurable partition changes only the compute balance. This decomposition limits the trusted control surface and makes failures attributable to sensing, policy, configuration, or datapath execution rather than to an opaque monolithic accelerator.

3. Evaluation Methodology

3.1. Model and Hardware Parameterization

The evaluation uses HuggingFaceTB/SmolLM2-135M-Instruct, an Apache-2.0 checkpoint with 30 decoder layers, hidden dimension d=576, intermediate width dff=1,536, nine attention heads, three KV heads, an 8,192-token context limit, and 134,515,008 parameters [11]. W8A8 is studied with symmetric per-output-channel weight quantization and per-token activation quantization. The architecture model uses two INT8 MACs per DSP48E2 per cycle, a 200 MHz design point, and an 80% effective arithmetic utilization factor. The hardware target is the ZCU102 XCZU9EG-2FFVB1156I. The complete model, numerical-format, and hardware parameter set is listed in Table 2.

SmolLM2-135M-Instruct is not presented as a substitute for the largest automotive language models. It is selected because its architecture is representative enough to exercise projection, grouped-query attention, gated FFN, KV-cache, and quantization behavior while remaining reproducible on a CPU. The smaller scale permits transparent software validation and rapid hardware design iteration; model-size scaling remains a separate question governed primarily by weight storage, bandwidth, and array reuse.

3.2. Target Platform and DFX Toolchain

The ZCU102 is selected because its XCZU9EG-2FFVB1156I provides 2,520 DSP slices, 600k system logic cells, and 40.9 Mb total on-chip RAM [21,22]. The board contains a 4-GB, 64-bit DDR4 SODIMM. UG1182 states that the module is rated up to DDR4-2666, while the ZU9EG/controller configuration supports DDR4-2133; the resulting 17.1 GB/s figure is a theoretical interface peak rather than sustained bandwidth [21].

Vivado 2024.2 is selected to match the cited DFX user guide and tutorial. Zynq UltraScale+ devices support a static shell with one or more reconfigurable partitions, each loaded with compatible reconfigurable modules [9,13]. The proposed design keeps memory access, runtime control, isolation, and performance counters in the static shell while changing attention/FFN replication inside the compute partition. PCAP and ICAP provide practical configuration paths; their effective transition time is treated as a controller parameter and swept in the evaluation. Table 3 summarizes the selected platform characteristics and the corresponding evaluation assumptions.

Platform constants are taken from AMD documentation, whereas frequency, arithmetic efficiency, buffer allocation, effective DDR bandwidth, and transition cost are explicit evaluation parameters. This separation makes the architecture model easy to recalibrate for another implementation or board without changing the controller equations. Software and Monte Carlo experiments use fixed seeds and recorded parameter sets so the reported tables can be regenerated deterministically.

3.3. Latency Transition and Energy Expressions

Let na and nf=NDSP−na denote DSPs assigned to attention and feed-forward kernels. The modeled compute latency is the sum of module times rather than total MACs divided by one undifferentiated DSP count.

T_compute(L,n_a) = W_att/(2fηn_a) + W_ffn/(2fη[N_DSP−n_a])

(4)

n_a*(L) = N_DSP√W_att /(√W_att + √W_ffn)

(5)

The runtime controller includes a reconfiguration penalty T_PR. A nominal 6.25 ms point, corresponding to a 2.5 MB payload at 400 MB/s effective configuration throughput, is used for the main comparison; an additional sweep from 1 to 20 ms tests whether the policy conclusion depends on this single point. Pipeline drainage, state restoration, and software service time can be absorbed into T_PR when platform-specific measurements are available.

T_PR = S_bitstream/B_ICAP = 6.25 ms

(6)

T_total = T_compute + T_PR/K

(7)

E_total = (P_static + N_DSP P_dyn)T_total

(8)

K is the expected number of consecutive requests that reuse a configuration; K=1 represents a one-shot switch. Equation (8) defines the energy accounting interface used by the controller model: measured or implementation-specific power coefficients can be inserted without changing the scheduling logic. The performance experiments in this paper focus on latency, switching behavior, numerical error, and robustness, for which reproducible data are available.

Equations (4) and (5) isolate the allocation problem under a fixed DSP budget. They do not imply that every integer split produces a routable array. The discrete library is therefore obtained by projecting the continuous trend onto a small set of implementation candidates. The analytical optimum guides candidate selection, while the event model adds memory and state-transition effects before a policy decision is accepted.

The transition term is divided by expected reuse K to express amortization. This representation makes the decision boundary interpretable: a large mode change is attractive for a persistent long-context burst but not for a single request near a crossover. The transition-cost sweep in Section 4.8 therefore evaluates both the nominal point and substantially slower or faster configuration paths.

3.4. Baselines and Evaluation Protocol

The analytical baseline is the C0 allocation. The expanded evaluation distinguishes a continuous compute optimum from the discrete C0–C3 mode library. Static, greedy, threshold, and stateful policies remain useful conceptual baselines, but only results regenerated under the revised SmolLM2/ZCU102 parameterization are reported. Values from the earlier Qwen/xczu3eg parameterization are removed rather than relabeled.

Baseline fairness requires identical model dimensions, precision assumptions, bandwidth scenarios, and output lengths. Static C0 represents a resident short-context design. Greedy selection ignores transition consequences, threshold selection uses context alone, and the proposed controller adds break-even, residency, hysteresis, phase, and bandwidth state. The offline oracle is an analysis bound and is not available to the real controller.

3.5. Discrete Configuration Library and Event Driven Simulator

The continuous optimum is not directly realizable because array dimensions, buffers, and interconnects are discrete structures. Four modes therefore retain a constant 2,048-DSP arithmetic budget. C0 favors the FFN path and serves as the resident short-context baseline; C3 favors long-context attention; C1 and C2 interpolate between them. Buffer targets range from 2.0 to 3.2 MiB to preserve headroom within the XCZU9EG on-chip memory budget. The four implementation-oriented configurations and their intended operating regimes are listed in Table 4.

For each Transformer layer, the event-cost model schedules projection, attention-score, SoftMax, value-product, and gated-FFN work. Compute time follows MAC count and allocated DSP throughput. Weight, activation, and grouped-query KV traffic are evaluated with scenario bandwidth. A stage completes at max(Tcompute,Tmemory), following the compute-versus-memory bottleneck interpretation of the Roofline model [23]. Pipeline fill/drain, host overhead, and DFX downtime remain explicit terms rather than hidden inside a single efficiency factor. The complete sequence from runtime inputs to event scheduling, mode selection, and metric collection is illustrated in Figure 3.

The event model preserves causal ordering. A layer cannot consume an activation before its producer and transfer complete; pipeline fill and drain are charged at request boundaries; DFX blocks use of the reconfigurable partition during the transition scenario. Compute and memory may overlap only where the architecture exposes independent resources. This prevents optimistic summation of isolated kernel rates.

The analytical model supplies a transparent compute-only reference, after which the event model adds grouped-query KV traffic, bandwidth scenarios, request persistence, and reconfiguration events. Agreement between the analytical crossover and the event-driven policy provides a useful consistency check, while the repeated-seed experiments quantify how strongly the controller depends on stochastic workload ordering.

3.6. Controlled Workloads Baselines and Metrics

Three controlled workload families are used for the controller study. Each family is generated as a persistence process over the listed context lengths, output lengths, and bandwidth states. Twenty independent seeds with 1,000 requests each are evaluated for every baseline comparison; the same seed sets are reused across policies to avoid trace-selection bias. STRIDE-QA tokenization provides a separate empirical anchor for the short-request regime. Table 5 defines the request count, context support, output length, and persistence setting of each workload family.

Static always uses C0. Greedy selects the lowest predicted request latency at each step and pays the transition penalty whenever the selected mode changes. The length-threshold policy uses context alone. The proposed stateful controller combines the break-even test with minimum dwell, hysteresis, estimated persistence, and bandwidth state. The offline oracle uses dynamic programming with complete future knowledge and therefore provides a lower bound on achievable latency for the same mode library and transition model. Reported uncertainty intervals are 95% confidence intervals computed as 1.96 standard errors across the 20 independent traces.

The synthetic traces are designed to isolate controller behavior under known persistence and bandwidth variation. They do not claim to reproduce the joint distribution of a production vehicle. Reporting the support, seed, and persistence parameters makes the stress test repeatable and allows a reviewer to distinguish policy robustness from dataset realism. The STRIDE-QA analysis supplies an empirical anchor for one dimension of the workload: human question length.

3.7. Reproducible Software and Numerical Evaluation

Checkpoint-level W8A8 screening uses the public SmolLM2-135M-Instruct revision 12fd25f77366fa6b3b4b768ec3050bf629380bac. Linear layers use symmetric INT8 weights per output channel and activations per token followed by dequantized arithmetic, while embeddings, normalization, RoPE, SoftMax, and lm_head remain in floating point. Perplexity is evaluated on a fixed 4,096-token WikiText-2 slice [24]. In addition, this revision executes an independent operator-level W8A8 test in Python 3.13.5 with PyTorch 2.10.0+cpu using three representative matrix shapes, 30 trials per shape, 32 tokens per trial, and deterministic seed 20260922.

Perplexity and next-token agreement characterize end-to-end checkpoint sensitivity, while the operator test measures normalized RMSE, cosine similarity, and peak relative output error for QKV-like, FFN-up, and FFN-down linear mappings. The operator inputs use Gaussian activations with a 0.2% sparse 5× outlier component to stress symmetric activation scaling. Together these measurements expose both model-level and datapath-level numerical effects of the W8A8 assumption.

For workload characterization, 390,510 human turns in the STRIDE-QA validation shard val/annotations/ego_centric_spatial_qa/annotations-00000.jsonl.gz are tokenized with the SmolLM2 tokenizer [10]. Model, dataset revision, random seeds, trace-generation rules, and controller parameters are recorded in Section 5 so that the computational experiments can be reproduced independently.

The fake-quantized path intentionally leaves embeddings, normalization, RoPE, SoftMax, and lm_head in FP32. This choice isolates linear-layer quantization, which dominates the proposed DSP datapath. A complete FPGA implementation must define accumulator width, rounding, saturation, nonlinear approximations, and scale transport; its numerical output must then be compared with the software reference token by token.

4. Results and Analysis

4.1. Experimental Structure

The evaluation combines four complementary levels: model-level quantization screening, benchmark-derived workload statistics, analytical/event-driven architecture modeling, and executed repeated-seed controller experiments. The first two establish the numerical and workload premises; the latter two test the resource-allocation hypothesis and runtime policy under controlled context, output-length, bandwidth, and transition-cost variation.

Device specifications define the design space, W8A8 screening checks the selected numerical format, STRIDE-QA establishes a realistic short-query reference, and the event-driven experiments quantify how the four-mode library behaves as context and persistence vary. Each numerical table distinguishes direct software measurements from architecture-model outputs.

This structure directly tests the paper’s central claim: resource allocation should change with context, and a reconfiguration should occur only when its expected residency benefit exceeds transition cost. Because frequency, arithmetic efficiency, bandwidth, and transition cost remain explicit parameters, the same controller can be recalibrated for different FPGA implementations without changing its decision logic.

4.2. W8A8 Software Quality

The actual CPU experiment shows a modest but measurable loss from the specified fake-quantization scheme. FP32 perplexity is 30.0631 and W8A8 perplexity is 30.9498, an absolute increase of 0.8867 or 2.949%. Next-token top-1 agreement is 88.7939%. These numbers support W8A8 as an architecture design point for this checkpoint, but they do not establish task-level safety or the quality of fully integer SoftMax, normalization, or accumulation. The measured checkpoint-level quality and reproducibility metadata are summarized in Table 6.

The observed 2.95% perplexity increase is small enough to justify continued W8A8 hardware exploration but large enough that it should not be dismissed as numerically lossless. The 88.79% agreement result shows that approximately one in nine next-token argmax decisions changes on the evaluated stream. Downstream task metrics may be more or less sensitive than perplexity, so the result is a screening gate rather than an acceptance threshold. SmoothQuant-style calibration and mixed-precision exceptions are natural next comparisons [17], while LLM.int8(), GPTQ, and AWQ provide alternative outlier or weight-only baselines [18,25,26].

4.3. Benchmark Derived Automotive Request Lengths

The STRIDE-QA shard contains 14,092 records and 390,510 human dialogue turns. Median turn length is 14 SmolLM2 tokens, p90 is 41, p95 is 48, p99 is 53, and the maximum is 54. The distribution confirms that individual driving questions are usually short. Long contexts in the architecture study should therefore be interpreted as accumulated dialogue, retrieved navigation material, system instructions, or serialized perception state—not as single questions from this shard. Table 7 reports the distribution statistics, whereas Table 8 provides the corresponding token-length histogram.

The request histogram has two architectural implications. First, C0 should remain a low-overhead resident mode because most individual text turns are short. Second, long-context support is motivated by accumulated dialogue, multimodal embeddings, retrieval, and serialized system state rather than by isolated user questions. The controller therefore treats full context length, not utterance length alone, as the relevant scheduling signal.

4.4. Revised Sequence Length and KV Traffic Analysis

Under the revised SmolLM2 dimensions, C0 remains best for very short prompts, C1 is selected in the middle range, and C2/C3 become preferable as the quadratic attention term grows. The compute-only gain rises to 22.01% at the model’s 8,192-token limit. These absolute latencies assume f=200 MHz and η=0.8 and therefore remain model outputs. The KV column is a grouped-query bandwidth floor at an effective 12.8 GB/s; it excludes weights, protocol overhead, bank conflicts, and host activity. Table 9 reports the context-dependent mode crossover, modeled prefill latency, and grouped-query KV bandwidth floor.

The revised sweep preserves the paper’s central architectural observation: resource adaptation becomes more useful as the attention share grows. Under the stated design point, the preferred mode shifts from C0 to C1, C2, and finally C3 as context length increases. Routing quality, BRAM/URAM placement, pipeline balance, DDR efficiency, and static-shell overhead enter the model as calibration factors and can move the exact crossover without changing the underlying trend.

The crossover pattern follows the arithmetic decomposition rather than an arbitrary threshold. At short context, the FFN term dominates and C0 avoids starving the FFN array. As L increases, the quadratic attention component grows until C1, C2, and finally C3 become preferable. The small benefit at L=256 and L=512 also shows why unconditional switching is undesirable: a plausible transition cost can exceed the compute saving.

The KV floor provides a second constraint on long-context performance. At L=8,192, the modeled 7.373 ms per generated token represents only grouped-query KV reads at 12.8 GB/s. Weight streaming, output projection, synchronization, and protocol inefficiency are additional. Consequently, an attention-heavy mode can improve prefill while decode remains memory-bound; the runtime should treat phase as an explicit state rather than infer it from context alone.

4.5. Controller Sensitivity and Switching Stability

The controller sensitivity experiment evaluates a 48-point grid formed by dwell D∈{1,3,5,8}, hysteresis h∈{0,0.5%,1%,2%}, and bandwidth-estimation error e∈{−20%,0,+20%}. Every grid point is evaluated on twenty independent 1,000-request multimodal traces. The nominal transition point is 6.25 ms; Section 4.8 separately sweeps this parameter from 1 to 20 ms. Table 10 summarizes the full sensitivity envelope, and Table 11 reports representative zero-error stability settings.

Across the full grid, mean gain versus static C0 spans 15.33–17.00%, while switching spans approximately 67–109 events per 1,000 requests. The ±20% bandwidth-estimation perturbation changes mode selection only rarely because the dominant ordering among C0–C3 is set primarily by context length and persistent compute imbalance in these traces. Dwell and hysteresis therefore account for most of the observed variation.

Across the tested bandwidth-estimation range, the selected-mode ordering is stable for most requests; consequently, moderate ±20% bandwidth bias has little effect on mean gain. Dwell and hysteresis primarily control switching activity: the representative zero-error settings reduce switching from 109 to 67 events per 1,000 requests as dwell increases from one to eight, while mean gain changes from 17.00% to 15.43%. This establishes a clear stability–responsiveness tradeoff rather than a single fragile tuning point.

4.6. Executed Cross-Policy Controller Evaluation

The baseline study uses the same twenty 1,000-request traces for all policies within each workload family. Static C0, greedy selection, the stateful controller, and the offline oracle are compared using identical request sequences and the same transition-cost model. This paired design isolates policy effects from trace-to-trace variation. The paired cross-policy results are reported in Table 12.

The repeated-seed comparison exposes two operating regimes. Routine traffic is already well served by C0, so the maximum available latency reduction is only about 1.7%; the stateful controller captures essentially all of this benefit with one mode change per 1,000 requests on average. In contrast, multimodal and long-context workloads strongly favor adaptation: the stateful policy reduces mean modeled request latency by 16.75% and 17.49%, respectively, relative to C0.

The stateful policy remains close to the offline oracle while avoiding the oracle’s unrealistic future knowledge. On the multimodal traces, the gap between the proposed controller and oracle is 0.38 percentage points of C0 latency; on long-context bursts it is 0.16 percentage points. Compared with greedy selection, the stateful controller lowers switching from 124.8 to 102.8 events per 1,000 multimodal requests while giving up only 0.38 percentage points of modeled gain. This is the intended role of break-even control: trading a small amount of instantaneous optimality for lower transition activity.

4.7. Executed W8A8 Operator Validation and Controller Overhead

An independent W8A8 operator experiment was executed for three Transformer-relevant linear shapes. Each case uses 30 trials of 32-token matrices, symmetric per-output-channel weight quantization, symmetric per-token activation quantization, and a sparse activation-outlier component. The measured normalized RMSE is 1.38–1.77%, mean cosine similarity is 0.999842–0.999904, and mean peak relative output error is 2.20–2.60%. The FFN-down projection is the most sensitive of the three shapes because its wider input accumulates quantization error across more products. The operator-level error statistics for all three matrix shapes are given in Table 13.

The controller implementation itself is lightweight. A host-side microbenchmark executed 200,000 multimodal decisions using the complete latency prediction, candidate search, break-even test, dwell, and hysteresis logic. On an AMD EPYC 9V74 virtual CPU with Python 3.13.5, the run completed in 1.026 s, corresponding to 5.13 µs per decision. This measurement characterizes algorithmic scheduling overhead rather than accelerator datapath latency; it shows that the policy computation is small relative to the millisecond-scale request latencies in Table 9, Table 10, Table 11 and Table 12. The microbenchmark configuration and measured decision time are listed in Table 14.

4.8. Discussion and Transition-Cost Robustness

The principal comparison with prior FPGA-LLM accelerators is methodological. FlightLLM emphasizes a complete mapping flow, memory-centric work emphasizes decoder bandwidth, and PD-Swap uses phase-specialized reconfiguration [7,14,16]. The present architecture instead asks when a fixed arithmetic budget should move between attention and FFN as context changes. This context-specialization dimension is complementary to phase specialization and can be layered beneath a coarse prefill/decode policy.

A transition-cost sweep was executed on the multimodal workload to test whether the result depends on the nominal 6.25 ms point. For T_PR = 1, 3, 6.25, 10, and 20 ms, the stateful controller achieves mean gains of 15.98%, 16.06%, 16.05%, 16.05%, and 16.01%, respectively, across twenty independent traces. Switching decreases from 125.2 events per 1,000 requests at 1 ms to 109.5 at 20 ms because the break-even rule becomes more selective. The small change in gain indicates that, for these persistent workloads, the controller automatically amortizes a broad range of transition costs.

The workload results also explain when adaptation is unnecessary. STRIDE-QA human turns are predominantly short, and the routine synthetic traces accordingly show only a 1.7% available benefit over C0. Long accumulated contexts, multimodal serialization, or retrieved state are the conditions under which C2/C3 become useful. A practical runtime should therefore treat C0 as the low-overhead resident configuration and invoke adaptation only after the context and persistence signals jointly justify it.

The main transferability parameters are achieved arithmetic efficiency, sustained memory bandwidth, and effective reconfiguration service time. They are intentionally explicit in the event model rather than absorbed into a single fitted constant. This makes the controller easy to recalibrate for another board or implementation flow and is also why the sensitivity experiments emphasize ranges instead of a single operating point.

Taken together, the checkpoint screening, operator-level quantization experiment, benchmark-derived request statistics, repeated-seed policy comparison, bandwidth-error grid, transition-cost sweep, and controller microbenchmark form a substantially stronger evidence base than a single analytical speedup curve. The resulting conclusions are therefore about context-aware allocation and cost-aware adaptation, which are the core contributions of the proposed architecture.

5. Reproducibility Statement

The computational experiments use deterministic seeds. Controller baselines use twenty 1,000-request traces per workload with seed families rooted at 20260922; the sensitivity grid reuses paired traces across parameter settings. The operator-level W8A8 experiment uses seed 20260922, Python 3.13.5, PyTorch 2.10.0+cpu, 30 trials per matrix shape, and 32 tokens per trial. The controller microbenchmark uses 200,000 pre-generated requests on an AMD EPYC 9V74 virtual CPU. Checkpoint-level screening uses SmolLM2 revision 12fd25f77366fa6b3b4b768ec3050bf629380bac, and STRIDE-QA uses the named validation shard and recorded dataset revision.

For each controller request, the reproducibility log stores context length, output length, effective bandwidth state, current mode, selected candidate, transition decision, predicted request latency, realized modeled latency, and residence count. The full baseline comparison, 48-point dwell/hysteresis/bandwidth grid, and five-point transition-cost sweep are generated from these logs. Pairing the same traces across policies removes one important source of sampling noise.

The architecture equations are parameterized by model dimensions, DSP allocation, frequency, arithmetic efficiency, KV geometry, bandwidth, and transition cost. Consequently, regenerating the study for a different checkpoint or FPGA requires changing parameter files rather than rewriting policy logic. This separation between model/platform parameters and controller code is useful for both teaching and implementation-oriented follow-on work.

6. Conclusion

This work presents a runtime-adaptive FPGA architecture for dynamic language-model inference in intelligent vehicles. A four-mode library redistributes a fixed 2,048-DSP budget between attention and feed-forward processing, while a stateful break-even controller decides whether the expected residency benefit justifies a mode transition. The evaluation is grounded by SmolLM2-135M-Instruct, STRIDE-QA request statistics, checkpoint-level W8A8 screening, an executed operator-level quantization experiment, and repeated-seed controller tests. The analytical sweep predicts a 22.01% compute-only benefit at 8,192 tokens, and the executed controller study achieves 16.75% and 17.49% mean modeled latency reductions for multimodal and long-context workloads relative to static C0.

The controller results show that adaptation is most useful when context is both long and persistent. For routine short traffic, C0 remains an effective resident configuration and the stateful policy performs only about one switch per 1,000 requests. For multimodal traffic, dwell and hysteresis reduce mode churn while keeping the result close to the offline oracle. A transition-cost sweep from 1 to 20 ms changes mean gain by less than 0.1 percentage point around the nominal operating region, demonstrating that the break-even logic automatically filters transitions that cannot be amortized.

The numerical experiments also support the W8A8 datapath assumption: representative linear layers exhibit 1.38–1.77% normalized RMSE with cosine similarity above 0.99984, and the controller itself requires only about 5.13 µs per host-side decision in the executed microbenchmark. Overall, the study shows that a small discrete mode library, coupled with explicit transition economics, can capture the context-dependent shift from FFN-heavy to attention-heavy execution without requiring continuous hardware retuning.