From Algorithms to Bitstreams: A Survey of FPGA-Based Acceleration for Transformer Inference

1. Introduction

Transformer inference combines matrix operations with reductions, normalization, data movement, and state management [1]. Its bottlenecks change across encoder inference, prompt processing, autoregressive decoding, and iterative generation [1,2,3]. FPGAs expose arithmetic precision, storage, interfaces, and scheduling to customization, making this workload variability a central opportunity and design constraint.

The central question is whether a customization produces an application-level gain. Quantization can reduce bytes without increasing arithmetic throughput; sparsity can remove multiplications while adding indexing and imbalance; additional devices can increase capacity without reducing request latency. Because reported speedups often combine model, implementation, and baseline changes, comparisons must identify the source of a gain, its system boundary, and the supporting evidence.

FTRANS combined structured compression and FPGA acceleration in 2020 [4]; subsequent systems explore adaptable datapaths, memory reuse, compressed decoding, and distributed execution [2,5,6,7]. We connect these choices to their potential savings and offsetting costs, asking: under which workload, implementation, and evaluation conditions does FPGA customization produce a defensible inference benefit? 

Scope.

We synthesize 266 publication records from 2021–2026 on non-spiking Transformer or attention-based inference with an identifiable FPGA contribution. The corpus spans complete models, operators, compiler flows, and heterogeneous deployments, keeping these evidence scopes distinct. Training is discussed when it prepares inference models; earlier foundational work lies outside the quantitative denominator. The database exports and prior cohort support an evidence map with incomplete historical search reconstruction and family reconciliation (Section 2).

Relationship to prior surveys.

Prior surveys cover cross-platform algorithm–hardware optimization [8], FPGA/ASIC acceleration [9], LLM hardware [10], edge ViTs [11], and FPGA LLM inference [12]. Blankestijn et al. provide a closely related FPGA-wide taxonomy and public selection code for a stated 2024–May 2025 collection [13].

Table 1 positions our complementary contribution: benefit attribution, linking an optimization’s saving and added cost to configuration-specific positive results, counterexamples, and unresolved cases.

Analytical approach and contributions.

Figure 1 connects the survey’s three contributions:

1.

A mechanism-based decision framework. We distinguish reduced work or traffic, denser arithmetic, better utilization/overlap, and added capacity. Break-even conditions expose conversion, communication, and integration costs (Section 4, Section 7, and Section 8); they are analytical conditions, not new accelerator measurements.

2.

Configuration-level critical synthesis. Worked comparisons connect outcomes to model changes, platform, phase, quality, and measurement scope, distinguishing configuration speedups, cluster throughput, and request latency (Section 7).

3.

A traceable evidence map with sensitivity analysis. Stable identifiers, label mappings, source checks, and cohort/documentation comparisons identify findings that persist or depend on corpus composition and coding (Section 6 and Section 6.4).

Research questions.

The synthesis addresses five questions:

1.

RQ1: How do model/compiler transformations and numerical formats change the work and representation presented to an FPGA?

2.

RQ2: How do FPGA architectures realize those changes under different workload and deployment constraints?

3.

RQ3: Which technique–architecture associations appear in the retained corpus, and how sensitive are their interpretations to cohort composition and coding?

4.

RQ4: When do reported local or configuration-specific improvements support an application-level performance, energy, or quality claim?

5.

RQ5: Which evidence gaps prevent stronger conclusions, and which controlled experiments would resolve them?

The paper proceeds from methodology and technical background to model/compiler and hardware mechanisms, corpus analysis, worked comparisons, and a research agenda.

2. Review Methodology

2.1. Review Design and Evidence Use

This structured evidence map and critical synthesis uses a 266-publication corpus frozen on 17 September 2026. The date identifies a dataset revision, not a verified final search. PRISMA 2020 and PRISMA-S guide distinctions among records, reports, and studies [14,15]; the preserved materials support partial selection accounting and reproducible synthesis, not a complete historical PRISMA process.

Eligible work concerns non-spiking Transformer or attention-based inference with an identifiable FPGA implementation, including architectures, mapping flows, and specific operators. Training-only, generic neural-network, software-only, ASIC-only, and spiking work is excluded; conventional binary and ternary Transformers remain eligible. Table 2 separates relevance from the claims supported by each result.

2.2. Sources and Selection Reconciliation

The exports contain 778 Scopus, 607 IEEE Xplore, and 219 ACM Digital Library occurrences. Scopus and IEEE filenames carry 9 September 2026, but execution times, exact update queries, and platform histories are unavailable. The source package preserves an earlier intended Scopus expression separately from verified searches. Redundant ACM batches and bibliography exports add no retrievals.

Identities were reconciled using normalized DOIs, documented missing-DOI matches, and metadata corrections, retaining distinct conference/journal reports. The ledgers support $1604-578=1026$ identities and $1026-20=1006$ candidates after language exclusions (Table 3). Of 266 included publications, 265 link to the exports; CoQMoE comes from the prior cohort [16]. Citation chasing, supplementary preprints, and retrieval recall were not assessed exhaustively.

Final reconciliation removed nine of 275 working records: three spiking reports, one fine-tuning report, one retracted report, three unresolved FPGA/inference cases, and one unretrieved report. These logged retrospective amendments are not preregistered decisions, and nonretrieval is not full-text rejection. Historical screening decisions for all 1006 candidates cannot be reconstructed; subtraction would create a spurious exclusion total.

Data collection and extraction were performed entirely by the authors. The documentation does not establish independent dual-human screening, adjudication, or inter-rater agreement. Explicit PDF-review entries cover 158 retained records; other fields include inherited extractions. A targeted revision check revisited LORA, the 2.5D group-vector accelerator, APTQ+, DLLM-OPU, SHARP, and the PCIe coarse-grained accelerator [17,18,19,20,21,22]. Its separate page/table notes strengthen those examples without implying equivalent new review of the entire corpus.

2.3. Extraction, Coding, and Units of Analysis

Publication records are the descriptive units. Stable identifiers connect bibliographic, workload, hardware, precision, technique, performance, quality, and provenance fields to export, review, merge, and exclusion ledgers. Related versions are linked where identified, but no independent-study count is asserted; LORA’s related reports, for example, share experiments alongside extensions [17,23,24].

Technique and numerical-format labels are multi-label; primary architecture, workload, and precision strategy receive one coarse assignment per record. A secondary mapping harmonizes aliases, preserves originals, and leaves ambiguity residual. Architectural emphasis is not a measured bottleneck, and overlap with technique definitions limits causal interpretation. Separate uniform-format experiments do not establish runtime switching; identified comparison-baseline formats are excluded.

Numerical comparisons use model–hardware configurations and measurement boundaries (Section 7). A spreadsheet field can combine configurations, so scope, provenance, baseline, and quality must be checked per metric. One report may mix board measurements, tool estimates, and software accuracy. The inherited Inferred label describes coding, not measurement; unresolved extraction does not prove source nonreporting.

2.4. Synthesis and Robustness

The descriptive analysis uses the current 266-record denominator, with year- or group-specific denominators for conditional shares. The 2026 interval is incomplete. Sensitivity checks compare retained original and added cohorts, records with and without explicit PDF-review entries, and alternative representatives of three documented publication families (Section 6.4). These are descriptive perturbations, not independent random samples or completed family reconciliation; no confidence interval or causal effect is inferred from them.

Worked cases deliberately expose changes in representation, executed work, capacity and concurrency, integration boundaries, and conflicting source values. Selection favors interpretable source anchors and retains positive results and reversals. We neither replicate accelerators nor pool speedups across unmatched configurations, and claim no formal risk-of-bias scoring.

The source package preserves extraction, mappings, scripts, source checks, and exclusions. Search-history gaps, access-related selection, uneven verification, and incomplete family linkage limit coverage. No prospective registration or public repository deposit is implied.

2.5. Use of Generative AI

GPT-5.6 Sol and GPT-6 Astra (OpenAI) assisted manuscript writing and portions of the data-analysis scripts. Scripting assistance supported the workflow in Section 2.4: descriptive summaries and sensitivity analyses of cohorts, review documentation, and publication families. The authors provided carefully specified instructions and performed all data collection and extraction from primary publications. The source package contains the scripts and mappings. The authors take full responsibility for the manuscript, data, analysis code and procedures, and results.

3. Technical Background

An inference configuration combines model, input shape, numerical representation, execution phase, and hardware. These determine the bottlenecks against which an optimization must be evaluated.

3.1. Transformer Operators and Inference Regimes

For one attention head, let $Q\in {\mathbb{R}}^{S_q\times d_h}$, $K\in {\mathbb{R}}^{S_k\times d_h}$, and $V\in {\mathbb{R}}^{S_k\times d_v}$ denote the query, key, and value matrices. Scaled dot-product attention is

$$O={softmax}_{\mathrm{row}}\left({\displaystyle \frac{Q{K}^{\mathsf{T}}}{\sqrt{d_h}}}+A\right)V,O\in {\mathbb{R}}^{S_q\times d_v},$$

(1)

Here A is zero for permitted interactions and negative infinity for excluded ones; a causal mask excludes future positions. Projections, feed-forward layers, normalization, residuals, and positional operations complete the block [1].

For batch B, $H_q$ query heads, $d=H_q{d}_h$, and $d_v=d_h$, the two dense attention products cost approximately $4B{S}_q{S}_{k}d$ operations, counting a multiplication and addition separately. Avoiding masked positions lowers executed work; dense-equivalent GOPS must state that convention. Projections and fixed-ratio feed-forward layers scale as $BS{d}^2$.

Encoder inference and prefill expose multiple token rows and weight reuse; self-attention has $S_q=S_k=S$ and quadratic attention work. Cached autoregressive decode has $S_q=1$, giving attention work linear in retained context per token. At batch one, projections become matrix–vector operations with little weight reuse [2,3].

Table 4. Inference regimes and the hardware questions they expose. These are analytical tendencies, not classifications of measured bottlenecks.

Regime	Workload characteristics	Design and evaluation implications
Encoder or prefill	Multiple token rows; weight reuse across rows; potentially large attention intermediates.	Measure sequence latency or time to first token; examine matrix utilization and attention IO.
Batch-one decode	One new token; repeated weight access; growing per-request KV state.	Measure time per output token; examine weight traffic, cache traffic, and operator launch or control overhead.
Batched decode	Shared weights serve multiple requests, each retaining its own KV state.	Report aggregate throughput together with batch policy and per-request latency; examine memory capacity and scheduling.
Long-context decode	More keys and values are read for each query; attention reductions span more tiles.	Report context-length dependence, cache representation, achieved bandwidth, and latency distribution.

Table 4. Inference regimes and the hardware questions they expose. These are analytical tendencies, not classifications of measured bottlenecks.

Regime	Workload characteristics	Design and evaluation implications
Encoder or prefill	Multiple token rows; weight reuse across rows; potentially large attention intermediates.	Measure sequence latency or time to first token; examine matrix utilization and attention IO.
Batch-one decode	One new token; repeated weight access; growing per-request KV state.	Measure time per output token; examine weight traffic, cache traffic, and operator launch or control overhead.
Batched decode	Shared weights serve multiple requests, each retaining its own KV state.	Report aggregate throughput together with batch policy and per-request latency; examine memory capacity and scheduling.
Long-context decode	More keys and values are read for each query; attention reductions span more tiles.	Report context-length dependence, cache representation, achieved bandwidth, and latency distribution.

Batching reuses weights but expands per-request KV state and attention work; effective batch depends on admission, padding, and request lengths. Aggregate throughput therefore needs per-request latency context. In vision, resolution, patch/window structure, and token reduction similarly determine shapes beyond the model-family label.

3.2. KV-Cache Capacity and Traffic

Multi-head, multi-query, and grouped-query attention use $H_{\mathrm{KV}}=H_q$, one KV head, and shared KV-head groups, respectively [25], changing storage independently of parameter count.

For L layers, equal key and value head dimensions $d_h$, uniform context length S, and $b_{\mathrm{KV}}$ bits per stored element, the cache payload is

$$M_{\mathrm{KV}}=2LBS{H}_{\mathrm{KV}}{d}_h{\displaystyle \frac{b_{\mathrm{KV}}}{8}}\mathrm{bytes}.$$

(2)

The factor two counts keys and values. Scales, zero-points, alignment, allocation metadata, and buffers are excluded. Unequal requests replace $BS$ with the sum of retained lengths; nonuniform layers and alternative attention representations require corresponding sums or storage models.

For $L=32$, $B=1$, $S=4096$, $H_{\mathrm{KV}}=8$, $d_h=128$, and 16-bit storage, the payload is 512 MiB; increasing $H_{\mathrm{KV}}$ to 32 yields 2 GiB. These calculated examples show how context and KV-head count change feasibility.

Full-attention decode consumes retained keys and values. On-chip placement can reduce external traffic; banking improves concurrency without reducing logical reads. Layout must match traversal across layers, heads, tokens, and channels [26,27].

3.3. A Bounded Compute–Memory Model

A roofline model relates arithmetic throughput to the traffic crossing a specified memory boundary [28]:

$$\mathcal{P}\le \min \left({\mathcal{P}}_{\mathrm{peak}},\mathrm{BW}\mathcal{I}\right),\mathcal{I}={\displaystyle \frac{N_{\mathrm{ops}}}{M_{\mathrm{transferred}}}}.$$

(3)

Throughput and peak use the same arithmetic format and operation convention; $\mathrm{BW}$ is a bytes/s ceiling and $\mathcal{I}$ is operations/byte. Routing, memory ports, dependencies, and occupancy impose further ceilings, so the bound does not predict end-to-end performance.

Consider the illustrative linear layer $Y=XW$, with $X\in {\mathbb{R}}^{B\times K}$ and $W\in {\mathbb{R}}^{K\times N}$. Assume that weights are streamed once per batch, weight traffic dominates, and no sparsity is used. With $b_W$ bits per weight, the weight-only intensity is

$${\mathcal{I}}_W\approx {\displaystyle \frac{2BKN}{KN{b}_W/8}}={\displaystyle \frac{16B}{b_W}}\mathrm{operations}/\mathrm{byte}.$$

(4)

Activation traffic, metadata, and repeated transfers lower intensity; different residency assumptions require a new traffic denominator.

With hypothetical 4-bit weights, 64 GB/s bandwidth, and a 2 TOPS peak, batch one gives 4 operations/byte and a 256 GOPS memory ceiling; batch eight gives 2.048 TOPS. This illustrative crossover depends in practice on conversion, cache traffic, shapes, and utilization; it is not a measured FPGA result.

Compression reduces bytes, narrower execution can raise arithmetic throughput, and scheduling increases reuse. These benefits are distinct; extra MAC capacity cannot remove an existing bandwidth limit.

3.4. FPGA Resources, Implementation, and Deployment

FPGAs combine LUTs, registers, routing, DSP blocks, and on-chip memories. Device-dependent arithmetic widths, packing, memory ports, and interconnect constrain parallelism; spare DSP capacity does not guarantee sufficient operand delivery.

DDR/HBM adds capacity with bandwidth constraints; achieved rates depend on controllers, banking, transaction size, and pipeline demand. Hosts, PCIe, networks, and device-specific hardened engines define further boundaries. Multiple FPGAs add resources together with partitioning and synchronization costs [3,7,29].

RTL and HLS are development methods; overlays describe architectural programmability and can use either. Compilers may target overlays or generate dedicated datapaths (Table 5).

Deployment spans graph preparation, mapping, verification, synthesis, place-and-route, and host integration. Simulation and tool estimates are not board measurements. End-to-end claims also require accounting for unsupported operators and host fallbacks [30,31,32].

Resource counts require device and precision context; unused logic may coexist with routing or bandwidth limits. Power must identify device, board, or system scope and match the throughput workload and interval.

3.5. Exact Attention Fusion and Operator Coverage

Tiling attention requires preserving normalization across all key tiles. For a fixed query tile, define score tile $D_j=Q_i{K}_j^{\mathsf{T}}/\sqrt{d_h}+A_{ij}$. An online formulation retains a row maximum m, denominator ℓ, and unnormalized output Z [33]:

$$\begin{array}{cccc}\hfill m^{\prime }& =\max \{m,rowmax\left(D_j\right)\},\hfill & \hfill \alpha & =\exp (m-m^{\prime }),\hfill \\ \hfill E_j& =\exp (D_j-m^{\prime }),\hfill & \hfill {\ell }^{\prime }& =\alpha \odot \ell +E_j\mathbf{1},\hfill \\ \hfill Z^{\prime }& =diag\left(\alpha \right)Z+E_j{V}_j,\hfill & \hfill O_i& =diag{\left({\ell }_{\mathrm{final}}\right)}^{-1}{Z}_{\mathrm{final}}.\hfill \end{array}$$

(5)

Broadcast row quantities across columns; initialize $\ell =0$, $Z=0$, and $m=-\infty$, letting the first valid tile establish the maximum and guarding fully masked tiles. Rescaling preserves previous contributions; independently normalizing tiles would change attention. The formulation avoids the full score matrix and is exact in real arithmetic, but still needs finite-precision validation and retains quadratic dense-prefill work [33].

Llama 2 uses RMSNorm, RoPE, and SwiGLU [34], requiring different reductions, positional rotations, and gating from LayerNorm, learned positions, and GELU. BERT or GPT-2 support therefore does not establish Llama support; comparisons need the actual operator graph and precision boundaries.

4. Model and Compiler Techniques

For RQ1, we organize transformations by representation, executed work, graph structure, and data movement. Their net benefit depends on the regime: reduced weight traffic can help decode while conversion penalizes compute-intensive prefill.

4.1. Transformation, Planning, and Runtime Selection

Pre-deployment transformations include quantization, pruning, distillation, factorization, and graph rewriting; compiler planning assigns tiles, buffers, and schedules. Runtime selection includes token removal, expert routing, activation sparsity, and early stopping. An offline-trained policy still incurs inference-time decisions and dispatch. Table 6 separates these responsibilities; the dataset’s historical “offline” field spans techniques with runtime components.

HeatViT trains selector placement and pruning behavior but selects and packages tokens during inference [35]. DLLM-OPU instead uses training-free activation reuse, column sparsity, and entropy-based stopping across diffusion-language-model refinement steps [20]. Both avoid work, but their variable execution lengths arise at different stages and require different control structures.

4.2. Quantization as a Representation and Execution Contract

Affine integer quantization approximates a real value by

$$x\approx s(q-z),q=clip\left(round(x/s)+z,q_{\min },q_{\max }\right),$$

(6)

where $s>0$ is a scale and z a zero-point. Floating point, logarithmic formats, binary or ternary weights, and codebook indices have different arithmetic contracts. BETA and TeLLMe implement binary and ternary Transformers, respectively; these quantized models are distinct from the excluded spiking category [36,37].

Precision must be traced through storage, multiplication, accumulation, nonlinear operators, and the KV cache. W4A8 does not specify accumulation or scaling, and packed weights may expand before multiplication. With one $b_s$-bit scale and one $b_z$-bit zero-point per G weights, the payload before padding is

$$b_{\mathrm{eff}}=b_W+{\displaystyle \frac{b_s+b_z}{G}}.$$

(7)

Symmetric schemes can omit the zero-point. Smaller groups adapt more closely to local ranges but increase metadata and can fragment transfers. Table 7 connects precision choices to these implementation costs.

Accumulation follows the reduction: for K terms with $|a_k|\le A_{\max }$ and $|w_k|\le W_{\max }$, the accumulator must accommodate $K{A}_{\max }{W}_{\max }$ unless tighter bounds or rescaling are justified. Such widening is not itself an operator-level mixed-precision policy. Post-training quantization calibrates an existing checkpoint; quantization-aware training exposes learning to quantization effects. Hardware-aware choices additionally account for DSP packing, LUT use, and scale application. SHARP combines integer and power-of-two representations, while Hessian-guided quantization aligns sensitivity with row-level processing groups [21,38].

Scale groups crossing tile boundaries add fetches or partial-sum conversion. Fine-grained precision complicates packing and dispatch; fixed groups regularize execution. Quality belongs to the deployed representation, including nonlinear and cache formats.

The deployed boundaries differ even among low-bit designs. TATAA reuses a transformable datapath for INT8 matrix operations and BF16 nonlinear execution [39]; EdgeLLM uses compressed INT4 linear weights with FP16 activations, attention, and scaling [40]. APTQ+ explores a mixed 2/4-bit policy in software, whereas its reported FPGA path uses INT4 weights and FP16 intermediate arithmetic [19].

Lookup execution further separates encoding from arithmetic. TFLOP stores product-quantization indices; LUT-LLM combines vector-quantized linear computation with wider attention [41,42]. Neither an index width nor T-ACE’s packed ternary weights specifies the complete datapath [43]. The decisive question is whether saved transfers and arithmetic exceed decoding, reconstruction, and conversion costs. This distinction underlies the configuration-specific comparisons in Section 7.

4.3. Sparsity, Structural Pruning, and Work Avoidance

Weight sparsity, activation sparsity, attention-link sparsity, token removal, and expert selection modify different execution stages. Structural pruning can yield a smaller dense graph. Block and N:M patterns regularize sparse scheduling; unstructured patterns require indices, irregular delivery, and load balancing [44,45,46]. Storage savings do not ensure skipped computation: compressed values may expand into dense tiles. Conversely, skipping operations can leave memory transactions inefficient.

Column-balanced block pruning constrains retained work to regularize processing [46]. DSPViT combines semantic token grouping and finer pruning with dynamic resource allocation [47]. The latter must recover its selection and scheduling costs. If fraction f of baseline time is accelerated by factor r, and new overhead occupies fraction h of baseline time, an additive timing model gives

$$\mathrm{Speedup}={\displaystyle \frac{1}{(1-f)+f/r+h}}.$$

(8)

The bound explains why high operator sparsity can yield modest application gains. Per-lane imbalance matters; N:M conventions and grouping dimensions determine shared decoding and skippable patterns.

Long-context sparsity also disrupts regular operand delivery. Block aggregation merges indices and schedules value-row fetches for GEMM-oriented engines [48]; FAST-Prefill couples block-sparse attention with fused index generation and liveness-managed KV storage [49]. ALISTA combines exact local attention with approximate global search [50]. Unlike exact attention tiling, these methods select or approximate interactions, coupling hardware savings to task quality.

Reuse avoids work along another axis: video tubelet merging shortens sequences, FlightVGM sparsifies activations, and Diff-DiT exploits temporal differences across diffusion steps [51,52,53]. Adaptive stopping further changes iteration count [20]. Their common mechanism is reduced executed work; the distinguishing costs are discovering reusable values, representing the retained work, and maintaining utilization as shapes change.

4.4. Model Structure, Attention Variants, and Runtime Policies

Graph changes include local windows, Fourier-based attention, deformable sampling, token reduction, and mixtures of experts. Exact attention fusion preserves the mathematical operation while changing materialization; replacing interactions changes the workload itself. Fourier-attention mapping, for example, replaces conventional attention with transforms mapped to matrix operations, with reported FPGA evaluation at attention-layer scope [54].

Regular windows provide predictable addresses; deformable sampling introduces gathers and interpolation. DATA and UEDA organize execution around these irregular feature accesses [55,56]. TAGT instead constructs and merges graph neighborhoods to reduce attention work and share fetches [57]. These mechanisms reduce different sources of redundancy and have different quality conditions.

Mixture-of-experts models distinguish total capacity from parameters active per token. Routing reduces active arithmetic without eliminating inactive-weight storage or dispatch imbalance. M3ViT, Edge-MoE, and MoE-Sched address these issues in vision workloads [58,59,60]. A distributed group-vector design combines tensor-train compression, multi-head latent attention (MLA), and expert-token reordering; Hummingbird+ targets embedded MoE execution through shared weight/KV storage and chunked prefill [18,61]. MLA changes attention-state representation, whereas expert scheduling determines where and when selected computation executes. Treating both as “model compression” would hide their distinct bottlenecks.

4.5. Distillation and Factorized Representations

Distillation benefits hardware through the student’s graph: fewer layers or narrower dimensions reduce storage and computation with a task-dependent quality trade-off. Distil-Whisper and compressed speech translation illustrate this route [62,63]. A low-rank approximation $W\approx UV$, with $W\in {\mathbb{R}}^{m\times n}$, $U\in {\mathbb{R}}^{m\times r}$, and $V\in {\mathbb{R}}^{r\times n}$, changes parameter count from $mn$ to $r(m+n)$ but introduces an intermediate tensor. Tensor-train representations add contraction or reconstruction stages [18,64]. Savings therefore depend on rank, intermediate storage, and reuse of reconstructed values. Low-rank adaptation for fine-tuning is a separate mechanism that does not automatically reduce base-model inference.

4.6. Graph Transformation, Fusion, and Compiler Mapping

Fusion removes intermediate materialization when dependencies, numerical semantics, and local storage permit. DTrans, attention-fusion designs, and QUARK integrate operators under different schedules [65,66,67]; exact tiled attention must preserve the normalization state in Section 3. Fusion lengthens pipelines and increases buffering, rate-matching, and shared-unit scheduling demands. CODO addresses these through dataflow transformations, FIFO or ping-pong selection, reuse buffers, and resource-aware scheduling [68]. Loop order and tile shape determine which operands remain stationary and whether memory ports and edge tiles can sustain the schedule [69,70].

Weight residency favors reuse across inputs; output residency avoids accumulation traffic. Tile shapes, bank capacity, and producer/consumer layout and rate determine the feasible schedule.

Retargeting spans generated dataflows, reusable overlays, and operator libraries. TRAC and FINN-T specialize hardware; OPU systems execute compiled instructions; TransLib and AttentionLib expose operator construction and scheduling [30,31,32,71,72]. LegoMap jointly selects AIE tiling, allocation, fusion, and scheduling, with layer-level emulation and analytical evaluation [73]. FINN-HPC additionally exposes host-memory, packing, and driver costs in a generated Transformer’s execution path [74].

Across these techniques, the central mapping task is to turn a smaller or reorganized workload into regular, sustained execution. Compression can shift the bottleneck toward decoding or nonlinear operators; fusion can shift it toward local storage or routing. The architecture must accommodate that shifted demand, rather than merely support the transformation in isolation.

5. Hardware Organization and Architectural Emphasis

For RQ2, primary architectural emphasis denotes the contribution emphasized in coding, not an exclusive physical type or measured bottleneck. Systolic, memory-oriented, programmable, heterogeneous, and distributed properties can coexist.

5.1. Primary Emphasis and Independent Architectural Properties

The retained labels are compute-dense, memory-oriented, heterogeneous multi-engine, overlay or programmable, multi-FPGA or distributed, specialized-operator, and other or unclear. Table 8 supplies complementary properties for interpretation; it does not imply complete independent recoding of every record.

CD-LLM combines dense compute, heterogeneous device roles, quantized layouts, and multiple FPGAs [29]; DLLM-OPU combines an instruction-driven substrate, sparse compute, and iterative reuse [20]. Such combinations motivate interpreting results through mechanisms and configurations rather than ranking architectural labels.

5.2. Compute-Dense Engines

DSP-centric arrays, vector engines, and spatial pipelines increase arithmetic parallelism when matrix dimensions and operand delivery match their shape [69,75,76,77]. Short reductions, edge tiles, and changing token counts idle lanes; replicated buffers and long interconnects can limit clock frequency or routability.

Spatial pipelines allocate resources across operators; reused engines execute them sequentially. HG-PIPE and SSR explore this balance, while Weight-Loop emphasizes reuse [70,77,78]. Specialization can require regeneration for new shapes. CD-LLM shows that dense arrays also support batched decode [29]; inference phase and reuse distinguish these workloads more precisely than vision/LLM labels.

5.3. Memory Organization and Streaming

Memory-oriented designs reduce transfers or enlarge the accessible working set through reuse, fusion, compressed storage, banking, and caching [6,26,27,79,80]. Capacity and bandwidth constrain different aspects of execution: fitting the model does not ensure timely operand delivery. Buffers, metadata, and replicated tensors also consume on-chip capacity.

Transaction granularity connects representation to useful bandwidth. Narrow accesses waste bursts, bank conflicts serialize requests, and scales or indices consume saved bandwidth. The mobile sparse Transformer design co-designs tiled scaling and AXI DMA with compressed computation [81]. ME-ViT emphasizes local reuse; deformable-attention engines manage irregular fetches; FAST-Prefill manages sparse-prefill liveness; Hummingbird+ coordinates shared weight/KV storage across inference phases [6,49,55,56,61]. Their common objective is sustained delivery, but their reuse opportunities differ.

Residency depends on schedule as well as capacity: concurrent stages may evict otherwise fitting tensors, while liveness-based buffer reuse can reduce storage without changing arithmetic.

CXL-SpecKV uses FPGA prefetch, compression, and adaptive cache representation to support GPU inference [82]; its benefits concern that service path, not complete FPGA inference.

5.4. Specialized and Shared Operator Support

Faster matrix engines expose Softmax, normalization, activation, reductions, and conversion costs. Dedicated units use lookup tables, piecewise or polynomial approximations, reduction trees, and integer arithmetic [83,84,85]. QUARK shares nonlinear patterns, TATAA reuses hardware across numerical modes, and reconfigurable approximators share computation across functions [39,67,86]. Sharing reduces duplication but introduces scheduling and mode-transition costs; dedicated units enable overlap at greater area cost.

Operator throughput also depends on buffering. LightningRMS overlaps accumulation and normalization through replay storage and stream control [87]. DQ-LUT couples dyadic scaling and approximation, but its synthesized FPGA scaling component and separate software quality evaluation provide a narrower result [88]. Operator gains affect the system in proportion to the accelerated time fraction in Equation 8; offload transfers can erase them. Publications mixing FPGA prototypes and ASIC evaluation require preserving those implementation boundaries [43,89,90].

5.5. Programmable Overlays and Generated Architectures

Overlays reuse a substrate through instructions, parameters, or schedules, as in Transformer-OPU, LTrans-OPU, FET-OPU, and ProTEA; ChatOPU and DLLM-OPU extend this approach to sparse or iterative execution [20,31,45,91,92,93]. Flexibility remains bounded by operators, formats, addressing, and storage. Generated architectures such as TRAC, FINN-T, and CODO specialize these structures, potentially removing overlay overhead at the cost of synthesis and implementation when the graph changes [30,32,68].

Adaptation occurs at several levels. InTAR switches execution modes using a statically prepared schedule [94]; RIFT changes scheduling and work selection within one bitstream [95]. Neither mechanism implies online schedule discovery or dynamic partial reconfiguration. The practical tradeoff is the performance of specialization against the frequency and cost of retargeting.

5.6. Heterogeneous Execution

Heterogeneity places operators on resources with different strengths. CHARM combines AIE matrix engines with PL buffers and nonlinear functions; EQ-ViT jointly maps quantized computation and communication; LegoMap explores layer-level tiling and interconnect choices [73,96,97]. Synchronization, interface traffic, and stage imbalance determine whether specialization survives integration.

Placement can change between phases. The PCIe-based coarse-grained design reports beneficial prefill offload but slower FPGA-offloaded decoding, motivating CPU decode [22]. DFVG uses an FPGA draft model and GPU verification, making gains depend on acceptance, branching, overlap, and cache management [98]. These examples show why a faster component need not yield a faster service: the partition must reduce the critical path after transfers and dependencies are included.

5.7. Multiple Devices and Distributed Inference

Multiple FPGAs expand capacity, aggregate bandwidth, or add computation. Tensor partitioning introduces collectives, pipelines introduce bubbles and imbalance, and expert placement depends on routing skew [3,7,17,29,99]. FMC-LLM and CD-LLM assign devices different compute and memory roles for batched decoding; DFX and LORA emphasize multi-card autoregressive execution [3,7,17,24,29]. LORA’s communication-aware normalization illustrates that distributing linear work also requires managing reductions and synchronization. Related conference and journal reports remain one research line when assessing independent corroboration.

Topology, dependencies, and partitioning determine whether links overlap computation or stall at collective barriers. More devices change both resources and synchronization.

For a fixed workload, throughput-scaling efficiency is $E_n=T_n/\left(n{T}_1\right)$ for n identical devices. This ratio requires the same workload to fit and execute on one device; otherwise capacity and absolute performance must be compared without a fictitious reference. Package dies and HBM channels should likewise not be counted as independent boards.

6. Corpus Composition and Cross-Dimensional Findings

For RQ3, all 266 retained publication records provide descriptive context for RQ1–RQ2. Year totals are 6, 18, 23, 60, 90, and 69 for 2021–2026; 2026 is incomplete (Figure 2). These selected publication counts are neither independent-study counts nor deployment prevalence.

A secondary mapping harmonizes exact aliases while preserving original labels, identifiers, and residual ambiguity. It does not constitute independent full-text recoding or retrospectively encode every architectural facet in Table 8.

6.1. Recorded Techniques and Numerical Formats

The multi-label technique counts are quantization 172/266 (64.7%), hardware-aware mapping 132 (49.6%), operator/graph optimization 92 (34.6%), sparsity/pruning 70 (26.3%), model/attention restructuring 64 (24.1%), distillation 11 (4.1%), and factorization 4 (1.5%). Each record counts once per category. Operator/graph coding includes approximation and substitution, not only fusion. Five records have residual technique labels and two have none; residual and recognized labels can coexist.

Figure 3 also shows format occurrences: INT/fixed-point 8-bit in 131 records (49.2%), 4-bit in 56 (21.1%), INT/fixed-point 16-bit in 55 (20.7%), FP16/BF16 in 46 (17.3%), sub-4-bit/binary/ternary in 22 (8.3%), and FP32 in 19 (7.1%). The 4-bit group includes distinct integer, fixed-point, FP4, and power-of-two representations. Other/unspecified formats occur in 93 (35.0%), including wider accumulators; 16 records (6.0%) have no format. Categories overlap; identified software-only and baseline formats are excluded.

Precision strategies are mixed/hybrid in 101 records (38.0%), uniform/single in 91 (34.2%), multiple/reconfigurable in 18 (6.8%), and unresolved in 56 (21.1%). Operand uniformity does not establish uniform accumulation or nonlinear arithmetic; multiple-format experiments do not establish runtime switching. A method name alone cannot resolve the precision assignment.

From 2024 to 2025, 4-bit occurrence increases from 12/60 (20.0%) to 23/90 (25.6%), while 8-bit occurrence changes from 34/60 (56.7%) to 44/90 (48.9%). Partial 2026 contains 18/69 4-bit records (26.1%) and 7/69 sub-4-bit/binary/ternary records (10.1%). Overlapping labels suggest diversification, not simple replacement; retrieval, workload, and coding changes prevent causal performance claims.

6.2. Architectural Emphasis and Workload Groups

Memory-oriented and compute-dense contributions total 147 records (55.3%; Table 9), separated by only three records; two reassignments would reverse their order. The 36 residual labels include streaming/pipelining, sparsity adaptation, and runtime reconfiguration, which do not uniquely identify a primary emphasis. These uncertainties preclude a stable field-wide ranking.

The largest primary workload groups are ViT (69), decoder-only or autoregressive LLMs (56), general or multiple Transformer workloads (36), and encoder or NLP workloads (25). Separate Swin, MoE, hybrid, and multimodal groups mean that 69 and 56 are not totals for all vision and language work. Generic language-model labels are not assumed decoder-only.

Decoder-only records include 24 memory-oriented and six distributed contributions; ViT includes 21 compute-dense and 17 memory-oriented records (Figure 4). These are workload-conditioned emphases, not categorical vision/LLM architectures or measured bottlenecks.

Between 2022 and 2025, encoder/NLP shares change from 7/18 (38.9%) to 5/90 (5.6%), and decoder-only shares from 1/18 (5.6%) to 22/90 (24.4%). Partial 2026 has 17/69 decoder-only records (24.6%). Retrieval and publication-mix changes remain confounded with these descriptive trends.

6.3. RQ3: Technique Co-Occurrence Within Architectural Groups

Table 10 uses each named architectural group’s size as its denominator. Multi-label techniques need not sum to that size; 36 residual architecture records remain in the supplementary analysis.

Quantization spans compute-dense (68.1%), memory-oriented (66.7%), heterogeneous (63.6%), and overlay (80.0%) groups, supporting its interpretation as a shared technique. Sparsity shares of 30.6% and 24.0% in compute-dense and memory-oriented groups describe occurrence, not usefulness.

Mapping occurs in 37.5% of compute-dense, 54.7% of memory-oriented, 63.6% of heterogeneous, 70.0% of overlay, and 71.4% of distributed records. Tiling, scheduling, and layout partly define both axes; similarly, 15/23 specialized-operator records have an operator/graph code. These associations neither establish causation nor show absence of mapping elsewhere.

With seven distributed records, one observation changes prevalence by 14.3 percentage points. Zero recorded sparsity does not establish unsuitability; attribution still requires matched cases and ablations.

6.4. Robustness to Collection History and Coding Uncertainty

Two documentation partitions test sensitivity: original retained versus added cohorts (143/123), and records with versus without a stable identifier in the PDF-review log (158/108). These are not validated quality levels; missing log entries do not prove absence of earlier inspection.

The broad prevalence of quantization is stable across these partitions: its share varies from 62.7% to 67.6% (Table 11). It remains the largest technique count in each stratum, although its advantage over hardware-aware mapping is only one record among logged reviews, 99 versus 98. Accordingly, the supported conclusion is that quantization is recurrent across collection histories; a uniquely dominant role relative to mapping is less secure. Mapping has a larger coded share among additions and logged reviews. That difference can reflect differences in selected workloads, contribution types, or extraction detail, and does not identify the causal effect of source review or a temporal increase in mapping research.

Architectural ranking is more sensitive. Memory-oriented records slightly outnumber compute-dense records in the original retained cohort, 51 versus 46, whereas the order reverses among additions, 24 versus 26; the logged-review stratum is tied at 35 each. Specialized-operator contributions are more prominent among additions, but all 36 residual architecture labels also occur there. The aggregate ranking therefore combines different label vocabularies and contribution mixes. No claim that the field is predominantly memory-oriented or compute-dense follows from these differences.

Among 230 resolved architecture labels, memory-oriented and compute-dense shares are 75/230 (32.6%) and 72/230 (31.3%). Assigning all 36 residuals to either category gives separate upper bounds of 111/266 (41.7%) and 108/266 (40.6%), which cannot hold simultaneously. These are sensitivity bounds, not proposed reclassifications.

All 56 unresolved precision strategies occur in the logged-review stratum, potentially reflecting uncertainty exposed by closer inspection rather than poorer reporting. Among 210 resolved records, mixed, uniform, and multiple/reconfigurable shares are 101/210 (48.1%), 91/210 (43.3%), and 18/210 (8.6%). Assigning 11 unresolved records to uniform would reverse its order relative to mixed. These conditional shares support coexistence, not a definitive corpus-wide predominance.

For the linked LORA [17,23,24], LlamaF [100,101], and MCoreOPU [102,103] groups, selecting one report per group yields 12 configurations of 262 bookkeeping units. Quantization ranges from 170–171 (64.9–65.3%), memory-oriented from 74–75 (28.2–28.6%), and compute-dense remains 72 (27.5%); distributed records fall from seven to five representatives. These links do not materially change broad findings, but unresolved families and distinct contributions prevent interpreting the units as independent studies. Supplementary scripts preserve every selection; the main denominator remains 266.

7. Comparative Evaluation

The 266 publications are not interchangeable experiments: each can contain multiple configurations, and related reports can share results. We compare capabilities and configuration-specific evidence, preserving source values and uncertainties rather than constructing a corpus-wide ranking.

7.1. The Configuration as the Unit of Comparison

We describe an experimental configuration by

$$\mathcal{C}=(m,d,b,s_{\mathrm{in}},s_{\mathrm{out}},q,r,h,n,\varphi ,\beta ),$$

(9)

where m identifies the model and checkpoint, d the dataset or request distribution, b the batch regime, $s_{\mathrm{in}}$ and $s_{\mathrm{out}}$ the input and output lengths, q the numerical formats, r the compression or routing policy, h the hardware, n the device count, $\varphi$ the execution phase, and $\beta$ the measurement boundary. Image dimensions and tokenization replace language-specific lengths for vision models. For mixture-of-experts (MoE) models, the active experts and routing distribution are additionally required; total parameter count does not specify the work performed per token. Diffusion language models require the denoising schedule and stopping policy because changing the number of iterations changes the workload.

Provenance is metric-specific: board throughput, tool-estimated power, and software accuracy can coexist without forming a measured end-to-end result. Unavailable extraction does not prove source omission. We distinguish source statements, calculations from stated inputs, and unresolved contradictions.

7.2. Capabilities Across Workloads and Deployment Scales

Table 12 distinguishes on-chip decode, distributed weights, long-context prefill, diffusion work reduction, and embedded-memory constraints. Retargeting must specify changes to weights, instructions, compiler output, bitstream, or manual design; multiple supported models alone do not establish programmability.

Table 13 and Table 14 retain selected numerical observations with their limitations. No entry supplies a missing batch size, power boundary, or quality score by assumption. In particular, a token-processing lane is not a request batch, a sparse-attention block size is not a sequence count, and alternative FPGA platforms are not necessarily a multi-device deployment.

7.3. Worked Comparisons and Failure Modes

Measured and estimated platforms: FlightLLM.

FlightLLM combines sparsity, compact weights, and on-chip decode retention [2]. Its approximately 55 tokens/s U280 result and 92.5 tokens/s estimated VHK158 result are not two board measurements or an isolated optimization gain. On LLaMA2-7B/WikiText-103, extracted perplexity changes from 8.7 to 10.2 under combined compression. Wider execution and floating-point nonlinear operators also separate stored width from arithmetic. Platform rate and quality must refer to the same configuration.

Aggregate throughput and service latency: the large-model clusters.

FMC-LLM and CD-LLM use different slave boards and documented batching [7,29]. CD-LLM’s batch-256 aggregate rate is not single-request generation speed; FMC-LLM’s 50 ms TPOT budget is not reciprocal aggregate throughput. Their rates cannot isolate compute-oriented architecture gains. A matched service comparison needs common lengths, precision, admission, request mix, and latency compliance.

Matched configurations and related reports: LORA.

LORA’s journal Table III compares floating-point and quantized GPT-2 on one U280: 678.4 versus 307.3 ms, approximately $2.21\times$ for the complete changed configuration, including frequency [17]. This is not a bit-width ablation. The earlier project and conference reports share the 213.0 ms result for larger GPT-2 with input 128 and output 1 [23,24]; they are citable reports, not independent replications. Journal extensions must not be assigned to precursors, and unreproducible aggregation rules in the project’s mean-speedup claims favor configuration-level comparisons.

Compression changes quality; reduced iteration count changes work.

The 2.5D accelerator’s Table 7 reports 63.98 versus 95.83 tokens/s for Qwen3-8B without/with tensor decomposition, a calculated $1.50\times$ ratio [18]. MMLU is 71.9% originally, 69.7% after W4A16, and 64.9% with decomposition: losses of 2.2 and 7.0 percentage points, respectively. The GPU row names only a “7B LLM,” preventing a matched-model GPU–FPGA superiority claim.

DLLM-OPU compares fixed 1024-step GPU execution with FPGA-supported early stopping, combining hardware effects and reduced work [20]. Dream scores change from 76.9% to 76.7% on GSM8K and 58.5% to 58.1% on HumanEval with caching/stopping; these qualify only the tested policy. FAST-Prefill reports relative TTFT and energy–delay, not decode throughput; its final and preprint baselines must remain distinct [49].

Attributing a baseline result to the proposed method.

The AWQ-on-ViT study’s cited 3-bit QAT HG-PIPE baseline reaches 6709 FPS, with 40.6 W in the surrounding presentation [105]. Neither value measures the proposed 4/5-bit AWQ settings, for which slowdown is reported without matched throughput. Combining those baseline values with AWQ accuracy would invent a configuration. Likewise, the structured-sparsity study’s 39.2 tokens/s is a dense GPU baseline, not FPGA throughput [106].

Technology and power boundaries.

FSSA’s prototype establishes FPGA relevance, but its frequency and TOPS/W figures concern ASIC evaluation [89]. SHARP compares Vivado-estimated FPGA power with CPU/GPU telemetry [21]; its efficiency ratios have mixed provenance. Hummingbird+ distinguishes a 7.33 W FPGA estimate from approximately 13 W measured system power [61]. Technology, power scope, and host work must remain attached to each metric.

Unresolved timing boundaries.

APTQ+ Table IX confirms both 0.48 ms BoolQ latency and 31.48 tokens/s, but not reciprocal single-token boundaries [19]. We retain this qualification without changing units. PIVOT and ARES discrepancies remain in the supplementary claim ledger rather than pooled comparisons.

7.4. Metric Definitions and Valid Normalization

For an autoregressive request delivering $N_{\mathrm{out}}>1$ output tokens, let $t_1$ and $t_N$ be the first- and last-token delivery times. Then

$$\mathrm{TTFT}=t_1-t_{\mathrm{arrival}},\mathrm{TPOT}={\displaystyle \frac{t_N-t_1}{N_{\mathrm{out}}-1}}.$$

(10)

TTFT includes only work within the declared boundary; prefill-kernel latency is not automatically TTFT. Inter-token distributions complement mean TPOT. Input tokens, delivered output tokens, requests, frames, and diffusion updates are different numerators.

Aggregate output throughput over an interval is

$$T_{\mathrm{out}}={\displaystyle \frac{N_{\mathrm{delivered}}\left([t_a,t_b]\right)}{t_b-t_a}}.$$

(11)

Under concurrency, $1/T_{\mathrm{out}}$ is not per-request TPOT; report batching, admission, lengths, and idle periods. Complete request throughput includes prefill, while reciprocal sustained FPS describes a steady-state initiation interval, not necessarily sample latency.

Energy per delivered output token requires matching measurement intervals and boundaries:

$$E_{\mathrm{token}}={\displaystyle \frac{{\int }_{t_a}^{t_b}{P}_{\beta }\left(t\right)dt}{N_{\mathrm{delivered}}\left([t_a,t_b]\right)}}.$$

(12)

The shortcut ${\overline{P}}_{\beta }/T_{\mathrm{out}}$ requires matching intervals. State whether power covers the chip, board, cluster, or system, whether idle power is subtracted, and whether hosts and networks are included. TDP is not measured inference power. For the same workload, $\mathrm{EDP}=E_{\beta }L$ has units J s; an EDP gain is not the same energy-only gain.

Dense $M\times K$ by $K\times N$ multiplication requires $MKN$ MACs or approximately $2MKN$ separately counted operations; GOPS/TOPS must state the convention. Sparse executed throughput counts performed work, whereas dense-equivalent throughput uses the unpruned count. Neither is application throughput. Percentage-point accuracy loss, relative loss, and perplexity also remain distinct.

7.5. What the Comparisons Establish About the Source of a Gain

Table 15 applies the five-mechanism framework. Its missing controls identify how to narrow attribution, not mandatory ablations for every system; conclusions remain configuration-specific.

A positive configuration result with a bounded interpretation.

LORA illustrates a useful configuration gain without isolating quantization’s contribution. SHARP’s DeiT-T setting similarly reaches 351.2 FPS on ZCU102 with Top-1 accuracy changing from 72.13% to 69.34% [17,21]. Selection depends on the application’s quality and throughput constraints.

A reversal that changes the placement decision.

For TinyStories-15M on Alveo U55 with a Ryzen 7 7700 host, the PCIe design’s HW-2 path raises prompt processing from 33.29 to 124.0 input tokens/s but lowers decode from 10.9 to 5.15 output tokens/s [22]. A tested hybrid retains CPU decode. The source’s combined prompt/output “overall” rate is not output-generation throughput; absent matched quality and a complete cost breakdown, neither accuracy-neutrality nor the precise cause of reversal is inferred.

Section 8.2 formalizes this procedure: identify a feasible quality/service region, the changed work or resources, and the added costs within the application boundary.

7.6. Supported Comparative Conclusions

Pareto dominance requires matched configurations and declared objectives: no worse on all, better on at least one. Across unmatched results, mechanisms and capabilities remain comparable, but headline ratios—even divided by device or DSP count—cannot remove model, memory, platform, or power-boundary differences.

A quality-qualified speedup includes its task score, baseline, and compression setting; it is not speedup divided by accuracy loss. Quantitative synthesis requires compatible subsets, report-family reconciliation, and metric-level provenance.

8. Discussion: Conditions for Useful FPGA Acceleration

8.1. From Reported Gains to Conditional Design Decisions

Useful design decisions identify the workload constraint, choose a suitable transformation, account for its added costs, and evaluate the complete configuration at a declared quality/service target. Capacity, request latency, and aggregate throughput remain separate objectives.

The five mechanisms in Figure 1 require different controls: hold executed work constant to isolate execution efficiency; track physical transfers to assess compression; measure useful arithmetic or occupancy to assess datapaths and scheduling; and compare feasible deployments to assess capacity. Several mechanisms can contribute simultaneously.

First establish that weights, activations, cache, metadata, and buffers fit, and that task quality and service constraints hold. Only feasible candidates enter the comparison; there is no universal conversion from accuracy or perplexity to latency.

Next use the configuration distinctions in Section 7: DLLM-OPU’s early stopping changes work, and the 2.5D accelerator’s decomposition changes representation and quality [18,20]. Matching these policies across platforms would isolate narrower execution effects.

Finally account for decoding/reconstruction in LUT-LLM and TFLOP [41,42], precision and scheduling changes in CODO [68], and concurrency in FMC-LLM/CD-LLM [7,29]. Table 16 summarizes the costs that can reverse a choice.

The following models identify discriminating measurements; they are simplified conditions, not fitted corpus-wide predictions.

8.2. Break-Even Conditions for Representation, Placement, and Scale

When does compression repay its representation cost?

Consider a fixed invocation of a region whose transfers and computation are serialized. Let $D_0$ be its original physical transfer volume and $B_0$ its achieved bandwidth. The compressed form transfers payload $D_p$, metadata $D_m$, and padding or alignment traffic $D_a$ at achieved bandwidth $B_c$. Let $H_c$ be added conversion or decoding time, and $\mathsf{\Delta }C$ the change in the remaining computation time, negative when the compressed computation is faster. Compression reduces this region’s latency precisely when, under this additive model,

$$H_c+\mathsf{\Delta }C<{\displaystyle \frac{D_0}{B_0}}-{\displaystyle \frac{D_p+D_m+D_a}{B_c}}.$$

(13)

Metadata, access inefficiency, and conversion can erase payload savings. A negative $\mathsf{\Delta }C$ includes arithmetic benefits that should not be attributed solely to traffic. Transfer volumes concern the actual access pattern, including repeated loads.

For overlapped execution, compare complete schedules or exposed critical-path costs rather than adding concurrent durations. Hidden decoding, pipeline imbalance, or a tile newly fitting on chip can change costs discontinuously; nominal bit width alone cannot predict these transitions.

The implemented APTQ+ path, reconstruction in LUT-LLM and TFLOP, and EdgeLLM’s dense and sparse settings with reported model quality illustrate these requirements [19,40,41,42]. Sweep group size, context, and batch across residency transitions; a latency break-even condition alone establishes no energy saving.

When should a subgraph move to the FPGA?

For a serialized offload boundary, let $T_h$ be host execution time for the candidate subgraph and $T_f$ its FPGA execution time. Let $D_{hf}$ and $D_{fh}$ denote incremental host-to-FPGA and return traffic, with achieved bandwidths $B_{hf}$ and $B_{fh}$. Let $H_o$ include launch, synchronization, format conversion, and residual host computation within the candidate boundary. Offloading reduces latency when

$${\displaystyle \frac{D_{hf}}{B_{hf}}}+{\displaystyle \frac{D_{fh}}{B_{fh}}}+H_o<T_h-T_f.$$

(14)

The right-hand side is the compute saving available to pay transfer and control costs. Count weight reloads when they occur; resident-weight reuse and model transitions require their actual schedules.

The PCIe design’s prefill/decode reversal demonstrates phase-dependent placement [22]; it does not identify the loss’s exact decomposition. Fusion or retained state creates a new configuration requiring reevaluation.

Asynchronous execution requires schedule-level costs. DFVG couples draft generation, GPU verification, acceptance, and cache handling [98]; CXL-SpecKV modifies a GPU memory path [82]. Both require evaluation at the full service boundary.

When does adding devices improve the target service?

For a fixed workload that fits on one device, let $T_1$ be the complete reference latency. Let $T_n^{\mathrm{comp}}$ be latency within the same boundary under a chosen n-device partition when new inter-device and control costs are removed, retaining local-memory service, unchanged work, and compute imbalance. In a critical-path cost model, let $C_n^{\exp }$ denote communication time that remains exposed after overlap, and $H_n$ the remaining scheduling and synchronization overhead. Latency improves when

$$C_n^{\exp }+H_n<T_1-T_n^{\mathrm{comp}}.$$

(15)

The model does not assume $T_n^{\mathrm{comp}}=T_1/n$: partitionability, heterogeneity, imbalance, and memory service can prevent ideal scaling. Do not double-count hidden communication or its stalls; when critical paths change, evaluate complete schedules.

LORA addresses communication and synchronization [17,24]; FMC-LLM/CD-LLM additionally require a specified batch/service target [7,29]. Increased batching need not improve fixed-latency service. If the workload cannot fit on one device, Equation 15 has no empirical single-device baseline: compare feasible alternatives at equal capacity and service requirements.

8.3. Evidence Boundaries and Transferability

The provenance and scope rules in Section 2 separate operator, integrated, and distributed results. Local numerical error is not model quality, and extracted power cannot be paired with throughput from another configuration.

Section 6.4 shows recurrent quantization but cohort-sensitive architecture ordering and concentrated residual labels. These collection effects favor mechanism-and-cost conclusions over architectural rankings; limited family checks do not resolve all overlapping reports.

Application conclusions require both a plausible mechanism and matched quality/service measurements. The following audit identifies where the extraction supports those assessments.

8.4. RQ5: Coverage of the Extracted Evidence

Coverage describes the 266-record extraction, including 158 logged source reviews, inherited values, and reconciliations. It is not a new full-text reporting-quality audit or independent double coding.

Figure 5 harmonizes original and later status labels. Available denotes a usable extracted value/category, not a complete configuration; partial denotes relative, derived, or qualified evidence; unresolved denotes absent/unclear extraction, not a verified source omission. Operator scope or simulation/estimation does not itself make a value unavailable. Context length and batch lack uniformly reconciled coverage rates.

Implementation coding yields 91 RTL, 86 HLS/high-level, six hybrid, one other explicit, and 82 unresolved records. These are not controlled performance comparisons of development methods.

Scope is full-model in 114 records, operator/kernel in 48, mixed in 21, layer/block in 13, subsystem in three, and unresolved in 67. One unresolved exact boundary explicitly reports BERT inference and is partial in the availability figure. Full-model labels do not establish inclusion of host work, transfers, tokenization, or the complete autoregressive loop.

The provenance field mixes evidence types and coding bases. Its 94 Inferred labels (35.3%) and remaining measured/estimated/simulated/mixed descriptions cannot yield a valid hardware-measurement percentage without metric-level auditing. A report can combine board timing, tool power, software accuracy, and ASIC comparisons; the supplementary mapping retains these descriptions.

RQ5 therefore identifies substantial but incompletely reconciled evidence. Availability alone does not make throughput, power, and quality configuration-compatible; missing extraction cannot be reported as a primary paper’s demonstrated omission.

8.5. A Minimum Context for Reusable Evaluation Evidence

Table 17 proposes configuration-level reporting requirements, not fields uniformly audited in the corpus. Attach them to each evaluated model/device/precision setting.

Pair power and throughput only at matching configurations and intervals; retain host and device boundaries and configured baselines. Otherwise workload or measurement changes can masquerade as architectural gains.

8.6. Research Questions with Observable Validation Outcomes

The following experiments test the identified trade-offs; they do not imply that prior work has never addressed the corresponding optimization.

When does adaptation outperform a fixed mapping? Compare fixed, runtime-adaptive, and per-shape specialized mappings on the same FPGA, formats, and checkpoints [30,71,72]. Vary lengths, batch, and model transitions under a declared distribution. Measure complete-request and tail latency, energy, operator coverage, and tuning effort. Report where adaptation repays dispatch, reconfiguration, and buffering costs.

When does compression improve the complete memory path? Sweep weight, activation, and KV-cache representations separately and jointly while holding checkpoints, platform, and quality evaluation fixed [2,26,27]. Measure payload, metadata, padding, conversion, transfers, and residency transitions across contexts and batches. The outcome is a quality-qualified latency–energy frontier, exposing when compression saves transfers or merely shifts costs.

When does an operator improvement survive integration? Compare isolated operators with integrated inference using the same graph and precision. Measure the original time fraction f, transfers, scheduling, host fallbacks, and model quality. The ideal no-overhead bound ${[1-f+f/s]}^{-1}$ for operator speedup s provides a reference for identifying integration losses; report complete-model latency and energy at a declared quality tolerance.

When does distribution provide a service-level benefit? Vary device count and partitioning for fixed models, formats, and request distributions [3,7,17,29]. Include host work, exposed links, imbalance, power, TTFT, and TPOT. If single-device execution is infeasible, compare feasible systems. Report requests or tokens per joule at the same service target across interconnects and memory configurations.

9. Conclusions

This survey organizes 266 FPGA Transformer-inference publications by the mechanism behind each claimed benefit and the evidence needed to establish it. Quantization recurs across all corpus strata, whereas small differences among architectural categories depend on cohort and coding choices; the resulting map therefore supports interpretation of the literature, not architecture ranking or independent-study counts.

Three findings emerge. Reduced representations yield savings only when these survive metadata, conversion, and the actual data path, so nominal bit width is insufficient. Local arithmetic or scheduling gains become application gains only when they lie on the critical path and integration costs are included. Distributed capacity, aggregate throughput, and per-request latency are distinct benefits requiring different references, even for the same model. The worked comparisons show how these distinctions change conclusions: quantized LORA supports a bounded same-workload comparison, tensor compression and diffusion early stopping couple gains to model or execution changes, and FPGA offload can accelerate one inference phase while slowing another. The derived break-even conditions, with explicit assumptions, complement the taxonomy by explaining when plausible optimizations fail.

Future work requires controlled evidence linking representation, execution, and deployment under explicit quality and service targets, particularly on adaptation crossover, complete compressed memory paths, operator integration, and communication-inclusive scaling. The accompanying scripts, mappings, and source checks make this synthesis inspectable, within the limits of historical search reconstruction and uneven source verification.