Binary and Ternary Neural Networks: A Survey of Learning and Acceleration

1. Introduction

Binary and ternary neural networks ask how much numerical detail a learned model actually needs. Replacing a wide weight with a sign, or with a sign and a zero state, can reduce storage and turn multiplication into selection, accumulation, or bitwise operations. The attraction is practical: more of a model can remain close to computation, and devices with modest memory or power budgets can execute workloads that would otherwise require larger processors. The same idea has supported compact visual recognition networks and now motivates language models with billions of discrete parameters [1,2,3,4,5]. Its significance is not limited to compressing an existing network. Discrete representations also change the optimization problem, the architecture that best uses a parameter budget, and the hardware that can exploit the resulting computation.

Progress has come from coordinating those choices rather than applying an increasingly aggressive rounding rule. Early binary networks retained real-valued training state, developed surrogate gradients, and recovered capacity through scaling and residual connections [2,6,7]. Transformer models added attention and distillation to this design space, while native low-bit LLMs moved quantization into pretraining [8,9,10]. More recent conversion methods adapt pretrained models through reconstruction, learned transformations, selective residuals, and calibration that preserves the target model’s reasoning behavior [11,12,13,14,15]. These developments make a chronology of bit widths insufficient: two ternary models can differ in their learning resources, task capabilities, sparsity constraints, and executable operators.

Acceleration has diversified just as substantially. Register-resident lookup on CPUs and shared-memory lookup on GPUs reuse partial activation sums, while other GPU kernels decode compact weights into the wider arithmetic supported by matrix units [16,17,18,19]. FPGAs and ASICs can specialize arithmetic and dataflow; compute-in-memory and fixed-weight ROM architectures instead attack movement between storage and compute [20,21,22,23]. None of these choices follows automatically from the number of weight states. A smaller payload may add unpacking work, a zero may require routing before it saves computation, and an optimized linear layer may expose attention or the output head as the next bottleneck. A useful survey must therefore connect the learning problem to these implementation choices without reducing the field to a hardware benchmark catalogue.

Existing surveys already provide strong foundations for binary-network optimization [24,25], ternary quantization [26], low-bit LLM algorithms and systems [27,28], and LLM hardware acceleration [29]. Our purpose is not to claim that these subjects have never been connected. Instead, we explain recurring design choices across successive model families: how capacity is recovered after discretization, which costs are paid during construction rather than inference, and how the resulting structure interacts with the workload and machine. This perspective links ideas that are often discussed separately, such as real-valued shortcuts in binary vision networks, residual error correction in low-bit LLMs, and heterogeneous execution around specialized ternary cores. Figure 1 summarizes the three complementary views used in this synthesis.

The survey develops three contributions.

• A taxonomy of representation, learning, and architecture choices that explains both the historical development of discrete neural networks and their transition to foundation models.
• A mechanism-based comparison of software and hardware acceleration, supported by within-study quantitative examples that preserve training, workload, and measurement conditions.
• A synthesis of current limitations and research opportunities, including reasoning preservation, activation and state precision, model mutability, and complete-system efficiency.

Literature discovery combined keyword searches and citation tracing with checks of primary papers, proceedings records, and author-maintained software artifacts. Searches paired terms such as binary, ternary, binarization, and low-bit with learning, vision, Transformer, LLM, kernel, accelerator, and memory-computing terminology. Sources were last checked on 4 September 2026; the bibliography identifies the examined versions where relevant. Binary and ternary computation supplies the topical focus, while multi-bit quantization, vector coding, sparsity, and efficient attention are included when they explain important alternatives or interactions. This is a critical narrative survey, not an exhaustive systematic review or an independent reproduction of every reported experiment. Public preprints and software reports are distinguished from archival publications, and source-reported measurements are distinguished from calculations and simulations.

2. Representations and Learning Principles

2.1. Discrete Values, Expressive Models

The number of discrete codes is only one part of a quantized representation. A binary parameter commonly takes values in $\{-1,+1\}$, but a scale changes the decoded values to $\{-\alpha ,+\alpha \}$, and a learned center can shift both levels. Similarly, ternary codes in $\{-1,0,+1\}$ may decode to a symmetric scaled alphabet, an asymmetric pair of nonzero levels, or an affine three-level grid [3,30,31,32]. The distinction matters computationally: a zero code represents an actual zero product only if the decoded middle value is zero. For a group with $\widehat{\mathbf{w}}=\alpha \mathbf{t}+\beta \mathbf{1}$,

$${\widehat{\mathbf{w}}}^{\top }\mathbf{x}=\alpha {\mathbf{t}}^{\top }\mathbf{x}+\beta \sum _j{x}_j,\mathbf{t}\in {\{-1,0,+1\}}^n.$$

(1)

The offset adds a group correction, but need not require reconstructing every weight or performing a separate dense matrix multiplication. Whether that correction is inexpensive depends on how it is shared and fused, as illustrated by modern ternary kernels [15].

Operand precision must be specified independently. We use W1A1 for binary weights and activations in the designated operators, W1A8 for binary weights with eight-bit activations, and W$\mathrm{T}$ for a ternary weight alphabet. These labels do not describe accumulators, normalization, embeddings, residual connections, gradients, or optimizer state unless those components are stated explicitly. For sign vectors, binary arithmetic provides the familiar identity

$${\mathbf{x}}^{\top }\mathbf{w}=2popcount\left(XNOR({\mathbf{x}}_{\mathrm{bit}},{\mathbf{w}}_{\mathrm{bit}})\right)-n.$$

(2)

This identity does not apply directly to W1A16: the activation magnitudes must still be incorporated, through sign-controlled addition, bit-plane decomposition, lookup, or wider arithmetic. For a symmetric ternary weight, multiplication can select $+x$, 0, or $-x$, while reductions and scale application generally remain wider [2,16,33].

Some compact models intentionally enlarge the representation around their binary core. Multiple binary bases can approximate one weight tensor; sparse exceptions or residuals can repair selected errors; and short vectors can be represented through a codebook [34,35,36,37]. These methods should not be excluded solely because they exceed a nominal bit threshold, nor conflated with a uniform one-bit matrix. For example, two binary components can generate up to four decoded levels, while a 16-bit index selecting an eight-dimensional codeword gives two code bits per component without restricting each scalar to four possible values. Such distinctions reveal how a method purchases expressiveness and what its implementation must subsequently compute.

Table 1. Representational choices and their consequences. Code widths describe the indicated payload, not complete-model memory. Wider paths and metadata are additional unless included explicitly.

Representation	What is represented	Consequence for learning and execution
Scaled binary	One sign plus a shared magnitude	Very compact core; scales and real-valued paths can recover capacity. XNOR requires compatible binary activations.
Symmetric ternary	Sign and optional zero	Adds a true zero state; dense two-bit, grouped, and sparse layouts have different decode and routing costs.
Affine/asymmetric ternary	Three levels with learned scale, shift, or separate magnitudes	Better distribution fit; offset corrections and scale placement must be supported by the kernel.
Multiple binary bases	A weighted sum of binary tensors	More expressive than one binary tensor, but requires multiple component products and combination.
Mixed or residual precision	Low-bit base plus selected corrections	Protects sensitive parameters; indices, masks, residual arithmetic, and wider storage remain.
Vector codebooks	An index for several weights	Low average code rate without a small scalar alphabet; codebook locality and decoding determine performance.

Table 1. Representational choices and their consequences. Code widths describe the indicated payload, not complete-model memory. Wider paths and metadata are additional unless included explicitly.

Representation	What is represented	Consequence for learning and execution
Scaled binary	One sign plus a shared magnitude	Very compact core; scales and real-valued paths can recover capacity. XNOR requires compatible binary activations.
Symmetric ternary	Sign and optional zero	Adds a true zero state; dense two-bit, grouped, and sparse layouts have different decode and routing costs.
Affine/asymmetric ternary	Three levels with learned scale, shift, or separate magnitudes	Better distribution fit; offset corrections and scale placement must be supported by the kernel.
Multiple binary bases	A weighted sum of binary tensors	More expressive than one binary tensor, but requires multiple component products and combination.
Mixed or residual precision	Low-bit base plus selected corrections	Protects sensitive parameters; indices, masks, residual arithmetic, and wider storage remain.
Vector codebooks	An index for several weights	Low average code rate without a small scalar alphabet; codebook locality and decoding determine performance.

2.2. Learning with Discrete Parameters

Most successful binary and ternary training methods separate the values used in the forward model from the state used to update them. BinaryConnect established the influential use of binary propagation with real-valued parameter updates, and Binarized Neural Networks extended the forward representation to activations [1,2]. If Q is a discrete quantizer and ${\mathbf{w}}_{\mathrm{latent}}$ is its underlying trainable state, a typical update has the form

$${\mathbf{w}}_{\mathrm{latent}}^{(k+1)}={\mathbf{w}}_{\mathrm{latent}}^{\left(k\right)}-\eta {\widehat{\nabla }}_{{\mathbf{w}}_{\mathrm{latent}}}\mathcal{L}\left(Q\left({\mathbf{w}}_{\mathrm{latent}}^{\left(k\right)}\right)\right).$$

(3)

The estimated gradient often uses a straight-through estimator (STE), replacing the quantizer’s unhelpful derivative with a chosen surrogate. The optimization variable can accumulate small updates until a discrete state changes, but the surrogate can also create oscillation or mismatch between the optimized and executed models. This motivates methods that shape weight distributions, improve surrogate derivatives, or alter the state-update rule rather than merely reducing forward rounding error.

Alternative formulations show that latent-weight SGD is not the only way to learn discrete networks. Expectation Backpropagation derives approximate Bayesian updates for continuous or discrete weights [38]. Bop interprets the magnitude of conventional latent weights as inertia and directly controls sign flips using a moving average of gradients [39]. Removing a latent-weight copy does not remove all wider state: Bop still maintains a real-valued moving average. More recently, Binary Error Propagation uses discrete target/error signals and masked integer updates in MLP and recurrent experiments [40]. These results broaden the learning vocabulary, but do not establish that contemporary LLM pretraining can be performed entirely with binary arithmetic. DoReFa-Net similarly demonstrates the value of distinguishing weight, activation, and gradient precision, rather than assigning one bit width to the whole training process [41].

Initialization, objective, and supervision provide a more reliable taxonomy than a rigid QAT/PTQ dichotomy. Native low-bit pretraining starts from an untrained model with quantization in its forward graph; checkpoint-based quantization-aware training (QAT) adapts an existing model under discrete constraints. Post-training quantization (PTQ) constructs a compressed model from a pretrained checkpoint, often using calibration activations and reconstruction objectives. Some PTQ methods optimize continuous scales or transformations for many iterations, so PTQ is not synonymous with “no gradients” [12,13]. Distillation describes a supervision mechanism that can accompany either native learning or checkpoint conversion. The relevant resource comparison includes the starting checkpoint, corpus, optimization passes, teacher computation, and retained trainable state, not just the number of unique calibration tokens.

2.3. Storage and Runtime Cost

Logical precision must be distinguished from both physical storage and working memory. For a stated reference set containing $N_{\mathrm{ref}}$ original scalar weights, we use

$$b_{\mathrm{eff}}={\displaystyle \frac{B_{\mathrm{codes}}+B_{\mathrm{metadata}}+B_{\mathrm{corrections}}+B_{\mathrm{wide}}}{N_{\mathrm{ref}}}}.$$

(4)

The terms partition storage without double counting: value-encoding indices are codes; auxiliary locations, scales, zero points, codebooks, and padding are metadata; selected residual values are corrections; and tensors retained above the target precision belong to the wider term. Exceptions within the comparison region remain in the denominator. Whole-model rates require inclusion of embeddings and the output head, while peak inference memory additionally contains activations, persistent state, workspace, and runtime allocations. Physical memory technologies also have device and peripheral costs that cannot be inferred from a logical code rate.

The frequently used “1.58-bit” description is the log-cardinality of a ternary alphabet, not an executable datatype. For an unrestricted group of G ternary values encoded by one fixed-width index,

$$b_G={\displaystyle \frac{\lceil {log}_2\left(3^G\right)\rceil }{G}}.$$

(5)

Scalar coding uses two bits, three trits use five bits, and five trits use one byte. The resulting payload rates are 2, $5/3$, and $1.6$ bits per weight, before padding and other storage. For nonuniform state probabilities, marginal entropy can be below ${log}_{2}3$; structured constraints can also reduce the number of legal joint patterns. Sherry, for example, restricts each four-weight block to exactly three nonzeros, leaving $\left({\displaystyle \genfrac{}{}{0pt}{}{4}{3}}\right)2^3=32$ patterns that fit in five bits [42]. Its 1.25-bit payload is achieved by restricting the model, not by losslessly encoding all $3^4=81$ unrestricted ternary patterns into five bits.

Runtime cost depends on where those codes are consumed. As an explanatory lower bound for one operation, ignoring overhead and assuming ideal overlap,

$$t_{\mathrm{op}}\ge max\left({\displaystyle \frac{D_{\mathrm{moved}}}{{\mathrm{BW}}_{\mathrm{effective}}}},{\displaystyle \frac{C_{\mathrm{executed}}}{{\mathrm{Rate}}_{\mathrm{effective}}}}\right).$$

(6)

Packing reduces the first term only for traffic that actually uses the packed representation. Decoding, lookup construction, sparse routing, and precision conversion change the second term and can introduce dependencies that prevent ideal overlap. Energy likewise includes data movement and the rest of the active system, not just a count of arithmetic operations [43]. These relationships explain why an inference format can be valuable even when it ultimately feeds a wider multiplier, and why a nominally multiplication-free design may not be the fastest option for every workload.

3. Learning Across Architectures and Tasks

3.1. Recovering Capacity and Optimization Stability

The early success of binary networks depended on preserving enough information around the discrete operation to make learning useful. BinaryConnect explored deterministic and stochastic weight binarization during propagation, with wider update state; its stochastic experiments also illustrate why the test-time representation must be checked separately [1]. Binarized Neural Networks made both main operands binary, while XNOR-Net retained real-valued scaling to approximate the magnitude lost by sign projection [2,3]. Ternary Weight Networks introduced a thresholded zero region, and Trained Ternary Quantization learned distinct positive and negative magnitudes [30,31]. These are different ways of assigning a small representational budget: a sign captures direction, a scale restores magnitude, and the zero state allows small weights to be suppressed instead of forced to either sign.

Later architecture designs recovered capacity by changing how discrete operations were connected. ABC-Net approximated weights and activations with multiple binary bases, making accuracy adjustable at the cost of multiple binary products [34]. Bi-Real Net instead connected real-valued pre-sign activations through identity shortcuts, preserving information without turning every convolution into a multibase expansion [6]. ReActNet combined generalized activation shifts with a redesigned network and distributional distillation [7]. These designs demonstrate that an effective binary network need not be an unchanged floating-point architecture with sign functions inserted. They also show why accuracy improvements cannot automatically be attributed to a quantizer alone: width, shortcut structure, teacher supervision, and boundary layers can change simultaneously.

Optimization research addresses the complementary problem of moving between useful discrete states. IR-Net combines weight balancing and standardization with a progressively adjusted gradient estimator, seeking to preserve information in both forward and backward passes [44]. ReCU clamps distribution tails to revive weights that otherwise rarely change sign, while AdaBin adapts the locations of binary values rather than fixing a symmetric alphabet [32,45]. For ternary networks, joint optimization of weights and quantizer thresholds offers an alternative to prescribing the zero region in advance [46]. The common objective is not simply to minimize instantaneous weight error, but to maintain an optimization trajectory with useful state transitions. Distributional entropy, surrogate smoothness, and reconstruction error are informative diagnostics; none alone guarantees better task performance.

The resulting accuracy record is substantial but sensitive to training provenance. ReActNet-A reports 69.4% ImageNet top-1 accuracy, while AdaBin’s ResNet-18 results change from 63.1% with scratch training to 66.4% with its two-stage procedure [7,32]. These figures should not be plotted as a controlled chronological race: the architectures and recipes differ. BiBench addresses this problem by benchmarking binarization under a common framework across vision, point clouds, language understanding, and keyword spotting [47]. Its deployment experiments further show that a training-side feature such as dynamic activation rescaling may lack support in the tested binary libraries. That observation connects algorithm evaluation to implementation feasibility without requiring every learning paper to supply a custom accelerator.

Table 2. Representative strategies for learning useful discrete networks. The third column identifies a cost or qualification that a precision-only taxonomy would hide.

Strategy and examples	Source of benefit	Cost or qualification
Scaling and thresholds [3,30,31]	Recover magnitude and select which values become zero.	Wider scales; threshold choice alters sparsity and capacity.
Multiple bases [34]	Sum binary components to approximate richer weights and activations.	More component products, coefficients, and reductions.
Real-valued paths [6,7]	Preserve information between discrete operators.	Wider residual state and elementwise computation.
Gradient/distribution shaping [44,45]	Improve movement between discrete states and reduce estimator mismatch.	Training schedule and distributional assumptions matter.
Adaptive value sets [32,46]	Fit the quantizer to the learned distribution.	Nonstandard levels and metadata complicate execution.
Direct/probabilistic updates [38,39,40]	Reconsider latent-state semantics or the update rule itself.	Wider auxiliary state or restricted demonstrated workloads.

Table 2. Representative strategies for learning useful discrete networks. The third column identifies a cost or qualification that a precision-only taxonomy would hide.

Strategy and examples	Source of benefit	Cost or qualification
Scaling and thresholds [3,30,31]	Recover magnitude and select which values become zero.	Wider scales; threshold choice alters sparsity and capacity.
Multiple bases [34]	Sum binary components to approximate richer weights and activations.	More component products, coefficients, and reductions.
Real-valued paths [6,7]	Preserve information between discrete operators.	Wider residual state and elementwise computation.
Gradient/distribution shaping [44,45]	Improve movement between discrete states and reduce estimator mismatch.	Training schedule and distributional assumptions matter.
Adaptive value sets [32,46]	Fit the quantizer to the learned distribution.	Nonstandard levels and metadata complicate execution.
Direct/probabilistic updates [38,39,40]	Reconsider latent-state semantics or the update rule itself.	Wider auxiliary state or restricted demonstrated workloads.

3.2. Attention, Structured Prediction, and Generative Tasks

Transformers introduced a distinction that becomes central in LLMs: projections multiply activations by learned weights, whereas attention also multiplies activations by other activations. TernaryBERT compresses embeddings and linear weights while using eight-bit activations and distillation from intermediate features, attention, and outputs [8]. BinaryBERT uses ternary weight splitting to initialize a wider binary model and retains multibit activations [48]. BiBERT goes further by redesigning attention for binary computation and using direction-matching distillation [9]. BiT combines precision-specific mappings with multistage distillation [49]. These are not interchangeable “binary BERTs”: their treatment of activations and attention determines both the approximation problem and the available execution primitives.

Vision Transformers expose similar pressure to redesign information flow. BinaryViT introduces pooling, residual scaling, and a pyramid structure rather than relying on a direct binarization of the original ViT [50]. BHViT combines convolutional and attention stages and uses a multibit decomposition within its binary implementation of attention [51]. Its use of multiple binary planes is important: the network’s binary core does not make every attention value a single bit. VAQF approaches the problem through automated joint exploration of quantization and hardware configurations [52]. Together these works indicate that attention topology, feature mixing, and hardware mapping can be as consequential as the choice between two and three weight states.

Structured prediction and generation require task-specific objectives beyond classification accuracy. BiDet uses an information bottleneck and sparse object priors to address false positives in binary detection [53]. BiDM adapts binary diffusion models to timestep-dependent activation statistics through temporal connections and distillation [54]. Its reported binary-convolution speedup is encouraging, but its LSUN-Bedrooms generation experiment still shows a substantial FID gap to the floating-point reference; local arithmetic efficiency does not imply preserved generative quality. Expanding-and-Shrinking BNNs provides a further architectural route, redistributing channels and using normalization and residual paths to diversify features [55]. The method illustrates a recurring trade-off: maintaining roughly similar convolution counts can still change activation storage and traffic.

Discrete networks also remain relevant to learning settings that are not static benchmark inference. Binary Error Propagation explores integer/discrete updates for MLPs and recurrent networks, while BiMU uses metaplastic binary Bayesian networks for continual adaptation and active label selection [40,56]. BiMU’s label and update savings concern continual-learning experiments, including frozen-feature settings, rather than binary LLM training. These directions should not be hidden by a narrative in which every earlier BNN idea exists only as a precursor to language models. Instead, they demonstrate that discrete representations can change memory, uncertainty, plasticity, and update cost as well as inference arithmetic. The challenge for the next generation of models is to preserve those benefits when tasks require richer outputs, longer dependencies, and larger training budgets.

4. Binary and Ternary Foundation Models

4.1. Native Pretraining and the Precision Budget

Native low-bit language modeling learns under a discrete constraint from the start, allowing the model to adapt its representations rather than approximate a fixed pretrained solution. BitNet introduced binary-weight BitLinear projections with eight-bit activations and real-valued training state; BitNet b1.58 extended the weight alphabet to include zero [4,10]. The zero state changes both the attainable approximation and the possible execution paths, but does not by itself specify a sparse layout or a physical 1.58-bit storage unit. These studies established useful scaling behavior for discrete-weight Transformers, while retaining wider embeddings, normalization, and attention operations. Their importance is the viability of learning at low forward precision, not a demonstration that the entire training process or inference graph consists of one-bit operations.

Spectra examines ternary language-model pretraining over a range of scales and compares it with wider alternatives [57]. Such studies ask a different question from same-parameter comparisons: under a fixed storage budget, a larger discrete model may be preferable to a smaller wide model. However, storage-matched models need not have matched training FLOPs, activation memory, or inference latency. FBI-LLM adds autoregressive distillation to randomly initialized binary-weight students, demonstrating another way to recover capability without inheriting pretrained student weights [58]. Its “fully binarized” label concerns the targeted weights, not binary activations or a teacher-free construction budget. The supervision cost is part of the method even when the teacher disappears from deployment.

BitNet b1.58 2B4T makes the native route concrete at the released-model level [5]. The model combines ternary linear weights and eight-bit activations with four trillion pretraining tokens, supervised fine-tuning, and preference optimization. Its evaluation includes knowledge, mathematics, code, and instruction following, extending the evidence beyond perplexity alone. Comparisons with other released small models are useful deployment comparisons, but different data, token budgets, and architectures prevent interpreting them as controlled estimates of the effect of ternary weights. Similarly, the report’s non-embedding memory figure and arithmetic-energy model should not be read as complete-runtime memory or measured system joules per token. Released model quality, physical storage, and runtime evidence answer related but distinct questions.

Controlled precision studies caution against declaring a universally optimal bit width. ParetoQ compares binary through four-bit configurations and finds a change in learning behavior between two and three bits, with favorable ternary and two-bit size–quality trade-offs under its training recipes [59]. QuEST combines distribution fitting, Hadamard normalization, and a trust-based gradient estimator to stabilize training down to binary weights and activations [60]. Its joint weight–activation analysis favors a different operating region from weight-only storage comparisons, and its largest-scale conclusions include fitted scaling-law extrapolations. The apparent tension is informative: the answer depends on whether the constrained resource is parameter storage, forward arithmetic, activation memory, or total learning compute. A strong comparison states that objective before interpreting the resulting frontier.

The model architecture itself can be part of the efficiency intervention. Scalable MatMul-free Language Modeling replaces attention with a recurrent token mixer and uses ternary projections in its recurrent and gated channel-mixing blocks [61]. This changes persistent-state growth as well as arithmetic, while retaining elementwise products and wider state. It should therefore be understood as a different language-model architecture, not merely a Transformer whose multiplications have been deleted. Its later neuromorphic implementation and TerEffic’s FPGA mapping demonstrate why model and hardware choices must be discussed together, while keeping their executed graph boundaries explicit [62].

4.2. Adapting Pretrained Capabilities

Checkpoint conversion trades the freedom of native pretraining for access to a model that has already acquired useful capabilities. A common local objective is to reconstruct the response of a pretrained layer on calibration activations,

$$\underset{\varphi }{min}\sum _{\mathbf{X}\in {\mathcal{D}}_{\mathrm{cal}}}{∥\mathbf{W}\mathbf{X}-{\widehat{\mathbf{W}}}_{\varphi }\mathbf{X}∥}_F^2,$$

(7)

where $\varphi$ can include discrete assignments, scales, transformations, and residuals. GPTQ uses approximate second-order information to compensate quantization error, while AWQ protects salient channels through activation-aware scaling [63,64]. They provide important wider-bit reference points, but successful four-bit conversion does not establish that the same recipe will preserve quality with two or three weight states. At extreme precision, the quantizer’s structure and the chosen reconstruction distribution become much more restrictive.

One family of methods spends a small additional budget on the errors that a binary base cannot represent. PB-LLM retains salient weights at higher precision, whereas BiLLM assigns additional binary residual capacity selectively [35,36]. DB-LLM uses two binary components, and ARB-LLM refines binary representations through alternating optimization [65,66]. These designs preserve more information than direct rounding, but their nominal binary component is not their complete storage cost. Selection metadata, scales, and correction paths must be included, and irregular grouping can make an otherwise compact format difficult to accelerate. The appropriate question is whether the quality recovered by those additions justifies their memory and execution costs, not whether a method can still be advertised as “one bit.”

Distillation and QAT offer a complementary route by adapting the model to its compressed representation. OneBit combines a sign matrix with two scaling vectors and a knowledge-transfer procedure; TernaryLLM learns ternary scale/shift parameters and uses feature distillation [67,68]. BitDistiller couples asymmetric two/three-bit quantization with self-distillation, while EfficientQAT separates block-wise parameter optimization from end-to-end quantization-parameter refinement [69,70]. EfficientQAT’s Llama2-70B two-bit result, for example, reaches 69.48% on its five-task average against 72.41% for its FP16 reference, but requires a conversion budget that must be distinguished from inherited pretraining. These methods occupy a continuum of adaptation cost, parameter freedom, and quality preservation rather than a clean separation into “training” and “no training.”

BitNet Distillation demonstrates that native BitNet pretraining is not the only way to obtain a ternary-weight, eight-bit-activation model [71]. It starts from pretrained Qwen3 checkpoints, inserts normalization, performs continued pretraining, and then uses task-level logit and relation distillation. Its language-understanding and summarization results establish task-specific conversion, not universal equivalence to native pretraining or preserved capability on every generative benchmark. Tequila addresses a different optimization obstacle: latent weights can oscillate near ternary decision boundaries despite receiving surrogate gradients [72]. Its latent-weight-derived bias introduces a direct gradient pathway and can be precomputed for inference. The distinction between a training aid that disappears or becomes constant and an inference-time residual is important for both storage and runtime accounting.

Recent ternary PTQ methods increasingly optimize the coordinate system and reconstruction objective rather than rely on a fixed threshold. ${\mathrm{PT}}^2$-LLM combines ternary fitting with activation-aware grid alignment and structural reordering [11]. CAT-Q learns weight modulation and progressively transitions from soft to hard ternarization within sliding-layer reconstruction [12]. Its group-wise reconstruction retains a zero-centered ternary output even though an intermediate redistribution uses a learned mean. The method’s approximately one million unique calibration tokens are reused across optimization epochs; this is not equivalent to learning a new model from that many tokens. The larger dense and mixture-of-experts experiments broaden the model families for which ternary conversion is supported, while the paper’s reasoning-task failures show that preserving commonsense accuracy need not preserve mathematics or code generation.

Structured residual and sparsity designs make the connection to execution more explicit. QTEA combines an affine ternary base with FP8 residuals assigned to selected columns and a local one-in-four pattern, then fuses those corrections into a GPU lookup kernel [15]. Its 1.7-bit payload counts the ternary values and residual values, not all indices, scales, or wider model tensors. Sherry instead constrains the base model itself to three nonzeros per four weights and uses an annealed training residual to recover quality [42]. The two approaches illustrate opposite uses of structure: one adds a small regular correction to a flexible base, while the other reduces the base’s legal patterns to improve packing and lookup alignment. Their quality and timing experiments should be matched by checkpoint before a joint quality–speed advantage is inferred.

Vector coding remains an important alternative when an extremely small scalar alphabet is too restrictive. The original QuIP uses incoherence processing and adaptive rounding to improve two-bit quantization [73]. QuIP# combines Hadamard transforms with a structured lattice codebook, whereas AQLM learns additive vector codebooks [18,37]. Both achieve small average code rates without requiring each scalar weight to be binary or ternary. They also shift the implementation problem from sign selection to codebook locality and reconstruction. Consequently, vector coding is not merely an out-of-scope bit-width variant: it is a meaningful comparison for whether a simple discrete alphabet is worth its learning constraints.

Table 3. Representative LLM construction choices. W$\mathrm{T}$ denotes ternary codes, not a physical 1.58-bit datatype. Precision refers to the targeted path; embeddings, nonlinear operators, accumulators, and state can remain wider.

Examples	Target precision	Learning pathway	Interpretation
BitNet / b1.58 / 2B4T [4,5,10]	W1 or W$\mathrm{T}$, A8	Native pretraining; 2B4T adds instruction and preference training.	Low-bit forward graph, wider optimization state.
FBI-LLM [58]	W1A16	Randomly initialized student with autoregressive teacher supervision.	Teacher cost is part of construction.
PB-LLM / BiLLM [35,36]	Binary plus exceptions or residuals	Saliency-guided checkpoint conversion.	Irregular metadata and correction paths.
OneBit / TernaryLLM [67,68]	W1 or W$\mathrm{T}$, A16	Quantized adaptation with learned scales and distillation.	Compact matrix is not the whole checkpoint.
EfficientQAT / BitDistiller [69,70]	W2–4 or W2–3	Block/end-to-end optimization or self-distillation.	Wider-bit alternatives with explicit adaptation cost.
${\mathrm{PT}}^2$-LLM / CAT-Q [11,12]	W$\mathrm{T}$; A16, also CAT-Q A8	Fitting and output alignment; CAT-Q uses learned soft-to-hard conversion.	Calibration size and optimization work differ.
TWLA / Bit-by-Bit [13,74]	W$\mathrm{T}$ with mixed A; W2A2	Joint activation/weight adaptation through transforms or progressive QAT.	Average activation budgets and channel expansion need accounting.
Sherry / QTEA [15,42]	Sparse ternary; ternary plus FP8	Structured QAT or residual-compensated PTQ.	Restrictions and corrections are designed for kernels.

Table 3. Representative LLM construction choices. W$\mathrm{T}$ denotes ternary codes, not a physical 1.58-bit datatype. Precision refers to the targeted path; embeddings, nonlinear operators, accumulators, and state can remain wider.

Examples	Target precision	Learning pathway	Interpretation
BitNet / b1.58 / 2B4T [4,5,10]	W1 or W$\mathrm{T}$, A8	Native pretraining; 2B4T adds instruction and preference training.	Low-bit forward graph, wider optimization state.
FBI-LLM [58]	W1A16	Randomly initialized student with autoregressive teacher supervision.	Teacher cost is part of construction.
PB-LLM / BiLLM [35,36]	Binary plus exceptions or residuals	Saliency-guided checkpoint conversion.	Irregular metadata and correction paths.
OneBit / TernaryLLM [67,68]	W1 or W$\mathrm{T}$, A16	Quantized adaptation with learned scales and distillation.	Compact matrix is not the whole checkpoint.
EfficientQAT / BitDistiller [69,70]	W2–4 or W2–3	Block/end-to-end optimization or self-distillation.	Wider-bit alternatives with explicit adaptation cost.
${\mathrm{PT}}^2$-LLM / CAT-Q [11,12]	W$\mathrm{T}$; A16, also CAT-Q A8	Fitting and output alignment; CAT-Q uses learned soft-to-hard conversion.	Calibration size and optimization work differ.
TWLA / Bit-by-Bit [13,74]	W$\mathrm{T}$ with mixed A; W2A2	Joint activation/weight adaptation through transforms or progressive QAT.	Average activation budgets and channel expansion need accounting.
Sherry / QTEA [15,42]	Sparse ternary; ternary plus FP8	Structured QAT or residual-compensated PTQ.	Restrictions and corrections are designed for kernels.

4.3. Activations, Persistent State, and Preserved Capabilities

Weight compression leaves activation computation and persistent state as independent design problems. BitNet a4.8 combines four-bit inputs with sparse eight-bit intermediate activations, whereas BitNet v2 uses online Hadamard transformations to make dense four-bit activation quantization more stable [75,76]. The v2 experiments use 95 billion tokens at A8 followed by five billion at A4 for the four-bit variant, with wider input/output embeddings. Figure 2 compares v2’s A8 and A4 variants within that study. It supports the feasibility of reducing activation precision with a compatible training recipe, but not a claim of statistical equivalence or free conversion of arbitrary A8 models. The transformation also has an online cost that an efficient implementation must absorb.

Checkpoint-based methods increasingly tackle the same activation challenge. TWLA combines output-aware ternary fitting with learned orthogonal transformations and an inter-layer activation-precision allocation [13]. Its A4 designation is an average layer budget selected from two-, four-, six-, and eight-bit candidates, not uniform four-bit execution. Bit-by-Bit progressively reduces precision and splits outlier channels, demonstrating W2A2 conversion while adding scale storage and expanded channels [74]. Its Llama2-7B WikiText2 perplexity increases from 5.47 at FP16 to 6.50 at W2A16 and 7.72 at W2A2. The additional loss identifies activation quantization as a substantive constraint, even when a progressive recipe avoids collapse. Reported kernel speedups and complete-generation gains also differ, reinforcing the need to evaluate the actual integrated path.

The key–value (KV) cache can erase much of the apparent memory advantage of low-bit weights at long contexts or large batches. For dense cached attention with equal key/value head dimensions, a simple payload calculation is

$$M_{\mathrm{KV}}={\displaystyle \frac{LBS{H}_{\mathrm{KV}}{d}_{\mathrm{head}}(b_K+b_V)}{8}}\mathrm{bytes},$$

(8)

where L is layer count, B is batch size, S is stored context length, $H_{\mathrm{KV}}$ is the number of KV heads, and $b_K,b_V$ are storage bits per element. The expression excludes scales, paging overhead, and workspace and changes with prefix sharing, eviction, or alternative attention. Weight precision does not appear in it. BitNet activation variants therefore investigate lower-precision KV state separately, retaining selected exceptions such as the beginning-of-sequence token [75,76]. Recurrent or hybrid attention changes the state representation more fundamentally, but must be evaluated as an architectural intervention rather than attributed entirely to quantization.

Capability preservation adds another axis that perplexity and short multiple-choice tasks cannot fully summarize. ScaleQ-1.58 builds on CAT-Q and calibrates with questions, reasoning traces, and answers produced by the pretrained model being converted [14]. Its Qwen3-4B ablation changes only the calibration scheme within the ternary-conversion setting and reveals large differences on mathematics and coding. Figure 3 includes the high-precision reference to show both the recovery and the remaining gap. The result supports the importance of matching the calibration distribution to the behavior being preserved; it does not imply that self-generated data is universally better than external supervision. Other domains, model sizes, contamination controls, and generation protocols remain relevant to that conclusion.

Learning efficiency must be interpreted with the same care. Four million calibration tokens in a converted model are additional to the original model’s pretraining, and repeated passes and trace generation consume resources beyond the corpus size [14]. Native pretraining and conversion can both be reasonable choices: the former permits architecture and quantizer co-design, while the latter reuses an existing model’s capabilities and ecosystem. The most useful comparison therefore reports inherited resources, additional adaptation, the capabilities retained, and the resulting executable artifact. It avoids treating a low nominal precision, a small calibration set, or an isolated benchmark score as a complete measure of model efficiency.

5. Execution on CPUs and GPUs

5.1. Lookup, Decoding, and Integer Arithmetic

Commodity processors can exploit discrete weights without a native instruction for every nominal format. The central choice is whether to avoid individual products, reduce their precision, or retain efficient wide arithmetic while reducing weight traffic. For a small activation group ${\mathbf{x}}_g$, a lookup implementation can precompute

$${\mathcal{T}}_g\left[\mathbf{c}\right]={\displaystyle \sum _{j=1}^G}{c}_j{x}_{g,j},$$

(9)

for allowed binary or ternary code patterns $\mathbf{c}$, then replace many repeated products with table lookup and accumulation. Table construction is amortized when the same activation group contributes to many output channels. Larger groups reduce the number of lookups but enlarge the pattern space exponentially; structured sparsity and sign symmetry can reduce that space [16,42,77]. The representation therefore determines both the table’s contents and whether it fits the relevant registers or memory.

T-MAC makes this approach practical on CPUs by decomposing low-bit weights into bit planes and using packed patterns to select activation sums from SIMD registers [16]. Tiling, table quantization, and weight interleaving align the computation with the processor’s register hierarchy. The important contribution is not simply substituting lookup for multiplication: without those layout and scheduling choices, lookup can itself be expensive. Its measured kernel and integrated-generation results establish a software path on existing ARM and x86 processors. Its M2 Ultra energy study uses sampled powermetrics telemetry, providing a different kind of evidence from an arithmetic-energy calculation. Optional approximation modes and inferred comparison points must remain distinct from directly measured, numerically matched configurations.

bitnet.cpp specializes more directly for ternary models [77,78]. Its family includes I2_S, which unpacks two-bit codes for arithmetic, and TL variants that exploit grouped ternary patterns and symmetry. Three-trit/five-bit packing can reduce payload relative to scalar two-bit storage, but the fastest choice depends on register pressure, unpacking, and bandwidth. The 2024 technical report’s synthetic size-sweep configurations establish implementation scaling, not the quality of trained models at all plotted sizes. The later paper also distinguishes table-quantized variants from variants that preserve wider table values. Consequently, “lossless” must refer to a particular kernel and numerical reference rather than every implementation distributed under the same project name.

Continued software development makes source versioning particularly important. The January 2026 BitNet CPU artifact integrates I2_S GEMM and GEMV and amortizes weight unpacking across multiple activation rows [79]. Figure 4 plots its within-kernel ablation: the same representation behaves differently when there is enough row reuse to share unpacking work. The artifact also includes wider Q6_K embedding support, illustrating why an effective ternary deployment may deliberately use more than one weight format. Sherry changes the legal weight patterns to fit four-element activation groups and small SIMD tables [42]. Its performance evidence should remain separate from its Llama-3.2 quality results because the reported timing configurations use different BitNet variants. These examples show how model restrictions and implementation choices can be exchanged for one another.

Instruction-set extensions explore what software cannot express efficiently with existing primitives. T-SAR proposes in-register table generation and fused lookup accumulation through TLUT and TGEMV, using a sign plane and a zero-indicator plane for ternary weights [80]. It extends the CPU datapath rather than merely providing another downloadable kernel. Its evaluation uses gem5 and synthesis of the modified SIMD logic, so the reported gains are architectural estimates, not measurements of shipping processors executing those new instructions. The comparison with T-MAC is therefore one of mechanism and implementation opportunity, not a direct contest between equally available software packages.

5.2. GPU Mapping and Complete-Model Serving

On GPUs, the meaning of “lookup” must be made explicit. LUT-GEMM uses shared-memory tables of activation-derived partial products, avoiding per-weight reconstruction before multiplication [17]. Its binary-coded weight planes still require scale application, accumulation, and communication across threads. FLUTE instead accelerates lookup-table-quantized weights by decoding codebook entries into operands for wider matrix arithmetic [19]. The first table represents products that depend on current activations; the second represents stored weight values. They have different construction, reuse, and memory requirements, even though both are described as LUT-based acceleration. This distinction is useful when deciding whether a quantizer’s codebook and a kernel’s lookup mechanism are actually compatible.

Wider arithmetic can be an effective way to execute a compact model. MARLIN pipelines four-bit weight decoding with FP16 tensor-core computation and addresses the intermediate batch sizes that arise in autoregressive inference [81]. QuIP# reconstructs lattice codewords for FP16 matrix operations, while Spectra 1.1’s TriRun adapts the Marlin-style approach to ternary-weight storage [18,82]. AQLM supplies another codebook-based path on both CPUs and GPUs [37]. These methods benefit when lower weight traffic outweighs decoding and transformation cost; different codebook sizes or matrix shapes can reverse that balance. Ladder generalizes part of this design problem through hardware-aware tensor transformations and scheduling [83]. Thus, a useful low-bit compiler must select layouts and conversion schedules in addition to choosing operand types.

Integer and bitwise GPU paths offer different trade-offs. The public BitNet GPU kernel packs two-bit weights, decodes them into an INT8 dot-product path using dp4a, and accumulates in INT32 [84]. This is neither FP16 tensor-core decoding nor a literal native ternary instruction. BWTA explores binary weights and ternary activations using packed bit planes and binary matrix primitives [85]. Its selected-layer experiments and baseline choices do not establish a uniformly binarized complete LLM deployment, but they illustrate another way to map model discreteness to GPU instructions. QTEA takes the activation-LUT route and fuses structured FP8 residual corrections into the same generation path [15]. The available choices reinforce that a scalar bit label is insufficient to predict the executed instruction sequence.

Table 4. Execution mechanisms on existing processors and proposed CPU extensions. Examples sharing a row need not have the same numerical semantics, quality, or performance.

Mechanism	What is reused or decoded	Examples	Main constraint
CPU activation LUT	Partial activation sums in registers	T-MAC, bitnet.cpp TL, Sherry [16,42,77]	Table size, index layout, register pressure.
Unpack and integer dot	Packed weights become integer operands	bitnet.cpp I2_S; BitNet GPU [79,84]	Decode cost and activation reuse.
GPU activation LUT	Activation-dependent partial sums	LUT-GEMM, QTEA [15,17]	Table placement, construction cost, lookup bandwidth, reduction.
Decode to matrix units	Quantized values or codewords become wider fragments	MARLIN, FLUTE, QuIP#, TriRun [18,19,81,82]	Conversion scheduling, shape, codebook locality.
Packed binary matrix	Bit planes enter binary matrix primitives	BWTA [85]	Operand semantics, packing, supported operators.
Proposed fused CPU ISA	Tables generated and consumed by extended SIMD logic	T-SAR [80]	Requires hardware changes; simulated evaluation.

Table 4. Execution mechanisms on existing processors and proposed CPU extensions. Examples sharing a row need not have the same numerical semantics, quality, or performance.

Mechanism	What is reused or decoded	Examples	Main constraint
CPU activation LUT	Partial activation sums in registers	T-MAC, bitnet.cpp TL, Sherry [16,42,77]	Table size, index layout, register pressure.
Unpack and integer dot	Packed weights become integer operands	bitnet.cpp I2_S; BitNet GPU [79,84]	Decode cost and activation reuse.
GPU activation LUT	Activation-dependent partial sums	LUT-GEMM, QTEA [15,17]	Table placement, construction cost, lookup bandwidth, reduction.
Decode to matrix units	Quantized values or codewords become wider fragments	MARLIN, FLUTE, QuIP#, TriRun [18,19,81,82]	Conversion scheduling, shape, codebook locality.
Packed binary matrix	Bit planes enter binary matrix primitives	BWTA [85]	Operand semantics, packing, supported operators.
Proposed fused CPU ISA	Tables generated and consumed by extended SIMD logic	T-SAR [80]	Requires hardware changes; simulated evaluation.

Complete-model measurements reveal costs that kernel benchmarks cannot capture. LUT-GEMM’s reported two-bit A100 GEMV speedup of $4.85\times$ concerns one large matrix-vector product, while its OPT-175B model experiments use different groupings and device counts [17]. QTEA reports a reduction from 41.13 to 5.70 ms per generated token for its Llama2-70B H200 experiment with CUDA Graphs enabled for both configurations [15]. That result is specific to the reported generation setup and should not be generalized to prefill, arbitrary context lengths, or matched-quality replacements without further evidence. Its custom-hardware energy estimates are separate from those GPU timing measurements. These distinctions allow performance results to support precise conclusions without discarding useful implementation evidence.

Serving introduces a further shift in the relevant objective. Prefill processes multiple prompt positions with substantial weight reuse; small-batch decode often streams weights repeatedly, while large batches and long contexts increase attention and KV traffic. QServe co-designs W4A8KV4 quantization with layouts and integer computation for throughput-oriented serving [86]. Although it is not a binary or ternary model, it is an essential alternative when a slightly wider representation better fits a target workload. FlashInfer addresses ragged queries, paged KV state, shared prefixes, attention kernels, and scheduling [87]. Its contributions remain relevant after projections become cheaper, because quantizing weights does not remove attention or request-management work. Maximum-capacity throughput, fixed-batch latency, time to first token, and inter-token latency should therefore be treated as different objectives rather than collapsed into one “inference speed” number.

6. Specialized Hardware and Memory Computing

6.1. Spatial Dataflow and Heterogeneous Digital Engines

Dedicated hardware can specialize not only arithmetic but also the movement and lifetime of intermediate values. FINN maps binary networks to layer-specialized FPGA pipelines, using folding and local storage to balance throughput across the graph [20]. BISMO instead offers a reusable bit-serial matrix engine that constructs multibit products from binary partial products [33]. The former exchanges flexibility for a tailored network dataflow; the latter permits precision and matrix-shape variation but pays the corresponding control and accumulation costs. Both provide physical FPGA evidence, although BISMO’s matrix-engine measurements should not be described as complete-model inference. Their designs remain relevant beyond early classifiers because streaming, folding, and precision-scalable arithmetic recur in later Transformer accelerators.

Binary and ternary ASICs demonstrate that weight precision and activation precision need not be reduced together. YodaNN specializes binary-weight convolution while retaining 12-bit activations, with evaluation based on post-place-and-route timing and power analysis [88]. TCN-CUTIE instead uses ternary weights and activations in an all-digital architecture evaluated on a fabricated SoC for classification and event-based gesture recognition [89]. The measured energy includes the accelerator’s local memories but not every external system component. The comparison is consequently architectural rather than a cross-process energy ranking: one design retains activation magnitude, while the other constrains both operands and relies on a compatible model and dataflow. Peak operation efficiency and average application energy are different quantities even within one chip.

Transformer acceleration requires handling multiple kinds of matrix products and wider nonlinear operations. BETA and BAT provide configurable binary engines and supporting scale/nonlinear paths, while COBRA co-designs binary attention with XNOR/AND, population-count, and correction operations [90,91,92]. The encoder setting uses bounded input sequences and differs from autoregressive generation with a growing cache. Moreover, RTL simulation, tool-estimated power, and on-board timing must be identified separately; a published FPGA target alone does not establish measured model energy. These encoder studies help explain the need for heterogeneous execution, but their GOPS or sequence latencies cannot be directly translated into LLM tokens per second.

Modern ternary LLM accelerators increasingly specialize around that heterogeneity. TENET combines a sparsity-aware ternary lookup core with wider attention cores and projection-aware scheduling [21]. Five-trit bytes reduce external weight traffic, while two-bit on-chip codes simplify consumption; dynamic activation sparsity changes which channels the lookup core processes. Its FPGA prototype and modeled ASIC configuration establish different kinds of evidence, and the FPGA power values come from design-tool reports. VitaLLM combines ternary projection engines with a reconfigurable attention engine, selective KV fetching, and dependency-aware scheduling [93]. Its post-layout simulations and analytical timing model explore a compact design point, but do not establish fabricated-chip measurements or complete memory-system energy. In both cases, the contribution goes beyond a cheaper multiply by addressing which operators share resources and how dependencies affect utilization.

Resource-constrained FPGAs expose the same scheduling problem from a different direction. TeLLMe and TeLLMe v2 use ternary lookup, mixed-precision operators, and streaming/fusion to support prefill and decode; PD-Swap explores phase-specific attention hardware through dynamic partial reconfiguration [94,95,96]. These related studies illustrate reuse versus specialization on one platform, rather than independent replications of one acceleration result. Their timing and host-side work must be interpreted using the stated end-to-end scope, and unspecified power-acquisition protocols do not establish measured energy savings. Slim-Llama is another relevant billion-parameter binary/ternary processor reported at ISSCC [97]; we do not use its headline power numerically because the corresponding measurement boundary could not be verified from the accessible primary material.

Changing the model can reduce the workload that hardware must support. TerEffic targets recurrent MatMul-free language models with on-chip and HBM-assisted FPGA configurations [62]. Its resident small-model configuration spans two FPGAs, its larger-model analysis includes bandwidth-derived projections, and embeddings and output layers fall outside the decoder-acceleration boundary. ELiTeFormer combines ternary projections with hybrid linear and sliding-window attention [98]. It reports physical VCK5000 execution for a scaled-down, roughly 1B configuration, while the 8B hardware results are simulated and the quality study uses the larger model. These designs reveal an important opportunity in reducing state growth, but their efficiency benefits include an architectural change and cannot be credited entirely to ternary arithmetic.

Table 5. Digital accelerator examples, separated by workload and evidence. “Modeled” includes simulation, synthesis, or post-layout analysis without physical execution of the reported design. The table is not a ranking of performance or evidence quality.

Examples	Workload	Architectural emphasis	Evidence boundary
FINN / BISMO [20,33]	Binary classifiers / multibit matrix products	Spatial pipelines / reusable bit-serial arithmetic	Measured FPGA execution; network versus kernel scope differs.
YodaNN / TCN-CUTIE [88,89]	Binary-weight CNN / ternary CNN and temporal network	Specialized convolution and local storage	YodaNN post-route analysis; TCN-CUTIE fabricated-chip measurements.
BETA / BAT / COBRA [90,91,92]	Encoder Transformers	Binary products plus wider attention/normalization support	Simulation/tool estimates versus on-board timing; energy protocols differ.
TENET [21]	Ternary LLMs	Sparse lookup, heterogeneous cores, packed weight streaming	FPGA validation with tool power; modeled ASIC.
VitaLLM [93]	Ternary Transformer prefill/decode	Core reuse, KV selection, dependency scheduling	Analytical timing and post-layout simulation.
TeLLMe / PD-Swap [95,96]	Small ternary Transformers	Streaming/fusion or phase-specific reconfiguration	Reported phase timing; power protocol unspecified.
TerEffic [62]	Recurrent language-model decoders	Resident weights and HBM alternatives	Post-route/tool analysis and projections; embeddings/head excluded.
ELiTeFormer [98]	Hybrid-attention language model	Ternary projections and bounded attention state	Small FPGA deployment; larger quality and simulation configurations differ.

Table 5. Digital accelerator examples, separated by workload and evidence. “Modeled” includes simulation, synthesis, or post-layout analysis without physical execution of the reported design. The table is not a ranking of performance or evidence quality.

Examples	Workload	Architectural emphasis	Evidence boundary
FINN / BISMO [20,33]	Binary classifiers / multibit matrix products	Spatial pipelines / reusable bit-serial arithmetic	Measured FPGA execution; network versus kernel scope differs.
YodaNN / TCN-CUTIE [88,89]	Binary-weight CNN / ternary CNN and temporal network	Specialized convolution and local storage	YodaNN post-route analysis; TCN-CUTIE fabricated-chip measurements.
BETA / BAT / COBRA [90,91,92]	Encoder Transformers	Binary products plus wider attention/normalization support	Simulation/tool estimates versus on-board timing; energy protocols differ.
TENET [21]	Ternary LLMs	Sparse lookup, heterogeneous cores, packed weight streaming	FPGA validation with tool power; modeled ASIC.
VitaLLM [93]	Ternary Transformer prefill/decode	Core reuse, KV selection, dependency scheduling	Analytical timing and post-layout simulation.
TeLLMe / PD-Swap [95,96]	Small ternary Transformers	Streaming/fusion or phase-specific reconfiguration	Reported phase timing; power protocol unspecified.
TerEffic [62]	Recurrent language-model decoders	Resident weights and HBM alternatives	Post-route/tool analysis and projections; embeddings/head excluded.
ELiTeFormer [98]	Hybrid-attention language model	Ternary projections and bounded attention state	Small FPGA deployment; larger quality and simulation configurations differ.

6.2. In-Memory Computation and Fixed-Weight Storage

Compute-in-memory attacks a different cost from a conventional low-bit arithmetic unit: moving operands between storage and processing. PIMBALL implements binary operations through stateful digital logic in spintronic memory, assembling neural computation from memory-resident Boolean primitives [99]. TiM-DNN instead uses ternary storage/processing cells with analog bitline accumulation, sensing, and peripheral reduction [22]. These are distinct mechanisms, not interchangeable implementations of one generic PIM primitive. PIMBALL’s architectural simulation depends on device and operation assumptions, while TiM-DNN combines circuit and system modeling. In both, array-level parallelism must be reconciled with peripheral logic, communication, numerical range, and the model mapped onto the array.

Physical memory experiments establish valuable but often narrower evidence than complete neural-network execution. The resistive-RAM ternary-synapse study of Laborieux et al. [100] uses differential 2T2R devices and demonstrates physical read/programming behavior, while network accuracy is evaluated separately in software. A working ternary synapse therefore does not establish measured full-network throughput or energy per inference. SiTe CiM explores signed ternary dot products across SRAM, eDRAM, and ferroelectric-FET implementations using circuit and architectural analysis [101]. The underlying technology matters because sensing range, conversion, storage reliability, and supported input encoding change the attainable trade-offs. An alphabet containing three values does not imply 1.58 physical memory bits or a device-independent energy cost.

Ternary’s zero state is also not an unconditional energy advantage in analog arrays. The RRAM scaling study of Cubero-Cascante et al. [102] examines parasitic effects, sensing, and calibration for binary and ternary CNN mappings. In its tested configurations, ternary representations improve robustness but require additional input cycles and can reduce energy efficiency. This counterexample is useful because it separates model tolerance from execution cost: a representation can improve accuracy under nonideal hardware while still consuming more energy. Similarly, lookup-core design-space exploration finds that the relative cost of adders and selection networks depends on activation precision [103]. A design favored for FP16 activations need not be favored for INT8, even when the ternary weights are identical.

Read-only memory offers another way to keep weights close to computation when the base model changes infrequently. BitROM explores ternary storage and computation in a ROM-centered design, while TOM synthesizes ternary weight contents into sparse standard-cell logic and distributes that storage near compute [23,104]. TOM is ROM-resident digital processing, not analog crossbar accumulation. Its writable state and adapter paths provide some tunability, but do not make the base weights arbitrarily replaceable. Both studies rely on modeled hardware evaluation rather than demonstrated fabricated full-model systems. Their central trade-off remains significant: eliminating repeated weight loading can improve locality, but fixes knowledge into a hardware lifecycle and changes the costs of model updates, fault correction, and general-purpose reuse.

Table 6. Memory and emerging-substrate approaches. A physical device experiment, a modeled neural mapping, and a measured system are different evaluation objects.

Examples	Mechanism	Demonstrated scope or limitation
PIMBALL [99]	Stateful digital logic in spintronic memory	Architectural simulation; current and projected devices must be distinguished.
TiM-DNN / SiTe CiM [22,101]	Ternary array operations with sensing/peripheral reduction	Circuit and system models; array throughput excludes some surrounding work.
Ternary RRAM synapse [100]	Differential 2T2R representation	Physical device measurements plus separate software network evaluation.
BitROM / TOM [23,104]	Fixed-weight ROM locality and digital computation	Post-layout or synthesis/simulation studies; adapters do not replace the immutable base.
TrueNorth mapping [105]	Ternary synapses with binary spike activity	Physical single-chip execution; larger mappings and power accounting require qualification.
MatMul-free on Loihi 2 [61]	Recurrent ternary model mapped to neuromorphic cores	Component and multichip evidence; embeddings/unembedding excluded from current experiments.
OXBNN [106]	Optical XNOR and photo-charge accumulation	Component/system simulation, not a fabricated complete optical processor.

Table 6. Memory and emerging-substrate approaches. A physical device experiment, a modeled neural mapping, and a measured system are different evaluation objects.

Examples	Mechanism	Demonstrated scope or limitation
PIMBALL [99]	Stateful digital logic in spintronic memory	Architectural simulation; current and projected devices must be distinguished.
TiM-DNN / SiTe CiM [22,101]	Ternary array operations with sensing/peripheral reduction	Circuit and system models; array throughput excludes some surrounding work.
Ternary RRAM synapse [100]	Differential 2T2R representation	Physical device measurements plus separate software network evaluation.
BitROM / TOM [23,104]	Fixed-weight ROM locality and digital computation	Post-layout or synthesis/simulation studies; adapters do not replace the immutable base.
TrueNorth mapping [105]	Ternary synapses with binary spike activity	Physical single-chip execution; larger mappings and power accounting require qualification.
MatMul-free on Loihi 2 [61]	Recurrent ternary model mapped to neuromorphic cores	Component and multichip evidence; embeddings/unembedding excluded from current experiments.
OXBNN [106]	Optical XNOR and photo-charge accumulation	Component/system simulation, not a fabricated complete optical processor.

6.3. Neuromorphic and Optical Implementations

Neuromorphic implementations demonstrate another relationship between model constraints and physical execution. The TrueNorth convolutional-network mapping uses ternary synaptic weights and binary spike activity under the chip’s connectivity constraints [105]. Physical single-chip execution is distinguished from simulated larger configurations, and energy accounting does not include all sensing and input-transduction work. This is relevant discrete neural computation, but it does not make every spiking neural network a conventional W1A1 network. The later MatMul-free language-model work maps recurrent ternary projections to Loihi 2 and reports component and multichip characterization [61]. Its current experiments exclude embedding and unembedding, and some system figures incorporate derived communication effects. The contribution is evidence that model architecture can make an unconventional platform useful, with a different completeness boundary from an ordinary end-to-end language-model runtime.

Optical implementations pursue parallel binary operations through different physical mechanisms. OXBNN proposes microring-based XNOR operations and photo-charge accumulation for bit counting, evaluated through component and system simulation [106]. Its opportunity comes from parallel physical signal processing, while laser power, conversion, tuning, and electronic interfaces remain part of the implementation cost. Those costs cannot be inferred from a binary operation count, and simulation does not establish a fabricated optical neural processor. Across digital, memory, neuromorphic, and optical designs, the useful common question is therefore which operand movement or computation has actually been removed and which new physical or programmability cost has replaced it.

7. Synthesis and Research Directions

The literature supports a conditional view of extreme quantization rather than a universal winner among bit widths. At a fixed parameter count, extra levels often ease approximation; at a fixed storage budget, a larger discrete model can recover capacity; at a fixed execution budget, layout and supported arithmetic may favor a wider format. ParetoQ and QuEST make the resource objective explicit, while CPU/GPU implementations show why model-size frontiers need not coincide with latency frontiers [16,59,60,81]. Future comparisons should therefore optimize a stated deployment objective under a capability requirement. The question is not simply how low precision can go, but which model, construction procedure, and execution strategy best satisfy that requirement.

A first opportunity is to make learning objectives aware of executable structure. Learned scales, shifts, residuals, and sparsity can recover quality, but their granularity determines how often a kernel must load metadata, route values, or perform corrections. Sherry and QTEA illustrate different ways to coordinate those decisions with lookup execution [15,42]. Rather than selecting a mathematical quantizer and retrofitting a kernel, training or calibration could explore hardware-supported group sizes, correction patterns, codebook capacities, and activation formats directly. The challenge is portability: optimizing tightly for one SIMD table or memory array can sacrifice efficiency on another processor. A useful direction is a small set of interchangeable representations with explicit conversion cost, rather than a nominal format that every backend interprets differently.

A second opportunity is to preserve capabilities instead of only local reconstruction or aggregate benchmark scores. The calibration results in Figure 3 show that similar ternary models can retain very different mathematics and coding behavior [14]. Long-form generation, multilingual tasks, multimodal inputs, tool use, and distribution shift may stress parts of the model that short calibration corpora do not expose. Improving a selected reasoning benchmark also does not establish preserved reliability or instruction-following behavior elsewhere. Research on calibration diversity, teacher choice, and state trajectories should therefore be paired with held-out capability evaluation and checks for benchmark overlap. This connects current LLM conversion to the older lesson that task-specific losses and architectures matter in discrete detection and generation [53,54].

A third opportunity is to treat activations and persistent state as first-class resources. Low-bit weights can make projections inexpensive enough that attention, KV movement, normalization, or sampling dominates. For an accelerated fraction f of an otherwise unchanged serial runtime, even an ideal local speedup s yields only

$$S_{\mathrm{total}}={\displaystyle \frac{1}{(1-f)+f/s}}.$$

(10)

This explanatory limit is not a model of every overlapped pipeline, but clarifies why optimizing an already small component has diminishing returns. BitNet v2 and TWLA address activation precision, QServe addresses both activation and KV formats, and hybrid/recurrent accelerators change state growth more fundamentally [13,76,86,98]. The research challenge is to recover complete-model efficiency without assuming that each approximation’s quality effect is independent of the others.

Hardware flexibility deserves the same attention as arithmetic efficiency. Spatial pipelines can specialize for a fixed network, configurable engines can serve multiple precisions, and fixed-weight ROM can remove movement at the cost of base-model mutability [20,23,33]. As models evolve, the cost of recompilation, reconfiguration, adaptation, and replacement becomes part of the design decision. In memory and emerging-substrate systems, reliability, sensing, and interface overhead add further constraints that do not appear in logical bit counts [100,102,106]. Progress requires connecting these physical limits to model tolerance and update requirements, rather than extrapolating a favorable array-level operation into an entire language-model system.

Finally, stronger empirical connections between learning and deployment would make results more reusable. The released checkpoint, transformation parameters, packing layout, operator coverage, and runtime configuration should describe the same model used for the reported quality and efficiency results. Within-study comparisons should identify inherited pretraining and additional adaptation, while complete-generation measurements should state batch, prompt and output length, KV policy, and whether host work is included. Energy should identify the measured device or rail and distinguish telemetry or direct metering from arithmetic and design-tool estimates. These are substantive requirements for interpreting the trade-off, not a demand that every architecture study provide fabricated silicon or that every learning paper implement a new kernel. They allow promising model ideas, measured software, physical device experiments, and hardware projections to contribute evidence without being mistaken for one another.

8. Conclusions

Binary and ternary neural networks have developed into a broad design space spanning discrete optimization, architecture, foundation-model construction, and cross-platform execution. Their shared opportunity is to replace unnecessary numerical detail with representations that preserve useful capabilities while reducing storage or computation. The successful mechanisms differ: real-valued paths and improved gradients recover capacity in binary networks, native learning and checkpoint adaptation support low-bit language models, and lookup, wider matrix units, spatial dataflow, or memory-resident computation exploit the resulting structure. These mechanisms should be compared through their complete costs and supported workloads rather than their nominal bit labels.

The next advances are likely to depend on how these choices are combined. Learning must account for executable formats and the capabilities that need to survive conversion; systems must account for activations, persistent state, and the work surrounding low-bit projections. Evidence that connects these layers can distinguish genuine deployment improvements from storage-only gains or local arithmetic opportunities. That connection provides a practical basis for deciding when binary or ternary computation is the right tool, and where wider or differently structured alternatives remain preferable.