Quantum-Enhanced LLM Cascade Routing: A QAOA Approach to Cost-Optimal Model Selection in Multi-Agent Systems

1. Introduction

1.1. The Model Selection Problem

Modern AI systems increasingly operate as multi-agent architectures where dozens to thousands of AI agents execute heterogeneous tasks ranging from simple classification to complex multi-step reasoning [1]. Each task must be routed to an appropriate large language model (LLM) from an expanding menu of options: from lightweight local models (Gemma 2B) to mid-range API models (Claude Haiku) to premium reasoning models (Claude Opus, o1).

The cost difference between tiers is dramatic—a factor of 100× to 1000× between local inference and premium APIs. Yet current production systems typically use static complexity-to-model lookup tables that ignore cross-task dependencies, budget constraints, and the combinatorial nature of batch assignment. When 80 agents simultaneously request model access, the assignment problem becomes: which tasks can be downgraded without quality loss, and which must be upgraded despite budget pressure?

1.2. Why Quantum?

The LLM Cascade Routing Problem (LCRP) is a constrained combinatorial optimization problem. Given N tasks and M models, the search space is $M^N$—growing exponentially with task count. For a realistic production scenario of 80 tasks and 5 models, this is $5^{80}\approx 8.3\times {10}^{55}$ possible assignments.

The Quantum Approximate Optimization Algorithm (QAOA) [2] is designed precisely for this class of problem. By encoding the objective function into a quantum Hamiltonian and performing alternating optimization and mixing operations, QAOA explores the solution space through quantum superposition and interference.

1.3. Contributions

This paper makes the following contributions:

1.

Novel problem formulation: The first QUBO encoding of the LLM Cascade Routing Problem, with cost minimization, quality constraints, budget limits, and latency SLAs.

2.

Real quantum hardware evaluation: QAOA executed on three IBM Heron 156-qubit processors via IBM Quantum, providing verifiable hardware results (16 quantum jobs).

3.

Shallow circuit advantage: Empirical demonstration that $p=1$ QAOA significantly outperforms deeper circuits on NISQ hardware for this problem class.

4.

Cross-backend reproducibility: Consistent results across three distinct quantum processors.

5.

Open-source implementation: Complete Cirq/Qiskit code and benchmark suite for reproducibility.

2. Related Work

2.1. Quantum Optimization for Scheduling

QAOA has been applied to scheduling problems structurally similar to LCRP. Deller et al. [3] formulated the Job Shop Scheduling Problem as a QUBO with time-indexed Hamiltonians. IBM demonstrated workforce task scheduling with 500–874 binary variables on 127-qubit Eagle hardware [4]. QTIS [5] introduced ancilla-assisted overlap detection for time-interval scheduling. D-Wave offers commercial workforce scheduling via quantum annealing [6]. However, none address the specific structure of LLM routing.

2.2. Classical LLM Routing

RouteLLM [7] uses preference-trained classifiers to route between model tiers. FrugalGPT [8] cascades from cheap to expensive models with early stopping. All use classical ML-based routing; none formulate the problem as combinatorial optimization.

2.3. Quantum Machine Learning

Variational Quantum Classifiers have shown promise for moderate-dimensional classification [9]. IonQ demonstrated quantum circuit layers for LLM fine-tuning [10]. No prior work applies quantum optimization to LLM model selection.

2.4. Research Gap

To our knowledge, no prior work: (a) formulates LLM routing as a QUBO/Ising problem, (b) applies QAOA to multi-agent model assignment, (c) benchmarks quantum vs. classical solvers on production LLM routing data, or (d) evaluates constrained QAOA on multiple quantum backends simultaneously.

3. Problem Formulation

3.1. The LLM Cascade Routing Problem

Definition. Given a set of N tasks $\mathcal{T}=\{t_1,\dots ,t_N\}$, each with complexity $c_i\in \{1,\dots ,5\}$, estimated token count $k_i$, minimum quality threshold $q_{min}\left(c_i\right)$, and latency SLA ${\ell }_{max}\left(c_i\right)$; and a set of M models $\mathcal{M}=\{m_1,\dots ,m_M\}$, each with cost-per-token $p_j$, quality score $q_j\in [0,1]$, and average latency ${\ell }_j$; and a budget constraint B:

Find a binary assignment matrix $X\in {\{0,1\}}^{N\times M}$ that minimizes total routing cost subject to quality, budget, and assignment constraints.

3.2. Objective Function

$$min\sum _{i,j}{X}_{ij}·p_j·k_i$$

(1)

Subject to:

$$\begin{array}{ccc}\hfill \sum _j{X}_{ij}& =1\forall i\hfill & \hfill (\mathrm{one}\mathrm{model}\mathrm{per}\mathrm{task})\end{array}$$

(2)

$$\begin{array}{ccc}\hfill q_j& \ge q_{min}\left(c_i\right)\mathrm{if}{X}_{ij}=1\hfill & \hfill (\mathrm{quality}\mathrm{floor})\end{array}$$

(3)

$$\begin{array}{ccc}\hfill \sum _{i,j}{X}_{ij}{p}_j{k}_i& \le B\hfill & \hfill (\mathrm{budget}\mathrm{cap})\end{array}$$

(4)

$$\begin{array}{ccc}\hfill {\ell }_j& \le {\ell }_{max}\left(c_i\right)\mathrm{if}{X}_{ij}=1\hfill & \hfill (\mathrm{latency}\mathrm{SLA})\end{array}$$

(5)

3.3. QUBO Encoding

We encode LCRP as a QUBO by converting constraints into penalty terms:

$$H=H_{\cos \mathrm{t}}+{\lambda }_A{H}_{\mathrm{assign}}+{\lambda }_Q{H}_{\mathrm{quality}}+{\lambda }_B{H}_{\mathrm{budget}}+{\lambda }_L{H}_{\mathrm{latency}}$$

(6)

where:

$$\begin{array}{cc}\hfill H_{\cos \mathrm{t}}& =\sum _{i,j}{X}_{ij}·{\widehat{C}}_{ij}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill H_{\mathrm{assign}}& =\sum _i{\left(1-\sum _j{X}_{ij}\right)}^2\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill H_{\mathrm{quality}}& =\sum _{i,j}{X}_{ij}·P_{ij}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill H_{\mathrm{budget}}& ={\displaystyle \frac{1}{B^2}}{\left[max\left(0,\sum _{i,j}{X}_{ij}{p}_j{k}_i-B\right)\right]}^2\hfill \end{array}$$

(10)

${\widehat{C}}_{ij}=p_j{k}_i/max\left(p_j{k}_i\right)$ is the normalized cost, $P_{ij}=10$ if $q_j<q_{min}\left(c_i\right)$ and 0 otherwise.

3.4. Complexity

Theorem 1.LCRP is NP-hard.

Proof sketch. By reduction from the Generalized Assignment Problem (GAP) [11]. Given a GAP instance, set task complexities uniformly to 1 (making quality constraints trivially satisfied), set latency SLAs to infinity, and map model costs to machine processing costs. The budget constraint maps to machine capacity. Since GAP is NP-hard, so is LCRP. □

4. Quantum Solution: QAOA

4.1. Algorithm Overview

QAOA [2] approximates the ground state of a cost Hamiltonian through p alternating applications of a cost unitary $U_C\left(\gamma \right)$ and a mixer unitary $U_M\left(\beta \right)$:

$$|\psi (\overrightarrow{\gamma },\overrightarrow{\beta })\rangle =\prod _{l=1}^p{U}_M\left({\beta }_l\right)U_C\left({\gamma }_l\right){|+\rangle }^{\otimes n}$$

(11)

The variational parameters $({\gamma }_1,\dots ,{\gamma }_p,{\beta }_1,\dots ,{\beta }_p)$ are optimized classically to minimize the expected cost $\langle \psi \left|H\right|\psi \rangle$.

4.2. Circuit Construction for LCRP

For LCRP with N tasks and M models, we require $N·M$ qubits. Qubit $q_{iM+j}$ represents the binary variable $X_{ij}$. Each QAOA layer consists of:

1.

Cost layer: $R_z\left(2\gamma h_{\mathrm{idx}}\right)$ on each qubit (linear terms) and $ZZ(2\gamma J_{ab}/\pi )$ on coupled pairs (one-hot enforcement).

2.

Mixer layer: $R_x\left(2\beta \right)$ on all qubits.

The coupling terms enforce constraint (2): for each task i, repulsive ZZ interactions are applied between all model pairs $(j,k)$, penalizing states where both $X_{ij}=1$ and $X_{ik}=1$.

4.3. Constraint Penalty Calibration

A critical practical challenge is calibrating penalty weights. Through systematic experimentation (Section 7.1), we determined:

• ${\lambda }_A=50$ (assignment)—must dominate to enforce valid solutions
• ${\lambda }_Q=40$ (quality)—must exceed maximum single-assignment cost savings
• ${\lambda }_B=5$ (budget)—soft constraint
• ${\lambda }_L=3$ (latency)—soft constraint, rarely binding

5. Classical Baselines

Greedy (Production Baseline). For each task, select the cheapest model satisfying quality and latency constraints. $O\left(NM\right)$ complexity, considers tasks independently.

Simulated Annealing (SA). Exponential cooling schedule ($T_0=50$, $T_f=0.1$, 2000 iterations), Metropolis acceptance, single-task reassignment moves.

Brute Force. Exhaustive enumeration for small instances ($M^N<{10}^6$), providing ground-truth optimal solutions.

6. Experimental Setup

6.1. Data Source

Task distributions derived from a production multi-agent platform operating 80 AI agents with cognitive tool-calling. Complexity distribution: trivial (30%), simple (25%), moderate (25%), complex (15%), expert (5%). Five production model tiers spanning 4 orders of magnitude in cost (Table 1).

6.2. Quantum Hardware

All hardware experiments were executed on IBM Quantum via the Open Plan (free tier). Three 156-qubit Heron-class processors were used:

• ibm_fez: Heron r1, heavy-hex topology
• ibm_kingston: Heron r1, heavy-hex topology
• ibm_marrakesh: Heron r1, heavy-hex topology

Circuits were built in Google Cirq 1.6.1, exported to OpenQASM 2.0, imported into Qiskit, transpiled for target backends (optimization level 1), and submitted via Qiskit Runtime SamplerV2 with 4000 shots per experiment.

6.3. Experiment Design

• Experiment A — Scaling: 6, 12, and 18 qubits ($2\times 3$, $4\times 3$, $6\times 3$) on all 3 backends. Tests noise impact as problem size grows.
• Experiment B — Circuit Depth: $p=1,2,3$ on 12 qubits (ibm_fez). Tests whether deeper QAOA improves solutions on noisy hardware.
• Experiment C — High-Shot: 8000 shots at 12 qubits (ibm_fez). Tests whether more samples compensate for noise.

QAOA parameters were pre-optimized on a noiseless Cirq simulator using COBYLA with 8 random restarts and 80 iterations each.

7. Results

7.1. Penalty Weight Sensitivity

We varied ${\lambda }_Q$ from 5 to 80 on a 4-task × 3-model instance (Table 2). Below ${\lambda }_Q\approx 20$, QAOA ignores quality constraints entirely, routing all tasks to the free model. Above ${\lambda }_Q=40$, QAOA matches the greedy baseline at 75% quality satisfaction. This critical penalty threshold corresponds to the point where the quality violation penalty exceeds the cost savings from downgrading.

7.2. Hardware Scaling (Experiment A)

Table 3 and Figure 1 show results across problem sizes on all three IBM backends. Key observations:

Finding 1: Steep noise scaling. Valid assignment rate drops from 37–43% at 6 qubits to 0.2–0.3% at 18 qubits on hardware. The 6-qubit instance retains 75–87% of simulator performance (∼10–15% degradation), while 12 and 18 qubits show hardware outperforming the simulator on valid rate—a counter-intuitive result likely caused by hardware noise acting as a form of implicit constraint regularization.

Finding 2: Hardware finds cheaper solutions. At 18 qubits, hardware backends found valid solutions at $1.41–$1.62, compared to the simulator’s $20.28 and the greedy baseline’s $14.22. While these solutions had lower quality satisfaction (17–33% vs. 67%), they demonstrate QAOA exploring a different region of the cost-quality Pareto frontier.

7.3. Circuit Depth Analysis (Experiment B)

Table 4 presents the most significant finding of this work.

Finding 3: Shallow circuits dominate on NISQ hardware. At $p=1$ (transpiled depth 52), QAOA achieves 15.4% valid assignments on ibm_fez—over 19× the $p=2$ rate (0.8%) and 123× the simulator rate (0.1%). This is a striking result: the simulator predicts that deeper circuits ($p=2,3$) should perform better, but real hardware noise inverts this relationship.

The mechanism is clear from the transpiled depths: $p=1$ produces a 52-gate-deep circuit, while $p=2$ requires 101 gates. Each additional layer roughly doubles the number of two-qubit CZ gates (48 to 96 to 156), and each CZ gate introduces ∼0.5–1% error. At $p=2$, the accumulated gate errors overwhelm the theoretical advantage of deeper parametrization.

The $p=3$ result (8.2% valid) is higher than $p=2$, which may seem contradictory. We attribute this to the specific parameter values found during optimization: the $p=3$ parameters happen to produce a circuit where certain layers partially cancel, effectively reducing the active circuit depth.

Figure 2. QAOA depth analysis on real hardware. (a) $p=1$ achieves 15.4% valid assignments vs. <1% for $p=2$. (b) Transpiled circuit depth scales linearly with QAOA layers.

Figure 2. QAOA depth analysis on real hardware. (a) $p=1$ achieves 15.4% valid assignments vs. <1% for $p=2$. (b) Transpiled circuit depth scales linearly with QAOA layers.

7.4. Cross-Backend Reproducibility

Figure 3 shows results for the same 12-qubit circuit across all three IBM backends. Valid rates are consistent within ±1.5% (fez: 0.9%, kingston: 0.8%, marrakesh: 1.1%), providing confidence that results are not artifacts of a single processor. Transpiled depths varied slightly (95–102) due to backend-specific qubit connectivity and transpilation routing.

7.5. High-Shot Experiment (Experiment C)

Doubling shots from 4000 to 8000 at $p=2$ did not meaningfully improve results (valid rate: 0.56% vs. 0.8%). This confirms that the bottleneck is gate noise, not sampling statistics—more shots from a noisy circuit do not produce better solutions.

7.6. Comparison Summary

Figure 4 summarizes all methods on the 12-qubit instance.

8. Discussion

8.1. The Shallow Circuit Advantage

Our most significant finding—that $p=1$ QAOA achieves 19× the valid rate of $p=2$ on real hardware—has direct practical implications. For near-term quantum optimization:

1.

Fewer layers, more shots: Rather than deepening the circuit, run many more shots at $p=1$ to better sample the solution space.

2.

Error mitigation at $p=1$: Apply post-processing techniques (zero-noise extrapolation, probabilistic error cancellation) to the shallow circuit where they are most effective.

3.

Warm-starting: Initialize $p=1$ parameters from classical solutions [13] rather than random restarts.

This finding aligns with recent theoretical work showing that QAOA at low depth can still capture the essential structure of combinatorial problems through quantum interference, even without deep parametrization [12].

8.2. Constraint Satisfaction on NISQ

The low valid assignment rates (0.2–15.4%) highlight a fundamental NISQ challenge: enforcing hard constraints through penalty terms in a noisy quantum circuit. Three mitigation strategies are available:

1.

Feasibility-first decoding: Post-process measurements by projecting each invalid bitstring to its nearest valid assignment (e.g., for each task, keep only the model with the highest measurement probability).

2.

Quantum-aware constraint mixers: Replace the standard $R_x$ mixer with a constraint-preserving mixer [14] that only explores the feasible subspace, ensuring every measurement is valid by construction.

3.

Hybrid approach: Use QAOA to generate a set of candidate solutions, then classically refine the top candidates to satisfy constraints.

8.3. When Will Quantum Be Practical for LCRP?

Based on our scaling data, we estimate the crossover requirements:

• Current: 6 qubits yields 37–43% valid on Heron (∼0.5% two-qubit gate error)
• Needed: 40+ tasks × 10+ models = 400+ qubits at <0.1% gate error
• Timeline: IBM targets 100,000+ qubit systems by 2033; error rates are improving ∼2× per generation
• Projected: Practical advantage for LCRP-scale problems: 2030–2035

8.4. Practical Implications

For production LLM routing systems, this work offers:

1.

Formal problem structure: LCRP as a QUBO enables any combinatorial solver—quantum annealing, QAOA, or classical SAT—not just heuristics.

2.

Solver-agnostic architecture: Production systems can implement a solver interface that swaps between greedy (today), SA (near-term), and QAOA (future).

3.

Quantum-ready encoding: The QUBO is directly executable on D-Wave quantum annealers or future fault-tolerant gate-based machines.

8.5. Limitations

1.

Small scale: Problem sizes up to 18 qubits are within classical solvability. The quantum advantage hypothesis requires testing at 100+ qubits.

2.

Static routing: We optimize batch assignment at a single time point, not dynamic real-time routing.

3.

Simplified quality model: A single quality score per model; real quality varies by task domain.

4.

No error mitigation: We report raw hardware results without applying quantum error mitigation techniques, which would likely improve valid rates.

5.

Pre-optimized parameters: Parameters were tuned on a noiseless simulator; noise-aware parameter optimization could yield better hardware results.

9. Conclusion and Future Work

We presented the first quantum computing formulation of the LLM Cascade Routing Problem—the task of optimally assigning AI agent tasks to language models across heterogeneous cost and quality tiers. Our QUBO encoding captures cost minimization, quality constraints, budget limits, and latency SLAs in a form directly executable by quantum optimization algorithms.

Experiments on real IBM Quantum hardware (156-qubit Heron processors) yielded three key findings: (1) shallow QAOA circuits ($p=1$) achieve 19× better constraint satisfaction than deeper circuits on noisy hardware; (2) valid assignment rates degrade steeply with problem size, quantifying the NISQ barrier; and (3) results are reproducible across three independent quantum processors.

The penalty calibration insight—that there exists a critical threshold below which QAOA ignores constraints—is applicable to any constrained QAOA formulation beyond LLM routing.

Future work includes: (1) execution on quantum annealers (D-Wave Advantage2) for comparison; (2) constraint-preserving mixers to guarantee valid assignments; (3) warm-starting QAOA from classical solutions; (4) dynamic routing with time-dependent arrivals; (5) noise-aware parameter optimization; and (6) scaling to 50+ qubits as hardware improves.