Improving Quantum Optimization Algorithms by Constraint Relaxation

1. Introduction

As a relatively new and rapidly evolving field of both science and technology, the full potential of quantum computing remains largely uncharted. Researchers are still exploring its practical usefulness across various domains. While significant advancements have been made, such as Shor’s algorithm [1] for factoring integers and Grover’s algorithm [2] for unstructured search problems, which demonstrate the theoretical advantages of quantum computing, its real-world applications are still being uncovered.

Among the different branches of quantum computing, one area has demonstrated significant practical potential: quantum annealers. Quantum annealing, a method specifically designed for solving optimization problems, has shown promising results and near-practical utility. Quantum annealers, such as those developed by D-Wave Systems, are being increasingly utilized for their ability to find approximate solutions to complex optimization problems more efficiently than any other quantum architecture paradigms [3]. However, the downside is that quantum annealers are not universal quantum computers. Despite the fact that a vast majority of combinatorial optimization problems can be represented as Quadratic Unconstrained Binary Optimization (QUBO) formulations, researchers continue to explore and improve other approaches. With simultaneous advancements in the quality of universal quantum computers, the true leader in quantum optimization has yet to be determined.

Most quantum optimization algorithms in the NISQ (Noisy Intermediate Scale Quantum) era involve representing the optimization problem as a Hamiltonian [4]. In physics, a Hamiltonian represents the energy of a system and governs its time evolution. In quantum computing, it often serves as a crucial component for determining measurable quantities and finding most favorable solutions by seeking the system’s ground state. Although some general guidelines for constructing such Hamiltonians have been established [5], there is no single, universal method for doing so. The construction of a Hamiltonian for a given problem can vary based on the emphasis placed on different aspects of the problem or by considering different quantum paradigms. For example, quantum annealers are designed to handle optimization problems in QUBO form, which constraints the Hamiltonian to contain entanglements of at most second degree, i.e. entangling at most two qubits with each other.

Another aspect of encoding the Hamiltonian is to introduce interpretability to the energy values. In its simplest form, the Hamiltonian can be monotonic, meaning that better solutions have lower energy than worse ones. However, the Hamiltonian can also be encoded to provide more detailed information. For instance, the energy value might represent the number of unsatisfied constraints, giving a clear indication of how far a solution is from being feasible. Additionally, each digit or component of the Hamiltonian’s energy value could correspond to different constraints or optimization criteria, allowing for a more granular analysis of the solution’s quality.

Choosing the encoding for a Hamiltonian should not be considered in isolation from the performance of the algorithm. Hamiltonians that encode more detailed information in their energy values tend to be more complex. This complexity often necessitates the algorithm to invest more time and computational effort to achieve convergence. Additionally, such Hamiltonians may require the construction of more intricate quantum circuits, which involve costly operations and are more susceptible to noise. This is particularly problematic for current quantum computers, which are limited by noise and error rates in the NISQ era. Therefore, a balance must be struck between the richness of the information provided by the Hamiltonian and the practical limitations of the quantum hardware.

In this paper, we will focus on a specific subset of gate-based quantum optimization algorithms, namely variational algorithms. We will explore an approach for constructing a Hamiltonian for the standard Quantum Approximate Optimization Algorithm (QAOA) [6]. Although we will primarily discuss the vanilla QAOA, our approach is also applicable to more recent and advanced versions of QAOA, potentially enhancing their performance in the same way as it benefits the standard QAOA.

2. Idea of Better Quantum Optimization

The QAOA is a gate-based hybrid classical-quantum algorithm inspired by the Adiabatic Theorem. The Adiabatic Theorem states that a quantum system initially in an eigenstate will remain in that eigenstate if the Hamiltonian governing the system changes sufficiently slowly [7]. The core idea of QAOA is to approximate this adiabatic process using a sequence of quantum gates, enabling the transition from an easy-to-prepare initial state to a state that approximates the solution to a given optimization problem. To implement this, QAOA uses a combination of two Hamiltonians: $H_C$, the problem Hamiltonian, and $H_M$, the mixing Hamiltonian. The evolution is discretized using Lie-Trotter product formula [8], resulting in a sequence of alternating unitary operations. The final state of the algorithm after p steps is given by:

$$|{\psi }_p(\overrightarrow{\gamma },\overrightarrow{\beta })\rangle =e^{-i{\beta }_p{H}_M}{e}^{-i{\gamma }_p{H}_C}\cdots e^{-i{\beta }_1{H}_M}{e}^{-i{\gamma }_1{H}_C}{|+\rangle }^{\otimes R}.$$

(1)

We can clearly see that the problem Hamiltonian serves a dual role in QAOA. Firstly, it acts as a quantum observable, measuring the energy of the system and providing feedback to adjust the variational parameters $\overrightarrow{\gamma }$ and $\overrightarrow{\beta }$. Secondly, it defines the structure of the quantum circuit, determining the gates and operations required for the algorithm. Because the Hamiltonian directly influences both the optimization process and the quantum circuit’s complexity, it is beneficial to simplify and reduce its complexity wherever possible. A simpler Hamiltonian can lead to shorter quantum circuits with less entanglement gates, which in turn might increase the chances of achieving good results and solving larger problem instances.

The structure of the Hamiltonian is also influenced by the chosen variable encoding of the problem. Often, a binary encoding is employed, where a qubit in the state |0〉 indicates that a particular option is not selected, and |1〉 indicates that it is selected. This encoding is straightforward for problems with binary variables. For problems with more than two choices, one-hot encoding is typically used. In one-hot encoding, a string of length m is constructed, where m corresponds to the number of possible choices. Each string is designed to have a Hamming weight of 1, meaning that only one qubit in the string is in the state |1〉 while all others are in the state |0〉. This ensures that only one option is chosen out of the m possible choices, providing a clear and unambiguous representation of the selection.

There are two main approaches for encoding the one-hot constraint. The first approach uses the QUBO formulation:

$$\begin{array}{cc}\hfill H_{\mathrm{QUBO}-\mathrm{onehot}}\left(x\right)& ={\left(\sum _{j=1}^m{H}_x\left(x_j\right)-1\right)}^2,\hfill \end{array}$$

(2)

where $x_j$ is further replaced using the Pauli-Z term:

$$H_x\left(x_j\right)={\displaystyle \frac{1}{2}}(I-Z_j),$$

(3)

and the Pauli matrices are defined as follows:

$$I=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right),Z=\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right).$$

The encoding from Equation 2 has the advantage of using at most ${\displaystyle \frac{n(n-1)}{2}}$ entanglements. However, if we need to sum together many such partial Hamiltonians as constraints, we cannot derive any meaningful interpretation from the energy value, as this encoding increases quadratically with the Hamming distance from the one-hot encoding.

If we want better interpretability, we can use entanglements with higher degrees, unavailable in e.g. D-Wave’s quantum annealears, with the following formula:

$$H_{\mathrm{full}-\mathrm{onehot}}=I-\sum _{j=1}^m\left(H_x\left(x_j\right)\prod _{\begin{array}{c}{j}^{\prime }=1\\ j^{\prime }\ne j\end{array}}^m{H}_{\mathrm{not}}\left(x_{j^{\prime }}\right)\right),$$

(4)

where

$$H_{\mathrm{not}}\left(x_{j^{\prime }}\right)={\displaystyle \frac{1}{2}}(I+Z_{j^{\prime }}).$$

(5)

Equation 4 evaluates to 1 if and only if the one-hot value is encoded correctly, and 0 otherwise. However, a significant disadvantage is that, after Hermitian evolution, this approach results in a quantum circuit with many entanglement gates of high degree, introducing substantial noise into the model.

In this paper, we observe that not all encodings known from classical problem definitions are as effective as possible for quantum computing. It might seem counterintuitive, but it turns out that by extending the search space, we actually increase the chances of measuring a feasible / optimal solution while simultaneously simplifying the Hamiltonian. For example, the one-hot constraint can, in some cases, be replaced by a relaxed “at least one” constraint, which ultimately serves the same purpose but significantly reduces circuit length and noise. We will show how to do that on a concrete example in Section 4.

3. Literature Review

Many researchers have investigated reducing noise and improving experimental results through circuit manipulations or truncations. For instance, using imperfect Hamiltonian representations for NISQ-era adiabatic quantum optimization has been shown to yield better results when employing specific techniques [9]. Additionally, several studies have examined the variable encoding procedures, noting that different encodings and embeddings significantly affect circuit performance [10,11,12].

A particularly interesting branch of research related to our approach involves specific variants of the QAOA algorithm [13]. One notable example is the ADAPT-QAOA algorithm, which iteratively selects operators to enhance the QAOA mixer Hamiltonians based on gradient descent algorithm outputs, thereby reducing the overall number of necessary entanglement gates [14]. Another algorithm, focusing on problem Hamiltonians, draws inspiration from classical neural networks by introducing a quantum dropout approach [15]. This method shows that selectively dropping clauses that define the problem Hamiltonian, while maintaining the cost function intact, can enhance QAOA performance.

Our work diverges from these approaches by being the first, to our knowledge, to design a QAOA problem Hamiltonian from the beginning to be simpler than its ideal counterpart. This simplification involves the relaxation of constraints within the problem Hamiltonian, aiming to retain the core structure and characteristics of the original problem while making it more amenable to efficient quantum optimization. By doing so, we strive to balance the complexity of the problem representation with the capabilities of current quantum hardware, which is often limited by noise and decoherence.

4. Exemplary Optimization Problem

In this paper, we will consider the tactical aircraft deconfliction problem; however, the following approach can also be used to deal with similar constraints in other problems, such as the Job Shop Scheduling Problem [16]. The tactical deconfliction problem involves predicting and resolving conflicts between aircraft in the airspace from 5 to 30 minutes ahead. A conflict is defined as a violation of the safety cylinder of an aircraft by another aircraft. Most approaches to solving this problem involve mixed integer linear/non-linear programming [17], but there are also some recent quantum approaches.

We base our work on the approach described by Pecyna et al. [18], who first formulated the quantum approach for this problem. For a detailed explanation of the approach, we refer the reader to the original paper. Following this approach, for each out of n aircraft we propose m maneuvers and define a set of $n\times m$ binary variables as follows:

$$X=\{x_{ij}:i=1,\cdots ,n,j=1,\cdots ,m,x_{ij}\in \{0,1\}\},$$

(6)

where the variable $x_{ij}$ taking the value 1 means that aircraft i is assigned maneuver j; it takes the value 0 otherwise. Each variable directly corresponds to a specific qubit in a quantum environment. From this set of variables, we form two types of constraints. The first constraint ensures there are no conflicts, while the second constraint ensures that an aircraft performs one and only one maneuver. The Hamiltonian representing the no-conflict constraint is originally written as:

$$H_1=\sum _{i,j,i^{\prime },j^{\prime }:\mathrm{CM}(i,j,i^{\prime },j^{\prime })=1}{H}_{\mathrm{and}}(x_{ij},x_{i^{\prime }{j}^{\prime }}).$$

(7)

where the CM matrix is defined as in [18], $H_{\mathrm{and}}(x_{ij},x_{i^{\prime }{j}^{\prime }})={\displaystyle \frac{1}{4}}I-{\displaystyle \frac{1}{4}}(Z_{ij}+Z_{i^{\prime }{j}^{\prime }}-Z_{ij}{Z}_{i^{\prime }{j}^{\prime }})$.

The Hamiltonian for the constraint that ensures an aircraft can perform one and only one maneuver is originally formulated as:

$$H_2=\sum _{i=1}^{n}I-\sum _{j=1}^m\left(H_x\left(x_{ij}\right)\prod _{\begin{array}{c}{j}^{\prime }=1\\ j^{\prime }\ne j\end{array}}^m{H}_{\mathrm{not}}\left(x_{i{j}^{\prime }}\right)\right),$$

(8)

where $H_{\mathrm{not}}\left(x_{i{j}^{\prime }}\right)={\displaystyle \frac{1}{2}}(I+Z_{i{j}^{\prime }})$ and $H_x\left(x_{ij}\right)={\displaystyle \frac{1}{2}}(I-Z_{ij})$.

If these two types of constraints are satisfied, it means that aircraft are assigned conflict-free trajectories and each aircraft performs only one maneuver. The deconfliction problem that is formulated this way does not have an optimization function so a solutions satisfying those two types of constraints is a correct, feasible solution to the problem.

In this work, we will focus on improving the second constraint. Note that due to the products in Equation 8, the number of summands containing Pauli-Z terms is $n{2}^m$, with most necessitating entanglements of degree higher than 2. This, in turn, can make it challenging to find the ground state of the final Hamiltonian and can make the quantum circuit prone to errors in noisy quantum environments.

Classical optimization of hard problems almost always benefits from reducing the search space. Setting up more constraints while simultaneously keeping the number of feasible solutions constant prevents classical algorithms from wasting their computing cycles on exploring unprofitable dead ends that don’t yield valuable solutions. In quantum computing, however, qubits can be put into superposition, allowing operations on many states simultaneously. This enables virtually cost-free computation on unfeasible solutions without adding any additional overhead. Considering this, we note that the one-hot constraint for the tactical aircraft deconfliction problem is superfluous when using quantum optimization algorithms, and removing it can lead to significant performance improvements.

Counterintuitively, let us replace the constraint that an aircraft can perform one and only one maneuver with the constraint that an aircraft must perform at least one maneuver. The no-conflict constraint remains intact. Feasible solutions would then include solutions with aircraft performing multiple maneuvers simultaneously, which is obviously impossible in real-life situations. This, however, does not concern us, because we can interpret the solution of an aircraft performing multiple maneuvers as if all those maneuvers were conflict-free. The solution would then be a subset of possible non-conflict maneuvers for the aircraft, with the specific maneuver selection postponed until the post-processing phase. Moreover, at this stage one can define an optimization criterion and select such solution, that optimizes the criterion.

As promising as it might sound, this approach does not help us much. The reason is that if we encoded a Hamiltonian for the at least one maneuver constraint, the number of Pauli-Z summands would be the same as for the one-hot constraint.

We can shift our perspective and identify the fundamental criteria that our constraints must meet:

• the state $0\dots 0$ must have a value of 1,
• each state with Hamming weight 1, representing the feasible solution, must yield a value of 0,
• all other possible states must have a non-negative value to ensure that no bitstrings have a lower Hamiltonian value than any correct solution.

These three requirements do not correspond to the one-hot or the at-least-one constraint. However, they are sufficient to form a sensible constraint that contains all essential and indelible requirements for a correct Hamiltonian for the tactical deconfliction problem.

There is one function that directly meets those criteria, which is the multi-variable NOT XOR function. For example, consider an aircraft with $m=5$ possible maneuvers. The Hamming weights for all $2^6=32$ bitstrings would range from 0 to 5. Moreover,

• bitstrings with Hamming weights of 0, 2, and 4 would yield a value of 1, fulfilling the first requirement and partially the third requirement.
• bitstrings with Hamming weights of 1, 3, and 5 would yield a value of 0, fulfilling the second requirement and completing the third requirement.

We present a more visual representation of the behavior of the NOT XOR function in Table 1. Subsequently, the Hamiltonian for the NOT XOR function can be encoded as follows:

$$H_{\mathrm{NOT}}\mathrm{XOR}=-\frac{1}{2}I+\frac{1}{2}{Z}_1{Z}_2\cdots Z_m.$$

(9)

Using the NOT XOR function, we see that there is only one Pauli-Z term with an entanglement of degree m. This represents a significant improvement, as the number of Pauli-Z terms directly corresponds to the number of entanglement gates needed to encode such Hamiltonians in a quantum circuit, as shown in Figure 1. For an aircraft with m possible maneuvers, the onehot encoding requires $\mathcal{O}\left(2^m\right)$ entanglements, the quadratic encoding requires $\mathcal{O}\left(n^2\right)$ entanglements, while the NOT XOR encoding requires only $\mathcal{O}\left(1\right)$ entanglements.

5. Computational Experiment

Having introduced the NOT XOR Hamiltonian we compare it with the standard one-hot encoding with many entanglements of high degree and the quadratic encoding which uses only second-degree entanglements. We benchmark those three approaches against artificially generated instances of $n=3$ aircraft with $m=5$ maneuvers. The difference between these instances lies in the number of potential conflicts between maneuvers, resulting in a varying number of feasible solutions to the problem. We choose to test instances with difficulties of 1, 3, 5, 10, 20, 30, 50, 100, and 124 solutions. The reason for choosing such a set of instances is that the first constraint for conflict avoidance (see Equation 7) already contains the quadratic Pauli-Z terms. Thus, with an increasing number of potential conflicts, the entanglement complexity of the constraint determining the number of maneuvers performed by the aircraft may lose its significance.

We have tested the instances in a noisy simulator environment, using the noise model derived from the ibm_torino quantum device. For the optimization of QAOA parameters, we used the Constrained Optimization BY Linear Approximation (COBYLA) [19] algorithm. We fixed 10 random initial points, optimized each instance starting from each of those random points, and averaged the results. The results are presented in Figure 2.

The figure answers two questions. Firstly, is the NOT XOR encoding better in terms of the number of feasible solutions we can find, i.e., does it make the optimization landscape simpler? Secondly, does the NOT XOR encoding introduce less noise to the circuit? One could argue that it would be unfair and biased to report that the NOT XOR solution gives better results when we treat all solutions that have no conflicts and where aircraft perform at least one maneuver as feasible. This is because optimizers with the one-hot and QUBO Hamiltonians as observables do not aim to increase the probability of measuring states where aircraft perform more than one maneuver. For this purpose, we also evaluate all these Hamiltonians against the one-hot evaluation strategy, which is theoretically not favorable from the perspective of an optimizer aiming to minimize the energy measured with the Hamiltonian that encodes the NOT XOR observable.

We report that in the considered instances, the NOT XOR Hamiltonian outperforms the standard one-hot Hamiltonian on both evaluation strategies. This means that using this kind of encoding is always favorable, as it both simplifies the optimization landscape and reduces potential noise. This also indicates that this approach would be preferred over the standard one-hot encoding even if the goal was to fully satisfy the one-hot encoding, i.e., it was not possible to choose one maneuver for an aircraft out of many at the postprocessing stage.

Moreover, for all instances, the NOT XOR Hamiltonian achieves better results than the QUBO Hamiltonian for the evaluation strategy where bitstrings of aircraft performing at least one maneuver are considered feasible. Additionally, for 7 out of 9 instances, the NOT XOR Hamiltonian outperforms the QUBO Hamiltonian when we consider bitstrings satisfying the one-hot encoding as feasible. The two instances where the QUBO encoding outperforms the NOT XOR encoding are the two easiest instances, i.e., instances with many possible solutions, which also means fewer conflicts. This result can be easily explained by noting that when there are few possible feasible solutions in an instance, most of them have a Hamming weight of 1. Conversely, when there are many feasible solutions, there are also many solutions having a Hamming weight greater than 1.

6. Conclusions and Future Work

In this paper, we have presented an alternative approach to formulating the problem Hamiltonian for the Quantum Approximate Optimization Algorithm that reduces the total number of necessary entanglements from exponential to linear. This formulation employs high-degree entanglements (higher than quadratic), resulting in significant performance benefits, such as fewer noise-induced errors and a higher probability of measuring the correct solution. Our solution outperforms the standard Hamiltonian formulation for the tactical deconfliction problem in all considered instances and surpasses the quadratic formulation known from quantum annealers in 7 out of 9 instances.

We see two evident directions for further research. Firstly, our experiments were conducted on a noisy simulator designed to closely resemble real quantum machines. However, there are quantum architectures, such as ion traps, that are particularly well-suited for performing high-degree entanglements in a single operation. It would be beneficial to repeat these experiments on real hardware that is optimized for such setups. Secondly, we have demonstrated only one use case of this Hamiltonian formulation. It would be valuable for the community to conduct in-depth research on various well-known optimization problems and explore how to reformulate existing encodings to enhance their efficiency. A key area of interest would be to propose a set of general rules for constructing custom Hamiltonians.