The Born Rule Without a Measurement Postulate

1. Introduction

A naive application of the core postulates of quantum theory leads to counterintuitive results such as Schrödinger’s cat states [1]. To avoid this, a seemingly arbitrary non-unitary measurement postulate is added, despite the fact that measurement devices are themselves physical objects subject to the laws of quantum mechanics. This postulate asserts that a measurement results in a single outcome even for a system in superposition, and that the various results occur with probabilities according to the Born rule. A complete quantum theory would explain measurement without resorting to either part of this postulate.

Progress has been made on the first part, the so-called measurement problem. Decoherence theory (see [2] for a review) shows that non-isolated superpositions naturally and quickly evolve into non-interfering components, which must evolve independently. If the superposition in question is a macroscopic observer modeled quantum mechanically, and which-way information escapes, which it must, then quantum collapse may be seen as simply an illusion in a unitary universe [3], analogous to the Coriolis effect or the Doppler effect. In this view, the measured system entangles with an observer through multiple layers of interaction, and the resulting correlated state of system and memory is diagonal among various results which evolve in such a way that an imagined collapse projection does not impact correlated observations. Among interpretations of quantum theory, this picture is closest to the so called many-worlds interpretation. The collapse effect and Schrödinger’s cat are simply natural consequences of the equations of the theory.

On the other hand, a satisfactory explanation of the Born rule has remained elusive. Specifically, why do observers believe in the Born probabilities when, in this picture at least, all measurement results actually occur. We will argue that this is also an illusion. We will show that an observer who believes collapse is real and who wishes to make approximate predictions will have no alternative measure to weigh the alternatives without being subject to a dutch book, a set of bets on the result of an experiment that generate a loss of money regardless of the outcome.

The paper is organized as follows: Section 2 discusses concepts of classical probability including dutch books. Section 3 introduces Gleason’s theorem and discusses what additional assumptions are needed to prove the Born rule. Section 4 reviews previous efforts. Section 5 shows that agents making a projective measurement on a pure state must expect the Born rule to avoid a dutch book. Section 6 considers generalized measurements. Section 7 analyzes what can be said about mixed states. Section 8 contains a few philosophical comments. We conclude in Section 9.

2. Classical Probability

Probability in classical physics is problematic because classical physics is deterministic. A statement about the final state of an experiment, such as “the coin will land on heads”, can be translated in principle to an equivalent statement about the initial conditions, such as “the coin was flipped in one of these ways...”. In practice, our physical knowledge of initial conditions is coarse grained so we can only specify a distribution of initial and thus final states. Furthermore, the final states can be highly sensitive to changes in initial conditions. Gambling houses design games using these two facts to prevent players from predicting outcomes. In this way, we can define concepts of chance, likelihood, etc. but these definitions are necessarily circular. Calculations always assume, at least implicitly, an a priori distribution.

It is useful to define conditions that ensure probability assignments are self-consistent. Specifically the measures of exclusive alternatives must be non-negative, they must add, and combined they must sum to 1. These conditions were first proposed by Ramsey [4]. He showed that any probability measure for a finite set of gambles that does not satisfy these conditions will be vulnerable to a dutch book, a set of wagers that will result in a profit for a bookmaker regardless of the actual events. These conditions were later independently found by Kolmogorov [5] for the infinite case, and they are known as the Kolmogorov axioms.

The concept of a dutch book can be understood as a game between an agent Alice and a bookmaker Bob. Bob is allowed to ask Alice what her expectations of probability are for any set of experiments. If he finds some discrepancy, he may propose a set of wagers. If each wager allows Alice to at least break even on average, Alice is obliged to accept the bets. If Bob can gain money regardless of the outcome of the experiments, the set of wagers is called a dutch book. If no dutch book is possible, Alice’s probability beliefs are called coherent, or self-consistent. We will assume the wagers are implemented as contracts that pay off depending on the result of the experiment. Each contract has a price which Alice must pay to implement. If the price is negative Bob will pay Alice. Each contract has a vector of payouts, one for each possible result of the experiment. If the payout is negative, Alice pays Bob if that result happens. The expectation value of the payout of each contract, according to Alice, is greater than or equal to the price. Reference [6] has examples of dutch books for violations of each of the Komolgorov axioms. Their examples are described in terms of “tickets”, which are easily converted to our contracts. For simplicity, if a dutch book is possible, our bookmaker will arrange that the sum of the payouts for all contracts is $\$0$ regardless of which experimental result occurs. He can do this because he has the freedom to adjust the cost of each contract and compensate by adjusting the payoff vector by the same amount. He performs the same adjustment on all contracts simultaneously to make the experimental result with maximum payout pay a total of $\$0$ across all contracts. He is still making money in all cases, so Alice must be paying a positive total amount to implement the contracts. He then offers a free contract that brings the total payout for each other result up to $\$0$.

It is important to understand that coherence does not imply correctness. Shown a fair coin, an agent may believe it will always come up heads. This belief is coherent and satisfies the Komolgorov axioms. There is no dutch book since even a large number of coin flips might come up heads each time. Bob must gain money no matter the outcome of the experiments.

Given that we lack a definition of probability, two main schools of thought on the interpretation of probability have arisen, which we will call the frequency interpretation and the Bayesian interpretation. The frequency interpretation is based on the assumption that the probabilities for an experiment can be explored by performing a large but finite number of trials, possibly implying the system has an inherent single-trial probability distribution or propensity. The Bayesian interpretation does not automatically assume inherent probabilities, but considers an agent’s statements of belief, ie. at what odds are they willing to gamble.

Using the frequency approach to define probabilities is inherently circular. For a large number of trials, the distribution of the mean result narrows, but it is still a distribution. Flipping a coin 100 times and getting 50 heads does not prove the coin is fair, or that it is probably fair, or even that it is close to fair. All it can prove is that its propensity is not 0 or 1. We can infer that the propensity is within a small range with some confidence, but that confidence is itself a probability.

The Bayesian approach sidesteps propensities altogether. Rather it is concerned with an agent’s expectations of probability and whether those expectations are coherent. There is nothing in this approach that guarantees an agent’s beliefs are correct. When put into practice, the Bayesian approach assumes a prior distribution, rendering this interpretation circular as well.

3. Gleason’s Theorem

Turning to the quantum case, Gleason’s Theorem [7] states that any probability-like measure (PLM) of subspaces of a Hilbert space of dimension $>2$ must be of the form

$$P\left({\Pi }_i\right)=tr\left(\rho {\Pi }_i\right),$$

(1)

where ${\Pi }_i$ is a projection operator, and $\rho$ is a density operator. “probability-like” means the measures of exclusive alternatives (an orthogonal set of projectors) are non-negative, they add, and, if a complete set, they sum to 1. This set of assumptions coincides with the Kolmogorov axioms. Gleason did not require the measures to sum to 1, and did not require the density operator to have unit trace, but we will assume those normalizations.

A few assumptions are needed to recover the Born rule from Gleason:

• We must identify the projection operators with testable propositions.
• The density operator in (1) must be the one which represents the state of the system.
• We must assert that experimental results are governed by probabilities, and thus a PLM.

We will refer to these assumptions by number.

For example, we might ask “what is the probability a system prepared in a pure state $\varphi$ will be found upon measurement to be in a pure state $\psi$?”. Inserting the projection operator $|\psi \rangle \langle \psi |$ for the target state and the density operator $|\varphi \rangle \langle \varphi |$ for the prepared state gives

$$\begin{array}{cc}\hfill P\left(\psi \right|\varphi )& =P_{{\rho }_{\varphi }}\left(|\psi \rangle \langle \psi |\right)\hfill \\ \hfill & =Tr\left(|\varphi \rangle \langle \varphi |\psi \rangle \langle \psi |\right)\hfill \\ \hfill & ={\left|\langle \psi |\varphi \rangle \right|}^2,\hfill \end{array}$$

(2)

which is the Born rule.

The identification in assumption 1 is simple for a projective, repeatable, or von Neumann measurement, as in the previous example. For such a measurement, the system is left in a state which is a projection of the original state, and a repeated measurement produces the same result. The phrase “the system is found to be in” should be familiar from elementary quantum mechanics textbooks.

Assumption 2 is commonly missed. It is needed because Gleason’s Theorem does not specify what density operator to use. Mathematically any density operator will do. For example, in a finite dimensional Hilbert space,

$$\rho =\widehat{I}/dim\left(\mathcal{H}\right)$$

(3)

says that a system is equally likely to be found in any state regardless of how the system is prepared. Alternatively, in a field theory, the vacuum density operator could be assigned to all states. Without an additional assumption, the Born rule is mathematically unprovable.

For pure states, an example of a sufficient additional assumption is that a state is certain to be found in its own state if that is one of the choices, and zero probability to be found in an orthogonal state. This is sometimes postulated and we will refer to it as the Exclusive Measurement Postulate or EMP. Working in a basis $\left\{|i\rangle \right\}$ where the prepared state is $\varphi =|0\rangle$ and writing $\rho$ for the unknown Gleason density operator associated with $\varphi$, we have

$$\begin{array}{cc}\hfill {\delta }_{i0}& =P\left(i\right|0)\hfill \\ \hfill & =Tr\left(\rho |i\rangle \langle i|\right)\hfill \\ \hfill & =\sum _{jkl}\langle j|k\rangle {\rho }_{kl}\langle l|i\rangle \langle i|j\rangle \hfill \\ \hfill & ={\rho }_{ii}.\hfill \end{array}$$

(4)

Then we have

$$\begin{array}{cccc}\hfill {\rho }_{00}& =1\hfill & \hfill & \\ \hfill {\rho }_{ii}& =0\hfill & \hfill i& \ne 0,\hfill \end{array}$$

(5)

and to ensure the operator is positive,

$$\begin{array}{cccc}\hfill {\rho }_{ij}& =0\hfill & \hfill i& \ne j.\hfill \end{array}$$

(6)

This is the usual density operator for the pure state $\varphi$.

Even if, after assumption 2, the PLM is unique, we still need assumption 3. Up to now, God seems to be playing dice [8] using the Born rule. At any time he may decide to abandon the dice and choose another way, such as selecting the most “likely” result each time. Furthermore, in a many-worlds multiverse, all results actually occur. Even if observers wrongly believe a collapse is occuring, it is not clear why they would believe one result occurs more often than another. They may instead view the unique Born values as a curious numerical artifact, similar to a conservation law.

4. Prior Efforts

There have been many attempts to prove the Born rule from other postulates. All have included one or more of the measurement postulates at least implicitly, and all include our assumption 3. A review from 2020 can be found in [9].

More recently, DeBrota et al. [6] have shown in the context of a QBist interpretation that a gambler (Alice) who wishes to avoid a dutch book must base her probabilities for quantum experiments on a set of probabilities associated with a density operator, echoing Gleason. Thier proof does not imply her probability assignments must be correct, but it does imply that they must be coherent. For example, suppose she believes the various results of any experiment are always equally likely. Then, using DeBrota’s notation, $p\left(i\right)=1/d^2$, $R\left(j\right|i)=1/d$, and the “QBist version of the Born rule” (their Equation 12) reads

$$\begin{array}{cc}\hfill q\left(j\right)& =\sum _{i=1}^{d^2}((d+1)p\left(i\right)-{\displaystyle \frac{1}{d}})R\left(j\right|i)\hfill \\ \hfill & ={\displaystyle \frac{1}{d}},\hfill \end{array}$$

(7)

and Alice feels vindicated. We consider this version of the Born rule to be incomplete. Our analysis will show that Alice’s expectations must be correct, not just coherent.

5. Pure States

If the starting state is pure, Bob firsts determines if Alice’s expectations can be expressed as a density operator. If they can, he determines if it accurately describes the correct pure state. If it does, the game is over. Alice is following the Born rule. If not, then let $p>0$ be Alice’s expected probability that the system will not be found in the starting state. Bob offers Alice a contract for $\$1$ that pays at least $\$1/p$ if the state is not found in the starting state. Alice is obligated to purchase this worthless contract. Then Bob constructs an experiment to perform the test.

At this point, we need to fulfill assumption 1. We are considering only projective measurements, so we will make assumption A: for any set of orthogonal projectors, there exists an apparatus that implements a projective measurement, up to but not including the collapse projection. Such observations are possible in principle, and in an age where, for example, arbitrary unitary operations can be performed on qbits, this assumption is not unreasonable when projective measurements are possible.

There is an additional implicit assumption B: Alice agrees that the apparatus works the way we say it does. Without this assumption it not clear what the Born rule means to Alice. We are trying to prove the Born rule without a measurement postulate, so we can assume Alice agrees with the other postulates of quantum theory. She also believes in collapse, and a version of the Born rule distorted in some way.

The apparatus implements a unitary evolution that entangles the system with a probe. Following reference [3], the entangled resulting state is

$${\rho }_p=\sum _{a{a}^{\prime }}{A}_a{\rho }_s{A}_{a^{\prime }}^{†}\otimes {|a\rangle }_p{\langle a^{\prime }|}_p,$$

(8)

where ${\rho }_s$ is the prepared state of the system, $A_a$ is a Measurement operator [10], and ${|a\rangle }_p$ is a probe state. In the case of a projective measurement, $A_a={\Pi }_a$, a hermitian projection operator.

Alice (or perhaps a referee) proceeds with a projective observation of the entangled state, again without the collapse projection. The result is

$${\rho }_m=\sum _{a{a}^{\prime }}{\Pi }_a{\rho }_s{\Pi }_{a^{\prime }}\otimes {|a\rangle }_p{\langle a^{\prime }|}_p\otimes {|a\rangle }_m{\langle a^{\prime }|}_m,$$

(9)

where ${|a\rangle }_m$ represents an observer’s memory state, and we have ignored many layers of entangled apparatus states. They do not affect the discussion. We are implicitly making assumption C: observers are physical objects governed by quantum theory. If this assumption is philosophically challenging to the reader, we can require the referee to be an AI.

In reference [3], the probe is discarded as an example demonstrating escape of information. In our case we can mandate discarding the system and the probe. For example they can be released to the environment as part of the procedure. Decoupling the Hilbert space of the system and probe from the observer is represented by tracing over the system and probe degrees of freedom. The usual justification of the trace procedure uses the Born rule implicitly, but it can also be justified by asserting the proper unitary evolution of the decoupled Hilbert space [11]. The resulting memory state is

$$\begin{array}{cc}\hfill {\rho }_m& =\sum _a{B}_a{|a\rangle }_m{\langle a^{\prime }|}_m\hfill \\ \hfill B_a& =Tr\left({\rho }_s{\Pi }_a\right),\hfill \end{array}$$

(10)

where $B_a$ is the Born value for result a. In our picture this number does not automatically represent a probability. We will revisit the interpretation of mixed states later. Note that in practice, the density matrix for the memory state of a macroscopic observer such as a human would have a large block of values for each result, but this does not affect the discussion.

The collapse projection could be applied at this point and the resulting state would be Alice having seen a single result. Even if the collapse is not applied, Alice in each “branch” has no way to know about the other branches and might believe collapse has occured. Conventionally, the collapse projection is applied before the probe is entangled with the observer. The result is the same either way.

In the case at hand (Alice’s expectations are based on the wrong density operator), Bob will employ a set of two Measurement operators. One coincides with the prepared density operator ${\rho }_s$. The other is its orthogonal compliment. The Born value for the first is 1 and there is only one term in the memory state, even though we have not applied a collapse projection.

At this point we consider assumption 2. Instead of an EMP, we simply assert that an observer in an exclusive memory state is defined to be found in that state. For all intents and purposes she acts like that state. For example, Alice knows her contract is worthless and she has lost $\$1$.

In the remaining case, Alice’s probabilities cannot be derived from a density operator. Then by Gleason’s theorem there exists a set of orthogonal projectors in which her probabilities do not satisfy the Komolgorov axioms. Bob proposes the bets, and if Alice does not capitulate, he constructs an apparatus to produce an exclusive memory state from these projectors. In this case there is an additional step. After the interaction with the probe, unitary operations should be performed to compute the final sum of the value of the contracts. The states used for the computation are also released to the enviroment with the system and probe. Since our bookmaker arranged for the value of the contracts to sum to zero in all cases, Alice will know the set of contracts is worthless and she lost the net amount she paid for them. For pure states at least, if Alice does not use the correct density operator, she is vulnerable to a dutch book.

Note that we have not made assumption 3. In fact without a measurement postulate there is no such thing as probability. All possible results actually occur in the final state.

6. Generalized Measurements

In practice most measurements are not projective. The system may change in a way which is non-recoverable. The propositions of assumption 1 are that the probe, rather than the system, will be left in one of it’s distinguishable states.

Attempting the above proof is problematic here. If the prepared state is pure, the entangled state of system and probe will be pure as well, but we are performing a generalized measurement because a projective measurement of the system is not practical. The measurement operators acting on the probe are

$${\overline{\Pi }}_a={\mathbb{I}}_s\otimes {\Pi }_a,$$

(11)

where ${\mathbb{I}}_s$ is the identity in the system Hilbert space. These are only a finite subset of the full space of projectors on the tensor product space. Furthermore, the “a” states are specifically designed to represent the desired measurement results, and not the arbitrary states that the proof requires.

Suppose all “a” states are actually physically possible (they have non-zero Born values) and Bob asks Alice what are her expected probabilities. If her answers sum to 1, he knows there is in general no experiment he can actually perform to implement a dutch book. Instead, he could ask Alice to give her probabilities for experiments that can be performed in principle. Specifically, projective measurements are always possible in principle for any set of projectors, even on the tensor product Hilbert space. Bob may inform Alice that he will delay the experiment until new technology is available. Alice, having been stung by the first set of experiments, might be convinced to use the correct density operator (8) to compute her probabilities, and thus answer with the Born result

$$p_a=Tr\left(A_a^{†}{\rho }_s{A}_a\right).$$

(12)

7. Mixed States

Our proof does not work for mixed states. Consider a system represented by a diagonal density matrix whose first two Born values (diagonal elements) are p and $1-p$. By now, Alice has learned that her expectations must derive from a density operator. If her density matrix has any components outside the $2x2$ upper left submatrix in the same basis, she is vulnerable to a dutch book. Just as in the pure case, she expects a non-zero probability for a result that will never happen.

But if the support of her density operator is the same subspace (or less) as the support of the system state density operator, then no dutch book is possible.

Proof: For Alice to accept contract i, it must satisfy

$$C_c\le \sum _{i=1}^N{p}_i{P}_{ic},$$

(13)

where $C_c$ is the cost of contract c, N is the number of possible experimental results, $p_i$ is her expected probability for experimental result i, and $P_{ic}$ is the payout for contract c if result i occurs. Summing over all contracts we have

$$\begin{array}{cc}\hfill \overline{C}& \le \sum _{i=1}^N{p}_i{\overline{P}}_i\hfill \\ \hfill & \le \sum _{i=1}^N{p}_i{\overline{P}}_{max},\hfill \end{array}$$

(14)

where ${\overline{P}}_{max}$ is the best payoff for Alice among the various experimental results. If Alice’s expectations are coherent, $\Sigma p_i=1$ and ${\overline{P}}_{max}$ is at least as much as the cost.

There remains a loophole that we have exploited previously. Bob may assign a large payout for a result that cannot physically occur. Thus it remains to prove that Alice will not assign a positive probability to such a result. Results associated with projectors with support outside our 2D subspace would be rejected by Alice. Furthermore, any projector in the 2D subspace will be associated with a non-zero Born value in (10) for one or both components of ${\rho }_s$ so that result can physically occur. This completes the proof. More generally Alice can evade a dutch book if the system density operator has rank $R>1$, ie. a mixed state.

This is all a little abstract, but it may be understood in terms of a coin flip. Alice assigns a probability $p_a$ to heads and a probability $1-p_a$ to tails. No dutch book is possible. If the “true” probability of heads is less than $p_a$, Bob can apply a high payout to this result and make money on average, but cannot guarantee making money because heads might occur many times despite its low probability. Furthermore, in the dutch book game, Alice could be allowed to choose her result (from physically possible ones), so she is free to assign probability 1 to one result and reject any wager that loses money in that one case.

Bob only cares that he makes money in all cases. He may consider the p parameter of the quantum state to be meaningless. Instead the only meaningful part of the state to him is the subspace in which the density operator has support. It should be possible to formulate quantum mechanics (without collapse or probability) using this state representation.

Alice disagrees. Probability is meaningful to her, and it is convenient to identify the p parameter as an a priori probability. Then she can use (10) or (12) to calculate final probabilities. This is a perfect analogy with the classical circular definitions, and indeed the density operator can be used to model a superposition/mixture/distribution of classical states. It can also be used to model an ensemble.

8. Philosophy

This picture is very counterintuitive. Even a classical experiment such as flipping a coin 25 times is subject to quantum fluctuations, and there will be a “branch” where the coin land heads every time. In that branch, the agent, with her expectations of probability, will be very surprised. Nature in our picture assigns a tiny Born value to that branch of the superposition, but makes no judgement about “likelihood” or “frequency”. But we can say this: extremely unlikely events do occur, and when they do we are surprised. In this sense the predictions of quantum theory are accurate.

Furthermore, there are branches where every event in history had a chaotic result. It would be impossible to learn Physics in that branch, and the inhabitants would not be able to have this discussion. Anthropic reasoning may help exclude that case.

9. Conclusions

Despite considerable effort, no one has previously succeeded in proving the complete Born rule from other postulates. The complete Born rule requires probabilities to be correct and not just coherent.

Probability is an ill defined concept in a determistic universe. The classical universe is deterministic, and without a measurement postulate, a quantum universe is as well. It follows that macroscopic superpositions must exist, and given that all possible experimental results actually occur, there is no intrinsic concept of probability. But an agent who believes collapse occurs and wishes to make approximate predictions will have no alternative measure to weigh the alternatives without subjecting herself to a dutch book. Agents may deviate from the Born rule for generalized measurements at their own risk.

Our proof is complete for pure states and projective measurements without resorting to any measurement postulate. A similar proof is impossible in the general case of mixed states. Nevertheless, with the usual convention for diagonal density matrix elements, probabilities for mixed states have the same validity and issues as they do in classical physics. With such a convention, Gleason’s version of the Born rule (1) applies to all experiments, both quantum and classical.

In this picture, the Born rule is not a behavior of the universe. It is a rule of thumb used by observers who wish to make sense of the apparent randomness that arises from the collapse effect.