A New Logic, A New Information Measure, and A New Information-Based Approach to Interpreting Quantum Mechanics

1. Introduction: Where Does QM Math Come from?

The mathematics of quantum mechanics (QM) is quite distinctive and different from the mathematics of classical physics. Where does that distinctive math of QM come from? The purpose of this paper is to expound a new way to interpret quantum mechanics (QM), or, to be more precise, to interpret the mathematics (not the physics) of QM.

One research program in the philosophy of quantum mechanics is “reconstruction” [30] which aims at new derivations of QM for both the new and distinctive math of QM and the physics usually obtained by quantization. The hope is that new and particularly perspicuous axioms may give hints about the ontology of QM. In our approach, only the math of QM is reconstructed as the Hilbert-space version of the math of partitions–and that alone is sufficient to see the type of objectively indefinite ontology described by that math. Quantization then shows the changes in classical physics to fit that new math frameworks, e.g., the De Broglie and Einstein relations.1 Our view is that thus treating the math and physics by those separate ways is “cutting at the joints” as opposed to the usual program of trying to reconstruct the math and physics of QM together.

In other approaches to the philosophy of QM, the modus operandi seems to be that the standard theory needs to be changed in order to be interpreted, e.g., Bohmian mechanics or spontaneous localization. The approach taken here is to provide a realistic interpretation of the standard non-relativistic finite-dimensional von Neumann/Dirac theory.

2. Materials and Methods

The key mathematical, indeed logical, concept is the notion of a partition on a set–or equivalently, the notion of an equivalence relation. The basic thesis is that the math of QM is the Hilbert space version of the math of partitions. Partitions are the basic logical concept to describe

• distinctions versus indistinctions,
• definiteness versus indefiniteness, and
• distinguishability versus indistinguishability.

The new logical information measure is just the quantitative version of partitions.

The key non-classical notion of state in QM is that of superposition–with entanglement being a particularly unintuitive special case.

Thus, superposition, with the attendant riddles of entanglement and reduction, remains the central and generic interpretative problem of quantum theory. [14], (27)

The result of the thesis is to give a partitional explication of superposition in terms of objective indefiniteness between definite alternatives–so that this approach to QM could be called the Objective Indefiniteness Interpretation of QM.2

From these two basic ideas alone – indefiniteness and the superposition principle – it should be clear already that quantum mechanics conflicts sharply with common sense. If the quantum state of a system is a complete description of the system, then a quantity that has an indefinite value in that quantum state is objectively indefinite; its value is not merely unknown by the scientist who seeks to describe the system. Furthermore, since the outcome of a measurement of an objectively indefinite quantity is not determined by the quantum state, and yet the quantum state is the complete bearer of information about the system, the outcome is strictly a matter of objective chance – not just a matter of chance in the sense of unpredictability by the scientist. Finally, the probability of each possible outcome of the measurement is an objective probability. Classical physics did not conflict with common sense in these fundamental ways. [55], (47)

Abner Shimony suggested calling this interpretation of the math or “formalism of quantum mechanics” as the Literal Interpretation.

These statements ... may collectively be called “the Literal Interpretation” of quantum mechanics. This is the interpretation resulting from taking the formalism of quantum mechanics literally, as giving a representation of physical properties themselves, rather than of human knowledge of them, and by taking this representation to be complete. [57], (6-7)

3. Results

3.1. The Lattice of Partitions

A partition $\pi$ on a set $U=\{u_1,...,u_n\}$ is a set of nonempty subsets or blocks $\pi =\{B_1,...,B_m\}$ that are mutually disjoint and jointly exhaustive (their union is U).3 An equivalent definition, that prefigures the Hilbert space notion of a “direct-sum decomposition” of the space in terms of the eigenspaces of a Hermitian operator, is a set of nonempty subsets $\pi =\{B_1,...,B_m\}$ such that every non-empty subset $S\subseteq U$ can be uniquely represented as the union of a set of nonempty subsets of the $B_j$–in particular $S=\cup \{S\cap B_j\ne \varnothing :j=1,...,m\}$.

As the mathematical tool to describe distinctions versus indistinctions, a distinction or dit of $\pi$ is an ordered pair of elements of U in different blocks of the partition, and the ditset $dit\left(\pi \right)$ is the set of all the distinctions of $\pi$ (also called an “apartness relation”). An indistinction or indit of $\pi$ is an ordered pair of elements in the same block of the partition, and the indit set $indit\left(\pi \right)={\cup }_{j=1}^m{B}_j\times B_j$ is the set of all indits of $\pi$–which is the equivalence relation associated with $\pi$ whose equivalence classes are the blocks of $\pi$.

The partial order on partition is usually defined as $\sigma \precsim \pi$ (where $\sigma =\{C_1,...,C_{m^{\prime }}\}$) if for every $B_j\in \pi$, there is a $C_{j^{\prime }}\in \sigma$ such that $B_j\subseteq C_{j^{\prime }}$, but it is easier to just define it by $\sigma \precsim \pi$ if $dit\left(\sigma \right)\subseteq dit\left(\pi \right)$. The join (least upper bound) and meet (greatest lower bound) operations on partitions on U form the partition lattice $\Pi \left(U\right)$. The most important operation for our purposes is the join operation where the join $\pi \vee \sigma$ is the partition on U whose blocks are the nonempty subsets $B_j\cap C_{j^{\prime }}$ for $j=1,...,m$ and $j^{\prime }=1,...,m^{\prime }$. It could also be defined using ditsets since: $dit(\pi \vee \sigma )=dit\left(\pi \right)\cup dit\left(\sigma \right)$.

The meet $\pi \wedge \sigma$ of $\pi$ and $\sigma$ can be defined as the partition determined by the intersection of all the equivalence relations containing the equivalence relations corresponding to $\pi$ and $\sigma$.

The partition lattice $\Pi \left(U\right)$ also has a top and bottom. The top is the discrete partition ${\mathbf{1}}_U=\{\left\{u_1\right\},...,\left\{u_n\right\}\}$ with only singleton blocks which makes all possible distinctions, i.e., $dit\left({\mathbf{1}}_U\right)=U\times U-\Delta$ (where $\Delta$ is the diagonal of self-pairs $(u_i,u_i)$). The bottom is the indiscrete partition (or “The Blob”4) ${\mathbf{0}}_U=\left\{U\right\}$ with only one block U and it makes no distinctions so $dit\left({\mathbf{0}}_U\right)=\varnothing$ and $indit\left({\mathbf{0}}_U\right)=U\times U$.

In terms of the old notions of matter or substance and form, the bottom of the partition lattice ${\mathbf{0}}_U$ has all the substance but no form (no dits). The quantum notion of distinction-making change is prefigured by moving up the partition lattice as the substance is increasingly in-formed by distinctions until finally reaching the classical state ${\mathbf{1}}_U$. As a metaphor for the Big Bang, in the beginning there was all the substance (energy) but with no distinctions, i.e., “perfect symmetry” [49], and then distinctions were introduced by symmetry-breaking to create (more definite) “its from dits.” That partition notion of becoming is particularly important for our purposes since it prefigures the notion of quantum (projective) measurement (von Neumann’s Type I processes) as well as quantum jumps.

The other logical notion of becoming or creating is illustrated by moving up the Boolean lattice of subsets. In the beginning, there was no substance (empty set) and then becoming takes place by the creation (ex nihilo) of “its” that are fully formed as illustrated in Figure 1.

Metaphors might help to conceptualize the difference between the classical notion of change and quantum notion of becoming. In a police book of mug shots, the faces are fully formed and a metaphor for classical change is flipping through the fully definite or determinate mug shots. The quantum notion of becoming is like the police artist’s sketch pad where the indefinite outline of a face becomes more and more definite. Or a white (superpositions of all colors) painter’s canvas becomes a painting as white superposition of colors “collapses” to take on definite colors. Since there are layers or levels of increasing definiteness as one moves up the lattice of partitions, another metaphor might be 3D printing where the printed object takes shape as the printer moves up to higher layers.

The mathematics for this transition from indefinite to more definite is the refining of partitions. The logical operation on partitions that results in a refinement is the join of partitions (i.e., the intersection of equivalence relations) which provides a skeletal model of quantum projective measurement. The mathematics of QM is the extension of that partition math to complex Hilbert space.

3.2. The Logic of Partitions

The partition join and meet operations were known in the nineteenth century (e.g., Dedekind and Schröder). Ordinary logic is based on the Boolean logic of subsets of a universe set U; propositional logic is the special case of a one element universe U. In subset logic, the formulas, in general, stand for subsets of some universe set U. And subsets (or generally subobjects or `parts’) are category-theoretically dual to partitions (or generally quotient objects). “The dual notion (obtained by reversing the arrows) of `part’ is the notion of partition.” [42], (85). Hence one would naturally expect there to be a dual logic of partitions, where the formulas stand for partitions on some universe set U. But that would require at least the operation of implication on partitions (corresponding to the Boolean conditional $S\supset T$), but no new operations on partitions were defined in the twentieth century. As acknowledged in a 2001 volume commemorating Gian-Carlo Rota: “the only operations on the family of equivalence relations fully studied, understood and deployed are the binary join ∨ and meet ∧ operations.” [10], (445) Only in the current century was the implication operation $\sigma \Rightarrow \pi$ on partitions (which turns the partition lattice $\Pi \left(U\right)$ into an algebra) defined along with general algorithms to turn subset logical operations into partition logical operations. The resulting logic of partitions cemented the notion of a partition as not just a mathematical concept of combinatorics but a logical concept. ([18,23])

There is a parallel development of subset logic and partition logic based on the dual connection between the elements or “its” of a subset and the distinctions or “dits” of a partition–which is summarized in Table 1. At the mathematical level, the dual lattices or logics are equally fundamental.

In subset logic, a formula is valid if for any U ( $\left|U\right|\ge 1$) and any subsets of U substituted for the atomic variables, the formula evaluates by the logical subset operations to the top U. Similarly in partition logic, a formula is valid if for any U ($\left|U\right|\ge 2$) and any partitions on U substituted for the atomic variables, the formula evaluates by the logical partition operations to the top ${\mathbf{1}}_U$. The partition validities are a proper subset of the Boolean subset validities but are neither contained in nor contain the valid formulas of intuitionistic logic (Heyting algebras).

3.3. Logical Information Theory: Logical Entropy

In his writings (and MIT lectures), Gian-Carlo Rota further developed the parallelism between subsets and partitions by considering their quantitative versions: “The lattice of partitions plays for information the role that the Boolean algebra of subsets plays for size or probability.” [41], (30) Since the normalized size of a subset $Pr\left(S\right)=\frac{\left|S\right|}{\left|U\right|}$ gives its Boole-Laplace finite probability, so the “size” of a partition would play a similar role for information:

$\frac{\mathrm{Information}}{\mathrm{Partitions}}\approx \frac{\mathrm{Probability}}{\mathrm{Subsets}}$.

Since “Probability is a measure on the Boolean algebra of events” that gives quantitatively the “intuitive idea of the size of a set”, we may ask by “analogy” for some measure “which will capture some property that will turn out to be for [partitions] what size is to a set.” [52], (67) The duality (represented in Table 1) tells us that it is the number of dits in a partition that gives its size (maximum at the top and minimum at the bottom of the partition lattice) that is parallel to the number of `its’ in a subset (maximum at the top and minimum at the bottom of the subset lattice).

That is the reasoning that motivates the definition of the logical entropy of a partition as the normalized size of its ditset (equiprobable points):

$h\left(\pi \right)=\frac{|dit(\pi \left)\right|}{|U\times U|}=\frac{|U\times U|-|indit(\pi \left)\right|}{|U\times U|}=1-\frac{{\cup }_j|B_j\times B_j|}{|U\times U|}=1-{\sum }_j{\left(\frac{|B_j|}{\left|U\right|}\right)}^2$.

In general, if the points of $U=\{u_1,...,u_n\}$ have general positive probabilities $p=\{p_1,...,p_n\}$, then $Pr\left(B_j\right)={\sum }_{u_i\in B_j}{p}_i$, so that:

$h\left(\pi \right)=1-{\sum }_{j=1}^{m}Pr{\left(B_j\right)}^2={\sum }_{j=1}^{m}Pr\left(B_j\right)(1-Pr\left(B_j\right))={\sum }_{j\ne j^{\prime }}Pr\left(B_j\right)Pr\left(B_{j^{\prime }}\right)$.

The parallelism carries through to the interpretation: $Pr\left(S\right)={\sum }_{u_i\in S}{p}_i$ is the one-draw probability of getting an `it’ of S and $h\left(\pi \right)$ is the two-draw probability of getting a `dit’ of $\pi$.

The logical entropy $h\left(\pi \right)$ is a (probability) measure in the sense of measure theory, i.e., $h\left(\pi \right)$ is the product measure $p\times p$ on the ditset $dit\left(\pi \right)\subseteq U\times U$. As a measure, the compound notions of joint, conditional, and mutual logical entropy satisfy the usual Venn diagram relationships.

The well-known Shannon entropy $H\left(\pi \right)={\sum }_{j=1}^{m}Pr\left(B_j\right){log}_2\left(\frac{1}{Pr\left(B_j\right)}\right)$ can also be interpreted in terms of partitions; it is the minimum average number of binary partitions (bits) it takes to distinguish the blocks of $\pi$. The Shannon entropy is not a measure in the sense of measure theory but the compound notions of joint, conditional, and mutual Shannon entropy were defined so that they satisfy similar Venn-like pictures (but with no underlying set). That is possible because there is a non-linear but monotonic dit-to-bit transform, i.e., $1-Pr\left(B_j\right)⇝{log}_2\left(\frac{1}{Pr\left(B_j\right)}\right)$, that takes $h\left(\pi \right)={\sum }_{j}Pr\left(B_j\right)(1-Pr\left(B_j\right))$ to $H\left(\pi \right)={\sum }_{j}Pr\left(B_j\right){log}_2\left(\frac{1}{Pr\left(B_j\right)}\right)$ and which preserves Venn diagrams [22].

If a partition $\pi$ is the inverse-image partition $\pi ={\left\{f^{-1}\left(r\right)\right\}}_{r\in f\left(U\right)}$ of a numerical attribute $f:U\to \mathbb{R}$, then $h\left(\pi \right)$ is the two-draw probability of getting different f-values. The notion of logical entropy generalizes naturally to the notion of quantum logical entropy ([22]; [60]) where it gives the probability in two independent measurements of the same state by the same observable that the result gives different eigenvalues.

3.4. Partitions as Skeletonized Quantum States

There is a very simple way to skeletonize a quantum state to arrive at the corresponding set notion. Consider $U=\{a,b,c,d\}$ as both a set of distinct points and also as a orthonormal basis for a 4-dimensional Hilbert space. Then a superposition state vector of the form (say) $\alpha |a\rangle +\beta |b\rangle$ is skeletonized by deleting the complex scalars $\alpha$ and $\beta$, the Dirac kets, and the addition operation to yield just the support, the set $\{a,b\}$ as a block in a partition, i.e., vector in QM math ⇝ support in partition math. A superposition of states skeletonizes to elements in the same block of a partition, i.e., superposition = indistinction (which is non-trivial in the sense of not being a diagonal pair like $(b,b)$). Figure 2 is the lattice of partitions starting with a pure state (indiscrete partition), non-classical mixed states (i.e., mixed states containing at least one superposition), and the classical mixed state (where we have used the partition shorthand of representing the partition $\left\{\right\{a,b\},\{c,d\left\}\right\}$ as $\{ab,cd\}$ i.e., the inner-most curly brackets are eliminated in favor of juxtaposition). The interpretation is that $a,b,c,d$ are distinct definite- or eigen-states of a particle according to some observable, where “particle” does not mean a classical (or Bohmian) particle but an entity that can have different levels of objective indefiniteness (superpositions of the eigenstates such as $a,b,c,$ or d) or the definiteness of an eigenstate.5

Figure 2 is the Hasse diagram of the partition lattice on a four element set which means that each line between two partitions represents the partial order of refinement with no intermediate partitions. Then the change from a partition to a more refined one is a “jump.” The skeletonized mixed states in the lattice of partitions of Figure 2 are all those that might be obtained by a “projective measurement” of the pure state $\left\{abcd\right\}$, i.e., by the partition join version of the Lüders mixture operation. All the lines or links in Figure 2 go from an indefinite state upward to a more definite state that is part of a join operation. A projective measurement turns a pure state into a mixed state one part of which, e.g., an eigenstate in a maximal measurement, is realized according to its probability.

The set level precursor of such a quantum jump from a superposition to an eigenstate is the “choice function” [31], (60) which assigns to each nonempty subset (like a block in a partition) an element of the subset. That determination of an element is non-deterministic except in the special case of a singleton set which is the precursor of the quantum measurement when the outcome has probability one, namely when the state being measured is a single eigenstate of the observable being measured as indicated in Table 2.

The indeterminism of the jump in the set level choice function shows that those indeterminate aspects of QM math are even prior to the introduction of probabilities.

The top discrete partition ${\mathbf{1}}_U$ is the skeletal version of a classical mixed state like randomly choosing a leaf in a four-leaf clover or randomly choosing a `letter’ in the four-letter genetic code $U,C,A,G$. One criterion of classical reality was the idea that it was fully definite or definite all-the-way-down as in Leibniz’s Principle of Identity of Indistinguishables (PII) [5], (Fourth letter, 22) or Kant’s Principle of Complete Determination (omnimoda determinatio).

Every thing, however, as to its possibility, further stands under the principle of thoroughgoing determination; according to which, among all possible predicates of things, insofar as they are compared with their opposites, one must apply to it. [37], (B600)

Thus two distinct things must have some predicate to distinguish them or if there is no way to distinguish them, then they are the same thing. This principle of classicality characterizes the `classical’ state ${\mathbf{1}}_U$:

For any $u,u^{\prime }\in U$, if $(u,u^{\prime })\in indit\left({\mathbf{1}}_U\right)$, then $u=u^{\prime }$

Partition logical version of Principle of Identity of Indistinguishables.

Any non-classical state in the skeletal representation is a partition $\pi$ with a non-singleton `superposition’ block, e.g., $\{a,b\}$ so $(a,b)\in indit\left(\pi \right)$ but $a\ne b$.

In contrast, quantum reality is not definite all-the-way-down. A state could even be maximally specified and still leave room for many particles (bosons) to be in that state.

In quantum mechanics, however, identical particles are truly indistinguishable. This is because we cannot specify more than a complete set of commuting ob servables for each of the particles; in particular, we cannot label the particle by coloring it blue. [53], (446)

In addition to PII, Leibniz had other metaphysical principles characteristic of the classical notion of reality. His Principle of Continuity was expressed by “Natura non facit saltus” (Nature does not make jumps) [43], (Bk. IV, chap. xvi) and his Principle of Sufficient Reason was expressed as the statement “that nothing happens without a reason why it should be so rather than otherwise” [5], (Second letter, 7). All these classical principles are violated in the quantum world; PII is violated by superpositions of bosons, Continuity is violated by the quantum jumps, and Sufficient Reason is violated by the objective probabilities of QM.

This skeletal representation of quantum states is summarized in Table 3.

4. Superposition as Indefiniteness in the Quantum `Underworld’

The biggest `enemy’ to understanding QM is the wave imagery, not to mention the name “wave mechanics.” That imagery interprets superposition ontologically as the sum of two waves to give another wave as in Figure 3.

The mathematics of waves is always present since any vector in a space over the complex numbers $\mathbb{C}$ automatically has wave imagery in the polar representation as having an amplitude and phase. But the physical interpretation of waves is a misleading artefact or by-product of the use of complex numbers in the math of QM. The complex numbers are used because (among other reasons) they are algebraically complete so that the observable operators will have a complete set of eigenvectors [64], (67, fn. 7),6 It is a fact of the mathematics that the addition of vectors in a vector space over $\mathbb{C}$ can always be represented as the superposition of waves with interference effects.

The wave formalism offers a convenient mathematical representation of this latency, for not only can the mathematics of wave effects, like interference and diffraction, be expressed in terms of the addition of vectors (that is, their linear superposition; see [26], (Chap. 29.5)), but the converse, also holds. [36], (303)

On the partitional approach, a superposition of two definite- or eigen-states states $\left\{a\right\}$ and $\left\{b\right\}$ is a state $\{a,b\}$ that is ontically indefinite or “blurred” between the two definite states. The partition approach requires the mental change-over from superposition-as-wave-addition to superposition-as-indefiniteness (since partitions are the math tool to describe indefiniteness and definiteness). Quantum reality is Indefinite World.

For a pictorial image, the left side of Figure 4 gives the superposition of two isosceles triangles with labelled edges and vertices as the triangle that is indefinite on the edges and vertices where they differ and only definite where the two triangles are the same–and certainly not the triangle that is doubly definite (like a double-exposure photograph). Forget waves an ontological entities; instead think of objective indefiniteness.

The fact that where superposed states have the same property is still definite (like vertex label a, the side label A) in the superposition is a little noticed fact about quantum superpositions.

It follows from the linearity of the operators which represent observables of quantum mechanical systems that any measurable physical property which happens to be shared by all of the individual mathematical terms of some particular superposition (written down in any particular basis) will necessarily also be shared by the full superposition, considered as a single quantum-mechanical state, as well. [4], (234)

This means that the notion of superposition in QM is the flip-side of the notion of abstraction (e.g., in mathematics) [21].7 In a superposition, the emphasis is on the indefiniteness resulting from where the elements of a set (i.e., a non-singleton equivalence class in an equivalence relation) differ like the vertex labels b and c, and the side labels B and C in Figure 4, while in abstraction, the emphasis is on the properties that are the same for the elements in the set (or equivalence class) like the vertex label a and side label A. It is like the flip-side different viewpoints of seeing the same glass as half-empty or as half-full.

In the older homotopy theory, the abstraction of a “homotopy type” was represented by a block in a partition, i.e., by an equivalence class. “Homotopy types are equivalence classes of spaces” under the equivalence relation “of deformation or homotopy.” [7], (4). In the modern treatment [61], the equivalence class of, say, unit-interval-coordinatized paths going once around the ring clockwise (right side of Figure 4) is abstracted to an object having those common characteristics but is not itself one of the differing paths in the equivalence class.

This insight that superposition as indefiniteness-on-differences is the flip-side of abstraction as definiteness-on-commonalities is, of course, unavailable when superposition is thought of as wave-addition.

Any quantum theorist who realizes that a superposition in the measurement basis does not have a definite value prior to the measurement–already has a strong hint as to the nature of quantum reality. Some quantum theorists have taken the hint and envisioned a quantum world characterized by that indefiniteness where that “blurriness is an ontic feature” [50], (p. 163). Thus it is not a new idea that there is a quantum `underworld’ of indefinite superposition states `beneath’ the classical space-time world of definite states. That view was often expressed in the language of the quantum world of potentialities (or latencies) versus the classical world of definite actualities ([34]; [47]; [36], (Sec. 10.2); [57]; [29]; [35], (Quantum Reality #8); [38,40]; [15]).

Heisenberg [34], (53)... used the term “potentiality” to characterize a property which is objectively indefinite, whose value when actualized is a matter of objective chance, and which is assigned a definite probability by an algorithm presupposing a definite mathematical structure of states and properties. Potentiality is a modality that is somehow intermediate between actuality and mere logical possibility. That properties can have this modality, and that states of physical systems are characterized partially by the potentialities they determine and not just by the catalogue of properties to which they assign definite values, are profound discoveries about the world, rather than about human knowledge. [57], (6)

Ruth Kastner uses the imagery of an iceberg [38], (3) with the classical world above water and the quantum world beneath the water—an imagery that is filled out by the Figure 2 imagery of the partition lattice as the skeletonized classical and quantum states. The language of “potentialities” or the Aristotelian notion of “potentia” is not very felicitous since they are taken to be realities, not mere possibilities. Hence some prefer the language of “latencies” (e.g., Henry Margenau and R. I. G. Hughes), but in both the cases of “potentialities” and “latencies,” the key idea is objective indefiniteness.

The historical reference should perhaps be dismissed, since quantum mechanical potentiality is completely devoid of teleological significance, which is central to Aristotle’s conception. What it has in common with Aristotle’s conception is the indefinite character of certain properties of the system. [56], (313-4)

And Margenau notes that the measurement of observables “forces them out of indiscriminacy or latency” [47], (10)–which indicates that Margenau also interprets “latency” in terms of indeterminacy or indefiniteness. Kastner also considers the indeterminacy of values as a key characteristic of the real potentia [38], (3). “Although the quantum possibilities represent properties that might be observed, they are not determinate (that is, they are not well defined).” [39], (27) In short, it is not real potentialities versus real actualities, but real (objective) indefiniteness versus real definiteness. There is only one type of reality and it occurs in varying degrees of indefiniteness as represented in skeletal form by the partition lattice (Figure 2).

The main brain-twisting reconceptualization required to understand quantum reality is:

• to replace the classical notion of fully definite reality by the idea of indefiniteness at the quantum level (not definite all-the-way-down),
• where the state-reduction jump from an indefinite state to a more definite state is due to making distinctions (“more-definite its from dits”), and
• where change with no distinctions stays at the same level of indefiniteness (unitary evolution)

. Classical mechanics has only one type of change, definite to definite at the level of fully definite classical reality.

After a century of the philosophically fruitless conceptualization of QM states in terms of wave functions, is there another equivalent mathematical tool to represent a quantum state that makes the indefiniteness obvious? Yes, it is the density matrix.

4.1. Quantum States

To demonstrate our thesis that the math of QM is the Hilbert space version of the math of partitions, we need to first focus on the three main concepts in the math of QM: 1) the quantum state, 2) the quantum observable, and 3) the quantum ( always projective) measurement.

A quantum state can be presented either as a state vector or as a density matrix [63]. The density matrix approach best displays the relevant information for the partitional interpretation in terms of indefiniteness. Hence we start by transferring the structure of a partition $\pi$ into its density matrix form $\rho \left(\pi \right)$. The initial data is a partition $\pi =\{B_1,...,B_m\}$ on U with (positive) point probabilities $p=(p_1,...,p_n)$. For each $B_j\in \pi$, define $|b_j\rangle$ as the column vector with the $i^{th}$ entry being $\sqrt{p_i/Pr\left(B_j\right)}$ if $u_i\in B_j$, else 0 so that $\langle b_k|b_j\rangle ={\delta }_{jk}$. We form the projection density matrix $\rho \left(B_j\right)=|b_j\rangle \langle b_j|$ with the $i,k$-entry being $\rho {\left(B_j\right)}_{ik}=\frac{\sqrt{p_i{p}_k}}{Pr\left(B_j\right)}$ if $u_i,u_k\in B_j$, else 0. It should be noted how the indefiniteness interpretation of superposition comes out in the mathematics. The set-level superposition of $u_i$ and $u_k$ in $B_j$ implies the non-zero off-diagonal element in $\rho \left(B_j\right)$ indicating indefiniteness.

These density matrices $\rho \left(B_j\right)$ of pure superposition states also show the origin of the Born rule in the outer product construction of a pure state density matrix. Taking $B_j=S$, an arbitrary non-empty subset of U, the $i^{th}$ entry of $\left|s\right.〉$ is $〈u_i|s〉=\sqrt{\frac{p_i}{Pr\left(S\right)}}{\chi }_S\left(u_i\right)$. Then the probability of $u_i$ conditioned on S is given by the Born rule: ${〈u_i|s〉}^2=\frac{p_i}{Pr\left(S\right)}{\chi }_S\left(u_i\right)=Pr\left(u_i|S\right)$.8

Returning to the partition $\pi$, the density matrix $\rho \left(\pi \right)$ is the probability sum of these projectors:

$\rho \left(\pi \right)={\sum }_{j=1}^{m}Pr\left(B_j\right)|b_j\rangle \langle b_j|$.

Then it is easily checked that $\rho {\left(\pi \right)}_{ik}=\sqrt{p_i{p}_k}$ if $(u_i,u_k)\in indit\left(\pi \right)$, else 0. Thus the non-zero entries in $\rho \left(\pi \right)$ represent the indits of $\pi$ and the zero entries represent the distinctions of $\pi$. A density matrix is not only Hermitian but positive so its eigenvalues are non-negative real numbers ${\lambda }_i$ which sum to 1, i.e., ${\sum }_{i=1}^n{\lambda }_i=1$. In the case of $\rho \left(\pi \right)$, these ${\lambda }_i$ are m non-zero eigenvalues $Pr\left(B_j\right)$ with the remaining $n-m$ eigenvalues of 0.

For the classical state ${\mathbf{1}}_U$, its density matrix $\rho \left({\mathbf{1}}_U\right)$ is a diagonal matrix with the point probabilities along the diagonal, e.g. “the statistical mixture describing the state of a classical dice before the outcome of the throw” [6], (176). Thus the non-classical states in the skeletal representation are the ones where $\rho \left(\pi \right)$ has non-zero off-diagonal elements indicating the `amplitudes’ $\sqrt{p_i{p}_k}$ of the corresponding diagonal states ($u_i$ and $u_k$) blobbing or cohering together in a superposition. Since superposition states are the key non-classical states, it is these non-zero off-diagonal “coherences” [13], (303) that account for the non-classical interference effects in the Hilbert space version.

[T]he off-diagonal terms of a density matrix ... are often called quantum coherences because they are responsible for the interference effects typical of quantum mechanics that are absent in classical dynamics. [6], (177).

In the full non-skeletal Hilbert space case of a density matrix $\rho$, it has a spectral decomposition $\rho ={\sum }_{i=1}^n{\lambda }_i|u_i\rangle \langle u_i|$ with an orthonormal basis $\left\{\right|u_i\rangle {\}}_{i=1}^n$so $\langle u_{i^{\prime }}|u_i\rangle ={\delta }_{i{i}^{\prime }}$ and where the non-negative eigenvalues ${\lambda }_i$ sum to 1. Thus for the concept of a quantum state, we have the skeletal set level presentation of a partition and the corresponding Hilbert space version of that partition math as summarized in Table 4.

4.2. Quantum Observables

We have seen how the notion of a quantum state was prefigured at the set level by (i.e., has the set level precursor of) a partition on a set with point probabilities. In a similar manner, the notion of a quantum observable is prefigured at the set level by the inverse-image partition ${\left\{f^{-1}\left(r\right)\right\}}_{r\in f\left(U\right)}$ of a real-value numerical attribute $f:U\to \mathbb{R}$.

In the folklore of mathematics, there is a semi-algorithmic procedure to connect set concepts with the corresponding vector (Hilbert) space concepts. We will call it a “yoga” [51], (251), the Yoga of Linearization:

For any given set-concept, apply it to a basis set of a vector space

and whatever is linearly generated is the corresponding vector space concept.

The Yoga of Linearization.

To apply the Yoga, we first take U as just a universal set and consider some set-based concept, and then we consider U as a basis set (e.g., ON basis of a Hilbert space) of a vector space V and see what is linearly generated. Starting with U, it is just the free vector space functor that takes a set to the free vector space ${\mathbb{k}}^U=V$ over a given field $\mathbb{k}$ which we take to be $\mathbb{C}$. A subset S of a basis set U generates a subspace $\left[S\right]$ of the space V. The cardinality of the subset gives the dimension of the subspace. A real-valued numerical attribute $f:U\to \mathbb{R}$ defines a Hermitian operator $F:V\to V$ (where $V=\left[U\right]$) by defining F on the basis set U as $Fu=f\left(u\right)u$ or, using the fancier notation, $F|u\rangle =f(u\left)\right|u\rangle$.

To better analyze the numerical attribute, let $f\upharpoonright S=rS$ stand for “the value of f on the subset S is r”. That is the set level version of the eigenvalue/eigenvector equation $F|u\rangle =r|u\rangle$. Hence we see that the set version of an eigenvector is a constant set S of f and the set version of an eigenvalue of an eigenvector is the constant value r on a constant set S. A characteristic function ${\chi }_S:U\to \{0,1\}\subseteq \mathbb{R}$ has only two constant sets $S={\chi }_S^{-1}\left(1\right)$ and $S^c=U-S={\chi }_S^{-1}\left(0\right)$. The Yoga yields the corresponding vector space notion which is a projection operator $P_{\left[S\right]}$ which is defined by $P_{\left[S\right]}|u\rangle =|u\rangle$ if $u\in S$, else $\mathbf{0}$ (zero vector) which has the eigenvalues of 0 and 1.

A Hermitian (or self-adjoint) operator F in QM has spectral decomposition $F={\sum }_{{\lambda }_i}{\lambda }_i{P}_{V_i}$ where the sum is over the real eigenvalues ${\lambda }_i$ and the projections $P_{V_i}$ to their eigenspaces. Bearing in mind the correlation given by the Yoga, we can define the `spectral decomposition’ of the numerical attribute $f:U\to \mathbb{R}$ as $f={\sum }_{r\in f\left(U\right)}r{\chi }_{f^{-1}\left(r\right)}$. Starting at the quantum level with a Hermitian operator $F:V\to V$ and a basis set U of eigenvectors of F, then going backwards to the set level, f is obtained as the eigenvalue function $f:U\to \mathbb{R}$.

An important application of the Yoga is to the notion of a set partition $\pi ={\left\{f^{-1}\left(r\right)\right\}}_{r\in f\left(U\right)}$ as the inverse-image of a numerical attribute. Applied to the basis set $\left\{\right|u_i\rangle {\}}_{i=1}^n$ used to define F by $F|u_i\rangle =f\left(u_i\right)|u_i\rangle$, each block $f^{-1}\left(r\right)$ of $\pi$ generates the eigenspace of eigenvectors for the eigenvalue r. These eigenspaces ${\left\{V_r\right\}}_{r\in f\left(U\right)}$ form a direct-sum decomposition (DSD) of V, i.e., $V={\oplus }_{r\in f\left(U\right)}{V}_r$, where a DSD is defined as a set of non-zero subspaces ${\left\{V_r\right\}}_{r\in f\left(U\right)}$ such that every non-zero vector $v\in V$ has a unique representation as a sum $v={\sum }_{r\in f\left(U\right)}{v}_r$ of vectors $v_r\in V_r$.9

It was noted previously that a set partition $\pi =\{B_1,...,B_m\}$ has a similar definition since every non-empty subset S has a unique representation as the union of subsets of the ${\left\{B_j\right\}}_{j=1}^m$. If the union of the $B_j$’s was not all of U, then $U-{\cup }_{j=1}^m{B}_j$ would have no representation, and if $S=B_j\cap B_{j^{\prime }}\ne \varnothing$, then S has two representations as subsets of the $B_j$’s. Moreover, $S\cap \left(\right):\wp \left(U\right)\to \wp \left(U\right)$ is a projection operator that takes any subset $T\in \wp \left(U\right)$ to $S\cap T\in \wp \left(U\right)$. Thus we have ${\cup }_{r\in f\left(U\right)}(f^{-1}\left(r\right)\cap \left(\right))=I:\wp \left(U\right)\to \wp \left(U\right)$ whose quantum version is resolution of unity ${\sum }_{r\in f\left(U\right)}{P}_{V_r}=I:V\to V$. And lastly, we might apply the Yoga to the Cartesian product $U\times U^{\prime }$ where U and $U^{\prime }$ are basis sets for V and $V^{\prime }$. Then the ordered pairs $(u,u^{\prime })\in U\times U^{\prime }$ (bi)linearly generate the tensor product $V\otimes V^{\prime }$ where the ordered pair $(u,u^{\prime })$ is customarily written as $u\otimes u^{\prime }$or $|u\rangle \otimes |u^{\prime }\rangle$.

These results of the Yoga of Linearization are summarized in Table 5 further showing how the math of QM is the Hilbert space version of partition math at the skeletal level of sets.

4.3. Quantum Measurement

The third basic concept to be analyzed is quantum measurement (always projective)–the quantum notion of becoming, i.e., change going from a level of indefiniteness to a level of more definiteness. The connection between the set level notion of measurement and the quantum level is the Lüders mixture operation ([45]; [6], (p. 279)) that can be applied at both levels. At the set level, we have the skeletal state represented by a density matrix $\rho \left(\pi \right)$ and we have an `observable’ or real-value numerical attribute, say, $g:U\to \mathbb{R}$ whose inverse-image is the partition $\sigma =g^{-1}={\left\{g^{-1}\left(r\right)\right\}}_{r\in g\left(U\right)}$. The Lüders mixture operation applies the `observable’ to the density matrix $\rho \left(\pi \right)$ to arrive at the post-measurement density matrix $\widehat{\rho }\left(\pi \right)$. The operation uses the $n\times n$ projection matrices for the blocks $g^{-1}\left(r\right)$ of $\sigma$ which are diagonal matrices $P_{g^{-1}\left(r\right)}$ whose diagonal elements are the values of the characteristic function ${\chi }_{g^{-1}\left(r\right)}$. Then the post-measurement density matrix is:

$\widehat{\rho }\left(\pi \right)={\sum }_{r\in g\left(U\right)}{P}_{g^{-1}\left(r\right)}\rho \left(\pi \right)P_{g^{-1}\left(r\right)}$

Partition version of Lüders mixture operation.

Then it is an easy result:

Theorem: $\widehat{\rho }\left(\pi \right)=\rho (\pi \vee g^{-1})$.

Thus the set version of quantum level projective measurement in the math of QM is the partition join operation where $dit(\pi \vee \sigma )=dit\left(\pi \right)\cup dit\left(\sigma \right)$ and, by DeMorgan’s Law, $indit(\pi \vee \sigma )=indit\left(\pi \right)\cap indit\left(\sigma \right)$ so the join of partitions operation can also be viewed as the intersection of equivalence relations. The non-classical quantum notion of becoming is the jump from $\pi$ to the mixed state $\pi \vee g^{-1}$ and then to one of the states in the mixture according to its probability.

Example: Let $\pi =\left\{\right\{a\},\{b,c\left\}\right\}$ with probabilities $Pr\left(\left\{a\right\}\right)=\frac{1}{3}$, $Pr\left(\left\{b\right\}\right)=\frac{1}{4}$, and $Pr\left(\left\{c\right\}\right)=\frac{5}{12}$ in $U=\{a,b,c\}$ and $\sigma =\left\{\right\{a,b\},\{c\left\}\right\}$ so $\sigma =g^{-1}$ for any $g:U\to \mathbb{R}$ that assigns the same g-value to a and b with a different value for c. Then the density matrix for $\pi$ and the projections matrices for the blocks of $\sigma$ are:

$\rho \left(\pi \right)=\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& \frac{\sqrt{5}}{4\sqrt{3}}\\ 0& \frac{\sqrt{5}}{4\sqrt{3}}& \frac{5}{12}\end{array}\right]$, $P_{\{a,b\}}=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]$, and $P_{\left\{c\right\}}=\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right]$.

The Lüders mixture operation is:

$\widehat{\rho }\left(\pi \right)=P_{\{a,b\}}\rho \left(\pi \right)P_{\{a,b\}}+P_{\left\{c\right\}}\rho \left(\pi \right)P_{\left\{c\right\}}$

$=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& \frac{\sqrt{5}}{4\sqrt{3}}\\ 0& \frac{\sqrt{5}}{4\sqrt{3}}& \frac{5}{12}\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right]$

$+\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& \frac{\sqrt{5}}{4\sqrt{3}}\\ 0& \frac{\sqrt{5}}{4\sqrt{3}}& \frac{5}{12}\end{array}\right]\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right]$

$=\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& 0\\ 0& 0& 0\end{array}\right]+\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& \frac{5}{12}\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& 0\\ 0& 0& \frac{5}{12}\end{array}\right]$.

Since $\pi \vee \sigma =\{\left\{a\right\},\left\{b\right\},\left\{c\right\}\}={\mathbf{1}}_U$, we see that the post-measurement density matrix is $\widehat{\rho }\left(\pi \right)=\rho (\pi \vee \sigma )=\rho \left({\mathbf{1}}_U\right)$. Thus the superposition of $\left\{b\right\}$ and $\left\{c\right\}$ in $\pi$ got distinguished since b and c had different g-values. The new distinctions in $dit\left(\sigma \right)-dit\left(\pi \right)$ are $(b,c)$ (along with $(c,b)$) and those were the non-zero off-diagonal elements (coherences) of $\rho \left(\pi \right)$ that got zeroed (distinguished or decohered) in $\rho (\pi \vee \sigma )$. Figure 5 shows the join of $\left\{\right\{a\},\{b,c\left\}\right\}$ and $\left\{\right\{a,b\},\{c\left\}\right\}$ is their least upper bound ${\mathbf{1}}_U=\{\left\{a\right\},\left\{b\right\},\left\{c\right\}\}$.

In the math of the partition join, the partitions are symmetrical. But the Lüders mixture operation is not symmetric even through the result is the join. One partition ($\pi =\left\{\right\{a\},\{b,c\left\}\right\}$) represents the state being measured and the other partition ($\sigma =\left\{\right\{a,b\},\{c\left\}\right\}$) represents the observable. Since the join in Figure 5 is the discrete partition, the measurement represented by the join is maximal. The arrow in Figure 5 represents the jump from a less refined mixed state $\pi$ to the maximal state ${\mathbf{1}}_U$.

The Lüders mixture is not the end of the measurement process. The measurement returns one of the g-values, say the (degenerate) one for $\{a,b\}$. Then the Lüders Rule [36], (Appendix B) gives the final density matrix which is the corresponding term in the Lüders mixture sum, e.g., $P_{\{a,b\}}\rho \left(\pi \right)P_{\{a,b\}}$, normalized so the final density matrix is the (in this case, classical) mixed state:

$\frac{P_{\{a,b\}}\rho \left(\pi \right)P_{\{a,b\}}}{tr\left[P_{\{a,b\}}\rho \left(\pi \right)P_{\{a,b\}}\right]}=\left[\begin{array}{ccc}\frac{1}{3}& 0& 0\\ 0& \frac{1}{4}& 0\\ 0& 0& 0\end{array}\right]\frac{1}{7/12}=\left[\begin{array}{ccc}\frac{4}{7}& 0& 0\\ 0& \frac{3}{7}& 0\\ 0& 0& 0\end{array}\right]$.

In the general Hilbert space case, the Hermitian operator G is given by its DSD of eigenspaces ${\left\{V_r\right\}}_{r\in g\left(U\right)}$ (where $g:U\to \mathbb{R}$ is the eigenvalue function assigning the appropriate real eigenvalue to each vector in an ON basis U of eigenvectors of G). The state being measured is given by density matrix $\rho$ expressed in the ON basis U, and the Lüders mixture operation uses the projection matrices $P_{V_r}$ to the eigenspaces of G to determine the post-measurement density matrix $\widehat{\rho }$. The Hilbert space version of the set operation is:

$\widehat{\rho }={\sum }_{r\in g\left(U\right)}{P}_{V_r}\rho P_{V_r}$

Hilbert space version of Lüders mixture operation.

We saw previously how the notion of logical entropy (and its quantum counterpart) was based on the notion of a quantitative measure of distinctions of a partition. Hence logical entropy is the natural notion to measure the changes in a density matrix under a measurement. For instance, in the above example where $\pi =\left\{\right\{a\},\{b,c\left\}\right\}$, the block probabilities are $Pr\left(\left\{a\right\}\right)=\frac{1}{3}$ and $Pr\left(\{b,c\}\right)=\frac{2}{3}$, and the logical entropy is: $h\left(\pi \right)=1-Pr{\left(\left\{a\right\}\right)}^2-Pr{\left(\{b,c\}\right)}^2=1-\frac{1}{9}-\frac{4}{4}=\frac{4}{9}$. When the partition $\pi$ is represented as a density matrix $\rho \left(\pi \right)$, then the logical entropy could also be computed as:

$h\left(\pi \right)=h\left(\rho \left(\pi \right)\right)=1-tr\left[\rho {\left(\pi \right)}^2\right]=1-tr\left[\begin{array}{ccc}\frac{1}{9}& 0& 0\\ 0& \frac{1}{6}& \frac{1}{18}\sqrt{3}\sqrt{5}\\ 0& \frac{1}{18}\sqrt{3}\sqrt{5}& \frac{5}{18}\end{array}\right]=1-\frac{10}{18}=\frac{4}{9}.$

The logical entropy is the two-draw probability of drawing a distinction of $\pi$ so it could also be computed as the sum of all the distinction probabilities (remembering that a distinction is an ordered pair of elements in different blocks so the probability of an unordered pair is doubled). Hence in general we have: $h\left(\pi \right)={\sum }_{(u_i,u_k)\in dit\left(\pi \right)}{p}_i{p}_k$, or in the case at hand:

$h\left(\pi \right)=2p_a{p}_b+2p_a{p}_c=2\times \frac{1}{3}\times \frac{1}{4}+2\times \frac{1}{3}\times \frac{5}{12}=\frac{1}{6}+\frac{5}{18}=\frac{4}{9}$.

There is then a general theorem [22] showing how logical entropy measures measurement.

Theorem (set case of measuring measurement): The increase in logical entropy from $h\left(\rho \right(\pi \left)\right)$ to $h\left(\widehat{\rho }\left(\pi \right)\right)$ in the Lüders mixture operation $\widehat{\rho }\left(\pi \right)={\sum }_{r\in g\left(U\right)}{P}_{g^{-1}\left(r\right)}\rho \left(\pi \right)P_{g^{-1}\left(r\right)}$, is the sum of the squares of the off-diagonal non-zero entries in $\rho \left(\pi \right)$ that were zeroed in the measurement operation $\rho \left(\pi \right)⇝\widehat{\rho }\left(\pi \right)$.

In the example, the logical entropy of the post-measurement state is:

$h\left(\widehat{\rho }\left(\pi \right)\right)=1-tr\left[\widehat{\rho }{\left(\pi \right)}^2\right]=1-tr\left[\begin{array}{ccc}\frac{1}{9}& 0& 0\\ 0& \frac{1}{16}& 0\\ 0& 0& \frac{25}{144}\end{array}\right]=1-(\frac{1}{9}+\frac{1}{16}+\frac{25}{144})=1-\frac{16+9+25}{144}=\frac{94}{144}$.

The sum of the squares of the non-zero off-diagonal terms (representing the coherences) of $\rho \left(\pi \right)$ that were zeroed (decohered) in $\widehat{\rho }\left(\pi \right)$ is:

$2\times {\left(\frac{\sqrt{5}}{4\sqrt{3}}\right)}^2=2\times \frac{5}{48}=\frac{5}{24}$

and the increase in logical entropy due to the making of distinctions is:

$h\left(\widehat{\rho }\left(\pi \right)\right)-h\left(\rho \left(\pi \right)\right)=\frac{94}{144}-\frac{4}{9}=\frac{94}{144}-\frac{64}{144}=\frac{30}{144}=\frac{5}{24}$.

And the quantum case follows mutatis mutandis.

Theorem (quantum case of measuring measurement): In the Lüders mixture operation $\widehat{\rho }={\sum }_{r\in g\left(U\right)}{P}_{V_r}\rho P_{V_r}$, the increase in quantum logical entropy from $h\left(\rho \right)$ to $h\left(\widehat{\rho }\right)$ is the sum of the absolute squares of the off-diagonal entries in $\rho$ that were zeroed in the measurement operation $\rho ⇝\widehat{\rho }$.

The dictionary relating the three basic concepts in the math of QM to their set partitional precursors is given in Table 6.

4.4. Other Aspects of QM Mathematics

4.4.1. Commuting, Non-Commuting, and Conjugate Operators

We have seen that a Hermitian operator is the QM math version of a real-valued numerical attribute and that the direct-sum decomposition of the operator’s eigenspaces is the QM math version of the inverse-image partition of the numerical attribute. Let $F,G:V\to V$ be two Hermitian operators with the corresponding DSDs of ${\left\{V_j\right\}}_{j\in J}$ and ${\left\{W_{j^{\prime }}\right\}}_{j^{\prime }\in J^{\prime }}$. Since the two DSDs are the vector space version of partitions, consider the join-like operation giving the set of non-zero subspaces formed by the intersections $V_j\cap W_{j^{\prime }}$. The additional generality gained over the join of set partitions is that these subspaces may not span the whole space V. Since the vectors in those intersections are simultaneous eigenvectors of F and G, let $\mathcal{SE}$ be the subspace spanned by the simultaneous eigenvectors of F and G. The commutator $[F,G]=FG-GF:V\to V$ is a linear operator on V so it has a kernel $ker[F,G]$ consisting of the vectors v such that $[F,G]v=\mathbf{0}$. Then there is a:

Theorem: $\mathcal{SE}=ker[F,G]$. [24]

Since commutativity is defined as $ker[F,G]=V$, we have the following definitions in terms of the vector space partitions or DSDs:

• F and G are commuting if $\mathcal{SE}=V$;
• F and G are incompatible if $\mathcal{SE}\ne V$;
• F and G are conjugate if $\mathcal{SE}=0$ (zero space).

Since the join-like operation on DSDs yields a set of subspaces that do not necessarily span the whole space, that operation is only the join of DSDs in the commuting case–or as Hermann Weyl put it: “Thus combination [join] of two gratings [DSDs] presupposes commutability...” [65], (257).

The set version of compatible partitions for the join operation is simply being defined on the same set. Hence our thesis gives a complete parallelism between compatible partitions and commuting operators.

Partition math: A set of compatible partitions $\pi ,\sigma ,...,\gamma$ defined by $f,g,...,h:U\to \mathbb{R}$ is said to be complete, i.e., a Complete Set of Compatible Attributes or CSCA, if their join is the partition whose blocks are of cardinality one (i.e., ${\mathbf{1}}_U$). Then the elements $u\in U$ are uniquely characterized by the ordered set of values $\left(f\right(u),g(u),...,h(u)$.

QM math: A set of commuting observables $F,G,...,H$ is said to be complete, i.e., a Complete Set of Commuting Observables or CSCO [16], (57), if the join of their eigenspace DSDs is the DSD whose subspaces are of dimension one. Then the simultaneous eigenvectors of the operators are unique characterized by the ordered set of their eigenvalues.

This is a particularly clear case of showing that the QM math is a word-for-word translation (using the Yoga dictionary) of partition math.

4.4.2. Feynman’s Treatment of Measurement

The partitional approach to understanding the math of QM shows that the key organizing concepts are indistinction versus distinction, indefiniteness versus definiteness, or indistinguishability versus distinguishability. Those concepts define the notion of logical entropy in the new partition-logical theory of information-as-distinctions ([22]; [46]). When a particle in a superposition state undergoes an interaction, what characterizes whether it is a measurement or not? As early as 1951 [25], Richard Feynman gave the analysis of “measurement or not” in terms of distinguishability.

If you could, in principle, distinguish the alternative final states (even though you do not bother to do so), the total, final probability is obtained by calculating the probability for each state (not the amplitude) and then adding them together. If you cannot distinguish the final states even in principle, then the probability amplitudes must be summed before taking the absolute square to find the actual probability.[27], (3-9)

This analysis has been further explained by John Stachel.

Feynman’s approach is based on the contrast between processes that are distinguishable within a given physical context and those that are indistinguishable within that context. A process is distinguishable if some record of whether or not it has been realized results from the process in question; if no record results, the process is indistinguishable from alternative processes leading to the same end result. [58], (314)

The same points are restated by Anton Zeilinger using, in effect, the notion of information-as-distinctions.10

In other words, the superposition of amplitudes ... is only valid if there is no way to know, even in principle, which path the particle took. It is important to realize that this does not imply that an observer actually takes note of what happens. It is sufficient to destroy the interference pattern, if the path information is accessible in principle from the experiment or even if it is dispersed in the environment and beyond any technical possibility to be recovered, but in principle still “out there.” The absence of any such information is the essential criterion for quantum interference to appear. [68], (484)

What Feynman calls “distinguishability,” Zeilinger calls “information” since as one of the founders of quantum information theory, Charles Bennett, put it, information “is the notion of distinguishability abstracted away from what we are distinguishing, or from the carrier of information.... [8], (p. 155)

Feynman gives a number of examples ([27], (§ 3-3); [28], (17-8)) such as a particle scattering off the atoms in a crystal. If there is no physical record of which atom the particle scattered off of (i.e., the indistinguishable case), then no measurement took place so the amplitudes for the superposition state of scattering off the different atoms are added to compute the amplitude of the particle reaching a certain final state. But if all the atoms had, say spin up, and scattering off an atom flipped the spin, then a physical record exists (i.e., the distinguishable case) so a measurement took place11 and the probabilities of scattering off the different atoms are added to compute the probability of reaching a certain final state.

The same analysis applies to the well-known double-slit experiment where the distinguishable case is where there are detectors at the slits and the indistinguishable case is having no detectors at the slits. But the important thing to notice about Feynman’s example is that the measurement is entirely at the quantum level; it involves no macroscopic apparatus. Hence the Feynman analysis bypasses the whole tortured literature trying to analyze measurement in term of the “decoherence” induced by a macroscopic measuring devices (e.g., [69]). Of course, the quantum level physical record in the distinguishable case has to be amplified for humans to record the result but such macroscopic considerations have no role in quantum theory.

The implicit principle in Feynman’s analysis of measurement is:

If the interaction distinguishes between superposed eigenstates,

then a distinction (state reduction) is made.

The State Reduction Principle

The mathematics of the State Reduction Principle can be stated in both the set case and the QM case.

Theorem(State Reduction Principle–set case). Measurement is described in the set case by the Lüders mixture operation $\widehat{\rho }\left(\pi \right)={\sum }_{r\in g\left(U\right)}{P}_{g^{-1}\left(r\right)}\rho \left(\pi \right)P_{g^{-1}\left(r\right)}$. The State Reduction Principle then states: if an off-diagonal entry $\rho {\left(\pi \right)}_{ik}\ne 0$ (i.e., $u_i$ and $u_k$ are in a same-block superposition), then: if $g\left(u_i\right)\ne g\left(u_k\right)$ (i.e., the interaction distinguishes $u_i$ and $u_k$), then $\widehat{\rho }{\left(\pi \right)}_{ik}=0$ (i.e., the `coherence’ between $u_i$ and $u_k$ is decohered and a distinction is made).

Theorem(State Reduction Principle–QM case). Measurement is described in QM by the Lüders mixture operation $\widehat{\rho }={\sum }_{r\in g\left(U\right)}{P}_{V_r}\rho P_{V_r}$ (measuring $\rho$ by G). The State Reduction Principle then states: if an off-diagonal entry ${\rho }_{ik}\ne 0$ (i.e., the G-eigenvectors $|u_i\rangle$ and $|u_k\rangle$ are in a superposition in $\rho$), then: if $|u_i\rangle$ and $|u_k\rangle$ have different G-eigenvalues (i.e., the vectors are distinguished by G), then ${\widehat{\rho }}_{ik}=0$ (i.e., the vectors are decohered12 and a distinction is made).

If no distinctions were made by the interaction, then no measurement took place.

4.4.3. Von Neumann’s Type I and Type II Processes

John von Neumann (vN) [62] made his famous distinction between the processes:

(1)

Type I process of measurement and state reduction, and

(2)

Type II process obeying the Schrödinger equation.

We have seen that the Type I processes of measurement involves distinguishability, i.e., the making of distinctions (like which atom the particle scattered off of), so a natural way to designate the Type II processes would be ones that do not make distinctions by preserving distinguishability or indistinguishability. The measure of indistinctness of two quantum states is their overlap or inner product. For instance, two states have zero indistinctness (zero inner product) then they are fully distinct (orthogonal). Hence the natural characterization of a Type II process is one that preserves inner products, i.e., a unitary transformation.13

The partitional approach highlights the key analytical concepts of indistinctions versus distinctions and the cognate notions of indefiniteness versus definiteness or indistinguishability versus distinguishability. Many people working on quantum foundations seem to ignore those key concepts, and then the division between the measurement and unitary evolution seems unfounded, if not “unbelievable.”

[I]t seems unbelievable that there is a fundamental distinction between “measurement” and “non-measurement” processes. Somehow, the true fundamental theory should treat all processes in a consistent, uniform fashion. [48], (245)

4.4.4. Hermann Weyl’s Imagery for Measurement

An industrial sieve is used to distinguish particles of matter of different sizes so it might serve as a helpful metaphor for the quantum process of making distinctions, namely measurement.

In Einstein’s theory of relativity the observer is a man who sets out in quest of truth armed with a measuring-rod. In quantum theory he sets out armed with a sieve. [17], (267)

Hermann Weyl quotes Eddington’s passage [65], (255) but uses his own expository notion of a “grating.” Weyl in effect uses the Yoga from the mathematical folklore to develop both the set notion of a grating as an “aggregate [which] is used in the sense of `set of elements with equivalence relation.”’ [65], (239) and the vector space notion of a direct-sum decomposition. In the set to vector space move of the Yoga, the “aggregate of n states has to be replaced by an n-dimensional Euclidean vector space” [65], (256). The notion of a vector space partition or “grating” in QM is a “splitting of the total vector space into mutually orthogonal subspaces” so that “each vector $\overrightarrow{x}$ splits into r component vectors lying in the several subspaces” [65], (256), i.e., a DSD. After thus referring to a partition and a DSD as a “grating” or “sieve,” Weyl notes that “Measurement means application of a sieve or grating” [65], (259), i.e., the making of distinctions by the join-like process described by the Lüders mixture operation.

This imagery of measurement (or non-measurement) as passing through a sieve or grating is illustrated in Figure 6.

One should imagine the roundish blob of dough as the superposition of the definite shapes in the grating or sieve. The interaction between the superposed blob and the sieve/grating forces a distinction in the state reduction case, so a distinction is made as the blob must pass through one of the definite-shaped holes. In general, a state reduction (`measurement’) from an indefinite superposition to a more definite state takes place when the particle in the superposition state undergoes an interaction that distinguishes the superposed states. If the grating did not distinguish between the alternatives, than no state reduction takes place in the transition from position A to B (unitary evolution) as in the Feynman rule for the indistinguishable alternatives case.

4.5. A Skeletal Analysis of the Double-Slit Experiment

Consider the skeletal case of a particle have three possible states $U=\{a,b,c\}$ which are interpreted as vertical positions in the setup for the double-slit experiment in Figure 7.

In the set level skeletal analysis, we have discarded the scalars from $\mathbb{C}$ but we are nevertheless left with the scalars 0 and 1 which are the elements of the field ${\mathbb{Z}}_2$. There is the natural correspondence between the zero-one vectors in the three-dimensional vector space ${\mathbb{Z}}_2^3$ (i.e., the column vectors ${[1,0,0]}^t$ is associated with $\left\{a\right\}$, and so forth) which establishes an isomorphism: ${\mathbb{Z}}_2^3\cong \wp \left(U\right)$, where the set addition is the symmetric difference, i.e., for $S,T\in \wp \left(U\right)$, $S+T=(S-T)\cup (T-S)$ [19]. That mimics the addition mod 2 in ${\mathbb{Z}}_2^3$ since, for instance, $\{a,b\}+\{b,c\}=\{a,c\}$. For our dynamics, we assume a non-singular linear transformation $\left\{a\right\}⇝\left\{a^{\prime }\right\}=\{a,b\}$, $\left\{b\right\}⇝\left\{b^{\prime }\right\}=\{a,b,c\}$, and $\left\{c\right\}⇝\left\{c^{\prime }\right\}=\{b,c\}$ which is non-singular since $\left\{a^{\prime }\right\}=\{a,b\}$, $\left\{b^{\prime }\right\}=\{a,b,c\}$, and $\left\{c^{\prime }\right\}=\{b,c\}$ also form a basis set for $\wp \left(U\right)$–so we also have a partition lattice $\Pi \left(U^{\prime }\right)$ on the basis set $U^{\prime }=\{a^{\prime },b^{\prime },c^{\prime }\}$.

We are interested in the analysis when the particle arrives at the screen in the superposition $|\mathrm{slit}1\rangle +|\mathrm{slit}2\rangle$, or in skeletal terms $\{a,c\}$.

Case 1: There are detectors at the slits to distinguish between the two superposed states so the state reduces to the half-half mixture of $\left\{a\right\}$ and $\left\{c\right\}$. Then $\left\{a\right\}$ evolves by the non-singular dynamics to $\{a,b\}$ which hits the wall and reduces to $\left\{a\right\}$ or $\left\{b\right\}$ with half-half probability. Similar $\left\{c\right\}$ evolves to $\{b,c\}$ which hits the wall and reduces to $\left\{b\right\}$ or $\left\{c\right\}$ with half-half probability. Since this is the case of distinctions between the alternative paths to $\left\{a\right\}$, $\left\{b\right\}$, or $\left\{c\right\}$ we add the probabilities to obtain:

$Pr\left(\right\{a\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\left\{a\right\}\mathrm{at}\mathrm{wall}\right|\left\{a\right\}\mathrm{at}\mathrm{screen})Pr\left(\left\{a\right\}\mathrm{at}\mathrm{screen}\right|\{a,c\}\mathrm{at}\mathrm{screen})=\frac{1}{2}\times \frac{1}{2}=\frac{1}{4}$.

$Pr\left(\right\{b\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\right\{b\left\}\mathrm{at}\mathrm{wall}\right|\left\{a\right\}\mathrm{at}\mathrm{screen})Pr(\left\{a\right\}\mathrm{at}\mathrm{screen}\left|\right\{a,c\left\}\mathrm{at}\mathrm{screen}\right)$

$+Pr\left(\left\{b\right\}\mathrm{at}\mathrm{wall}\right|\left\{c\right\}\mathrm{at}\mathrm{screen})Pr\left(\left\{c\right\}\mathrm{at}\mathrm{screen}\right|\{a,c\}\mathrm{at}\mathrm{screen})=\frac{1}{2}\times \frac{1}{2}+\frac{1}{2}\times \frac{1}{2}=\frac{1}{2}$.

$Pr\left(\right\{c\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\left\{c\right\}\mathrm{at}\mathrm{wall}\right|\left\{c\right\}\mathrm{at}\mathrm{screen})Pr\left(\left\{c\right\}\mathrm{at}\mathrm{screen}\right|\{a,c\}\mathrm{at}\mathrm{screen})=\frac{1}{2}\times \frac{1}{2}=\frac{1}{4}$.

Hence the probability distribution in the Case 1 of measurement at the screen is given in Figure 8.

Case 2: There are no detectors to distinguish between the slits in the superposition $\{a,c\}$ so it linearly evolves by the dynamics: $\{a,c\}=\left\{a\right\}+\left\{c\right\}⇝\{a,b\}+\{b,c\}=\{a,c\}$. Hence the probabilities at the wall are:

$Pr\left(\right\{a\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\left\{a\right\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{wall})Pr\left(\{a,c\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})=\frac{1}{2}\times 1=\frac{1}{2}$.

$Pr\left(\right\{b\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\right\{b\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{wall})Pr(\{a,c\}\mathrm{at}\mathrm{wall}\left|\right\{a,c\left\}\mathrm{at}\mathrm{screen}\right)=0\times 1=0$.

$Pr\left(\right\{c\left\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})$

$=Pr\left(\left\{c\right\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{wall})Pr\left(\{a,c\}\mathrm{at}\mathrm{wall}\right|\{a,c\}\mathrm{at}\mathrm{screen})=\frac{1}{2}\times 1=\frac{1}{2}$.

Hence the probability distribution in the Case 2 of no distinctions at the screen is given in Figure 9.

The Case 2 distribution shows the usual probability stripes due to the interference in the linear evolution of the superposition state $\{a,c\}$, i.e., the destructive interference in the evolved superposition $\{a,b\}+\{b,c\}=\{a,c\}$.

Our classical intuitions insist on asking: “Which slit did the particle go through in Case 2?”. That question assumes that the evolution of the state $\{a,c\}$ was at the classical level where the slits were distinguished. But in Case 2, the slits were not distinguished so the evolution took place at the lower level in the skeletal lattice of partitions. In Figure 10, the Case 2 (and vN Type II) evolution (solid arrow) is illustrated as going from the superposition state $\{a,c\}$ in the partition lattice of states on $U=\{a,b,c\}$ to the superposition state $\{a^{\prime },c^{\prime }\}$ lattice of states on $U^{\prime }=\{a^{\prime },b^{\prime },c^{\prime }\}$. The Case 1 (and vN Type I) becoming (dashed arrows) does not take place with no detection at the slits.

The important `take-away’ is that there are different levels of indefiniteness (as illustrated in the partition lattice) and evolution can take place at a non-classical level of indefiniteness so, in that case, there is no matter of fact of the particle going through one slit or the other at the classical level.

Sometimes metaphors can serve as an aid or crutch to our biologically evolved intuitions. Suppose we have a fence across a field with two gates (like in the double slit experiment). When the sun is shines, that is a metaphor for detecting which gate a creature may walk through to get from A to B. Our creature is Hegel’s Owl of Minerva who only flies at night [33], (p. 13). Hence when the sun shines, the owl is limited to “flatland” [1] so it has to go through one gate of another to get from A to B as in Figure 11.

But when the sun sets, the Owl of Minerva’s “flights and perches” [32], (198) can go from ground perch A to ground perch B without going through one gate or the other as in Figure 12.

Our classical (“flatlander”) intuitions see only the definite ground-level paths or trajectories and, in the absence of the projecting light source (or sun) as in Case 2 above, will insist on asking: “Which gate did the owl go through?”. Our intuitions insist: “Why can’t I always imagine what happened at the classical level of fully distinguished states?”. But with no detections at the slits in the double-slit experiment, there is no matter of fact of the particle going through a slit at the classical level since the evolution is at the non-classical quantum level as in Figure 10 (illustrated by the third dimension in our flatlander metaphor of Figure 12).

Quantum states are built from below, as it were, through being in-formed by information-as-distinctions. Von Neumann’s two processes separate the Type I becoming from below between levels of indefiniteness and the Type II unitary evolution at the same level of indefiniteness. Figure 10 shows how quantum evolution transforms one superposition into another at the same level of indefiniteness (which allows interference) without any becoming or rising to the level of classical fully distinguished states. That is hard to understand for those who insist on “interpreting” QM with only our evolved intuitions based on the macroscopic (superposition-less) world.

5. Discussion

Our thesis is that the math of QM is the Hilbert space version of the math of partitions, or, put the other way around, the math of partitions is the skeletonized version of QM math. There are many other aspects of QM math that could be investigated such as group representations on sets or on vector spaces over $\mathbb{C}$ since a group is essentially a `dynamic’ algebraic way to define an equivalence relation or DSD ([65], (73); [12]) (e.g., the orbit partition in a set representation or the DSD of irreducible subspaces in the vector space over $\mathbb{C}$ representation). But in this introductory treatment, we have hopefully analyzed enough aspects of QM math to illustrate our thesis.

The main mystery in QM is superposition (entanglement being a particularly mysterious special case) and the quantum state of becoming wherein a superposition state jumps to a more definite state by being in-formed by distinctions. The rules governing when an interaction involving a superposition constitutes a state reduction or not, are given by the Feynman rules based on the notion of distinguishability or indistinguishability. At the logical level, the mathematics of distinguishability versus indistinguishability (or distinctions versus indistinctions) is the math of partitions. The quantitative version of partitions gives the notion of logical entropy that is the measure of information-as-distinctions so that in the Type I measurement process, distinctions in-form an indefinite state to jump to a more definite state.14 In contrast to the usual imagery of superposition as the addition of waves, the notion of ontological indefiniteness provides the alternative characteristic feature of non-classical superposition states.15

But, it will be said, “from the logical viewpoint, a property or the complementary property has to definitely apply to a particle.” That is the viewpoint of the classical logic of subsets; an element has to be in a subset or in its complement; there is no indefiniteness. There is, however, an equally fundamental logic at the mathematical level, the dual logic of partitions.16 In a (non-discrete) partition on the definite- or eigen-states of an observable, each non-singleton block represents the (skeletonized version) of objective indefiniteness of those eigenstates in the block. In a state reduction, distinctions are made to create more definiteness–with the classical mixed state of fully distinguished states being the maximal result. That is the notion of becoming described in partition logic, not the sort of transformations considered in classical physics, not to mention the classical logic of subsets.

Subsets and partitions are equally fundamental (indeed, dual) logical concepts. The notion of partitions (or equivalence relations) is the logical concept to describe indistinctions versus distinctions, indistinguishability versus distinguishability, or indefiniteness versus definiteness. Thus the partitional approach to QM does not use some jury-rigged concepts (e.g., Bohmian mechanics or spontaneous localization theory) to have yet another “interpretation” of QM. It should not be surprising that fundamental logical concepts can explicate the fundamental physical theory. [67]

This approach to better understanding or interpreting QM works with the standard von Neumann/Dirac quantum theory. It does not involve any new physics, unlike the pilot-wave or spontaneous localization theories. In that sense, the partitional approach shows how to develop Shimony’s idea of the Literal Interpretation of the math or “formalism of quantum mechanics” [57], (6-7). Furthermore, the partitional analysis substantiates the analysis of Heisenberg, Shimony, and others which describes the quantum world in terms of potentialities or latencies, where, in both cases, the key attribute was the reality of objective indefiniteness.17

• Superposition is the key non-classical notion of state in QM math;
• objective indefiniteness is its ontological counterpart;
• the quantum notion of becoming (vN Type I) is the jump from an indefinite state to a more definite state; and
• the quantum notion of evolution (vN Type II) is between states at the same level of indefiniteness.

This way of understanding or interpreting quantum mechanics might be called the Objective Indefiniteness Interpretation ([24]).