Thermodynamics of Observations

1. Introduction

The laws of thermodynamics outline the constraints that systems obey in their statistical behavior, usually in terms of state variables and for (quasi) equilibrium states. They can be summarized as (T0) transitivity of thermal equilibrium; (T1) energy conservation; (T2) entropy increase with time; and (T3) non-zero temperature. The universality of these laws has surprised for nearly two centuries, since the dawn of thermodynamics as a discipline, and is related to what one can possibly know about a system near thermal equilibrium [1]. It is typically irrelevant, however, how this thermodynamic information is obtained. In contrast therewith, in this work we explicitly assert that this information, and the thermodynamic assessment based thereon, results from registered perceptions (whether fictive or real) of a predefined measurand m that is subject to noise. This noise component can originate from random variations in the measurand, called variability, or from random errors in the measurement system, called precision. Usually, in practice, both appear combinedly. This does not affect the analysis below, although it is easier to consider either one or the other in order not to mix the degrees of freedom of the observed and observing systems (also see the section on demonstrative applications). In any case, our goal is a thermodynamic assessment resulting from an ensemble of observations, meaning that its statistics is captured by the observation distribution’s state variables and the laws of thermodynamics connecting them.

The distribution of repeated discrete observations is typically approximated by continuous Gaussian statistics. A connection with the arguably more appropriate equibinomial distribution, which is discrete by construction but has the normal distribution as its continuum limit (cf. De Moivre–Laplace theorem), is missing. This work starts with the derivation of a thermodynamic partition function from an equibinomial observation distribution, without the usual need for variational calculus [2]. A discussion on the statistical (observation) ensembles that relate with this partition function follows next, and enables a tailored reformulation of the four laws of thermodynamics. Before conclusions are drawn, five demonstrative applications put this work into experimental context. These touch upon, but are not limited to, uncertainty assessment discussions within the atmospheric research community that initiated this work [3,4].

2. Partition Function

The trajectory of a single ball on a Galton board with n levels is essentially a sequence of n left-right choices. It can be described by a Bernoulli process consisting of n trials. A single observation of a measurand m can be identified with a Bernoulli process analogously, with each of its n degrees of freedom contributing a noise component $+ϵ$ or $-ϵ$ to the measurand m fully randomly, i.e., with equal probability $p=0.5$ (irrespective of whether the noise originates from the observed system or the observing system). A sequence of N independent observations of m hence consists of $Nn$ Bernoulli trials within N trajectories. Combinatorics dictates that $2^n$ unique trajectories can be followed, each with equal probability $1/2^n$, but some trajectories will yield the same result in agreement with the binomial coefficients for an unordered selection of k out of n: The observation outcomes are located at $m+(2k-n)ϵ$ if k represents the number of positive noise contributions, irrespective of their order within the trajectory, with $0\le k\le n$. This means that the N measured values of m are distributed over $B=n+1$ result bins of width $2ϵ$, centered at m. They follow the equibinomial distribution by construction if $N=2^n$, see Figure 1, as assumed in the remainder of this section. The distribution is under-sampled for $N<2^n$, while it is over-sampled for $N>2^n$ (also see next section). On the other hand, if the number of result bins can be chosen at will, the distribution can be called under-resolved for $B<n+1$ and over-resolved for $B>n+1$, respectively.

The binomial probability density $p_k$ of bin k is given by:

$$p_k=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)p^k{(1-p)}^{n-k}=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)2^{-n}$$

(1)

where the last equation holds for the equibinomial distribution with $p=0.5$. This means that for $N=2^n$ observations, the number of results per bin $P_k$ equals the binomial coefficient:

$$P_k=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)={\displaystyle \frac{n!}{k!(n-k)!}}$$

(2)

This is demonstrated in Figure 1 for $n=4$. The sum over all states $P_k$ is called the partition function Z (from Zustandssumme in German), which here is fully characterized:

$$Z=\sum _{k=0}^n{P}_k=\sum _{k=0}^n\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)=\sum _{k=0}^n{2}^n{p}_k=2^n$$

(3)

The equibinomial distribution corresponds to a discrete probability density function $p_k$ that, according to the De Moivre–Laplace theorem, approaches the continuous probability density function $p_x$ of the normal distribution $\varphi (x,\mu ,\sigma )$ for large n (or N, and possibly given a continuity correction):

$$p_k\approx p_{x_k\pm ϵ}={\int }_{x_k-ϵ}^{x_k+ϵ}\varphi (x,\mu ,\sigma )dx$$

(4)

for $x_k=m+(2k-n)ϵ$ to match the k binomial bins of width $2ϵ$, numbered 1 to $n+1$. Instead of using the Gauss error function to determine this definite integral, for sufficiently large n it can be straightforwardly approximated by $2ϵ\varphi (x_k,\mu ,\sigma )$, resulting in:

$$p_k\approx {\displaystyle \frac{2ϵ}{\sigma \sqrt{2\pi }}}exp\left[-{\displaystyle \frac{{(x_k-\mu )}^2}{2{\sigma }^2}}\right]$$

(5)

By insertion of the expression for $x_k$, the expectation (or mean) $\mu \equiv m$, and the standard deviation $\sigma =2ϵ\sqrt{np(1-p)}$ in units of the bin width, or $\sigma =ϵ\sqrt{n}$ for $p=0.5$ here, one obtains for the partition function:

$$Z=\sum _{k=0}^n{2}^n{p}_k\approx \sum _{k=0}^n{\displaystyle \frac{2^{n+1}}{\sqrt{2\pi n}}}exp\left[-{\displaystyle \frac{{(2kϵ-nϵ)}^2}{2n{ϵ}^2}}\right]$$

(6)

As the normal distribution is the (continuous) distribution that maximizes the Shannon or Gibbs entropy for a specified mean and variance, this approach provides a succinct approximation to the classical and discrete canonical partition function for thermodynamic equilibrium states, without the need for variational calculus as in the Jaynesian maximum entropy approach [2]. In its general thermodynamic form, it is written as:

$$Z=\sum _k{z}_{k}exp(-\beta E_k)$$

(7)

with $z_k$ accounting for the number of particles (or, here: observation trajectories $P_k$ yielding bin k) involved and $E_k$ the energy of state (or bin) k, to be determined later on. $\beta$ equals the reciprocal of the Boltzmann constant times a temperature. Assuming a perfect equibinomial distribution, $z_k$ is uniquely determined by n (again see Figure 1 for deviations therefrom). In contrast with Equation (7), however, Equation (6) provides an exact formulation of the normalization factors and coefficients involved. It is therefore not surprising that the statistical “state variables” like (quadratic) mean, variance, and information content can be derived therefrom, fully analogous to the derivation of the energy E, temperature T, and entropy S, respectively, from the partition function in classical thermodynamics.

The relationships between the partition function and the thermodynamic variables of the system involve $\partial {log}_2\left(Z\right)$, which straightforwardly equals $\partial n$ for $Z\equiv N=2^n$ here, without having to go through the full differentiation of Equation (6). For the expectation value of the thermodynamic energy, one obtains:

$$〈E〉=-{\displaystyle \frac{\partial {log}_2\left(Z\right)}{\partial \beta }}={\displaystyle \frac{n{k}_{B}T}{2}}={\left(nϵ\right)}^2$$

(8)

from $\beta =1/k_{B}T$, and upon identification of the exponential denominators of Equations (6) and (7): $k_{B}T\equiv 2n{ϵ}^2$. This result confirms that $〈E〉$ equals the quadratic mean of the observation distribution, apart from a constant offset. The temperature variable thus provides a measure of the energy per degree of freedom and of the observation ensemble variance, apart from half the Boltzmann constant $k_B$ as a unit conversion factor. The entropy of the distribution as determined from the partition function equals $\partial \left(k_{B}T{log}_{2}Z\right)/\partial T=k_{B}n$ as a direct measure of the system’s degrees of freedom. The entropy as such is proportional to the base-two logarithm of the number of possible Bernoulli process outcomes indeed. In other words, the partition function provides a missing parameter, being the equibinomial distribution integral as a scaling factor, for fully characterizing a distribution of observations in thermal equilibrium (also see next sections). In the absence of scaling, the latter still requires three state variables (m, n, $ϵ$) instead of two as for the normal distribution ($\mu$, $\sigma$).

3. Statistical Ensembles

The partition function for equibinomial observation distributions presented in Equation (6) can easily be related to the well-known statistical ensembles of classical thermodynamics [2]: For a fixed number of observations $N\gtrsim 2^n$ with result binning according to the equibinomial distribution, the canonical ensemble is obtained (see Table 1). The macro-canonical ensemble includes several canonical ensembles with the same random error $ϵ$, but different numbers of degrees of freedom n. For $N=1$, one has one observation (bin) only, and hence the variance is undefined, in agreement with the fixed energy of the micro-canonical ensemble (again see Table 1). One however also obtains the micro-canonical ensemble, with finite entropy, upon forcing $N>1$ observations into a single (under-resolved, see Figure 1) result bin.

By assuming that within a single observation each of the n degrees of freedom of the system contributes a unique assessment of the measurand, each observation can also be regarded as a sequence of n discrete sub-observations (Bernoulli trials in the previous section). Therefore, in the common first law of thermodynamics, $\delta E=T\delta S+{\mu }_E\delta N_E$ (with subscripts E to distinguish from previous variables), the first term expresses that in a given distribution each degree of freedom (sub-observation) comes with an energy (change) that is proportional to the variance ${\sigma }^2=k_{B}T/2$, or for each Bernoulli trial in Figure 1, m changes by $ϵ$. On the other hand, the second term expresses how the total energy (distribution) is affected by a change in the total number of degrees of freedom, which corresponds to varying the number of Bernoulli trials (rows) in Figure 1. However, for each degree of freedom added or removed, m again changes by $+ϵ$ or $-ϵ$, respectively.

It hence becomes clear that each contribution to the first law of observation thermodynamics has to take the form $\delta m^2=\delta {\left(nϵ\right)}^2$ for both canonical (constant n and $ϵ$) and macro-canonical (constant $ϵ$) observation ensembles, while micro-canonical ensembles correspond to a single observation result bin having $\delta m^2=0$. For canonical ensembles with a fixed random error, one has $\delta m^2=\left(2n{ϵ}^2\right)\delta n$, with $2n{ϵ}^2$ representing the temperature as the double ensemble variance, while for a fixed number of degrees of freedom $\delta m^2=\left(2n^2ϵ\right)\delta ϵ$ still holds, capturing that the (quadratic) expectation value can vary due to a change in the system’s random error just as well.

Table 1. Statistical observation ensembles for the perfect equibinomial distribution with $Z\equiv N=2^n$. For different numbers of observations N, a correction factor $N/2^n$ applies.

Ensemble	Partition function	Interpretation
Micro-canonical	$W(n,k)=P_k=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)$	Single bin, $ϵ$ undefined
Canonical	$Z(n,ϵ)={\sum }_k{z}_{k}exp\left(-{\displaystyle \frac{{(2kϵ-nϵ)}^2}{2n{ϵ}^2}}\right)$	Sum over equibinomial bins
Macro-canonical	$Z\left(ϵ\right)={\sum }_{k,n}{z}_{k}exp\left(-{\displaystyle \frac{{(2kϵ-nϵ)}^2}{2n{ϵ}^2}}\right)$	Sum over equibinomial distributions

Table 1. Statistical observation ensembles for the perfect equibinomial distribution with $Z\equiv N=2^n$. For different numbers of observations N, a correction factor $N/2^n$ applies.

Ensemble	Partition function	Interpretation
Micro-canonical	$W(n,k)=P_k=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{k}}\right)$	Single bin, $ϵ$ undefined
Canonical	$Z(n,ϵ)={\sum }_k{z}_{k}exp\left(-{\displaystyle \frac{{(2kϵ-nϵ)}^2}{2n{ϵ}^2}}\right)$	Sum over equibinomial bins
Macro-canonical	$Z\left(ϵ\right)={\sum }_{k,n}{z}_{k}exp\left(-{\displaystyle \frac{{(2kϵ-nϵ)}^2}{2n{ϵ}^2}}\right)$	Sum over equibinomial distributions

4. Laws of Observation Thermodynamics

The above can be summarized into four laws of equibinomial observation statistics: (O1) An equibinomial observation distribution is fully characterized by its three state variables m, n, and $ϵ$, apart from an offset (additive bias) and scaling (N) that do not affect its equilibrium shape. (O2) The observation degrees of freedom n and random error per degree of freedom $ϵ$ are positive numbers. (O3) Apart from a constant offset, an observation ensemble’s expectation value m is fully determined by n and $ϵ$: $\delta m=\delta \left(nϵ\right)$ and therefore $\delta m^2=\left(2n{ϵ}^2\right)\delta n$ for constant $ϵ$. The double ensemble variance $2{\sigma }^2=2n{ϵ}^2$ is also called its temperature. Micro-canonical ensembles correspond to a single observation result bin having $\delta m^2=0$. (O4) Every observation (ensemble) provides information that cannot be undone: $\delta n\ge 0$ (and hence $\delta m\ge 0$). This law essentially captures information conservation. It can also be expressed as $\partial n/\partial N\ge 0$ or $\partial n/\partial t\ge 0$, reflecting that subsequent observations (in time) cannot decrease the combined number of degrees of freedom. The equality only holds for the (over-)sampling (equivalent to scaling) of a given distribution.

This summary, however, does not match the laws of classical thermodynamics in their typical format. A rephrasing of the above laws in line with the known laws of thermodynamics may sound as follows: (T0) When two or more equibinomial observation distributions with the same $T=2n{ϵ}^2$ are combined, the resulting distribution will (be over-sampled but still) have the same temperature. One may denote this as transitivity of the variance, see Table 2. Note that this actually follows from O3. T0 therefore has no equivalent above. (T1) Apart from a constant offset, an ensemble’s expectation value m is fully determined by n and $ϵ$ (again O3, as conservation of the expectation value). (T2) Information or entropy increase (cf. O4): Every observation (ensemble) provides information that cannot be undone, or $\partial n/\partial N\ge 0$. For $\delta n=0$, no information is added and hence T0 is obtained (over-sampling). For $n_{+}=n_1+n_2$ (independent observations of the same measurand), one has ${\sigma }_{+}^2={\sigma }_1^2+{\sigma }_2^2$. For correlated observations, ${\sigma }_{+}^2={\sigma }_1^2+{\sigma }_2^2+2\rho {\sigma }_1{\sigma }_2$ with $\rho$ the correlation coefficient according to Gaussian statistics, meaning that the combined variance can go down ($\rho <0$ for anti-correlated observations) despite the increasing number of degrees of freedom. (T3) For any $n>0$, not all observations can yield exactly the same outcome (intrinsic random observation error or $ϵ>0$), from O2. This could be read as follows: As soon as there is a degree of freedom (or even a fraction thereof, see next section), you obtain an observation distribution, as you have to make a choice between going left or right by $ϵ$ in the Bernoulli trajectory.

The laws of classical thermodynamics do not have a direct equivalent to O1, possibly because of it being just too obvious. It could be phrased as follows: A system in thermodynamic equilibrium is maximally statistically characterized by its state variables. Following Ginsberg’s theorem [5], O1 could say that there’s just a few game pieces to play with, see first row of Table 2. On the other hand, black hole mechanics, which can be phrased in terms of the laws of thermodynamics as well [6], does have an equivalent to O1. It is known as the no-hair theorem [7], but typically not related with black hole thermodynamics (BHT) as such in the literature. BHT makes a special case, however, as the relation between the three state variables mass M, surface gravity $\kappa$, and horizon area A is fixed at the Schwarzschild horizon, where the number of degrees of freedom is maximized ($N_E\equiv n$). This means M is exactly determined by $\kappa$ and A, and not up to a constant: $\left(4\pi G\right)M=\kappa A$, with $\kappa =c^4/4GM$, because $A=16\pi G^2{M}^2/c^4$ as well (from the Schwarzschild radius, with G the gravitational constant and c the speed of light). Actually, the laws of equilibrium thermodynamics apply more generally to all spherical surfaces that are concentric with the Schwarzschild horizon for Schwarzschild scenarios, as rigorously proven in [8] and fully formulated in [9]. However, the energy that is confined within a spacetime region is only determined up to a constant, now in analogy with the first law of thermodynamics, for Schwarzschild–de Sitter scenarios with a cosmological constant, while the second term in T1 that accounts for a changing number of constituents can incorporate black hole evaporation by Hawking radiation.

Table 2. Comparison table for different versions of the laws of thermodynamics. The (+) denotes the possibility of additional terms. Italics in the first row are own additions.

#	Observation thermodynamics	Classical thermodynamics [2]	Black hole thermodynamics [6]	Ginsberg’s theorem [5]
-	An equibinomial observation distribution is maximally statistically characterized by its state variables (m,n,$ϵ$) and the number of observations N.	A system in thermodynamic equilibrium is maximally statistically characterized by its state variables.	No-hair theorem.1	There’s just a few pieces.
T0	Transitivity of the variance: ${\sigma }_{n,ϵ}=$ constant, irrespective of N.	Transitivity of thermal equilibrium: $T_S=$ constant.	The surface gravity of a black hole is constant over the event horizon: ${\kappa }_A=$ constant.	There is a game.
T1	$\delta \left(m^2\right)=\delta {\left(nϵ\right)}^2$	$\delta E=T\delta S+{\mu }_E\delta N_E$ (+)	$\delta M=(\kappa /8\pi )\delta A$ (+)	You can’t win.
T2	If two observation distributions are combined, the joint d.o.f. at least equal the sum of the initial d.o.f.: $n_{+}\ge n_1+n_2$ or $\partial n/\partial t\ge 0$.	If two thermodynamic systems are combined, the joint entropy at least equals the sum of the initial entropies: $S_{+}\ge S_1+S_2$ or $\partial S/\partial t\ge 0$.	If two black holes coalesce, the area of the final event horizon is greater than the sum of the areas of the initial horizons: $A_{+}>A_1+A_2$ or $\partial A/\partial t\ge 0$.	You can’t break even.
T3	$n,ϵ>0$	$T>0$	$\kappa >0$	You have to play.

¹ The no-hair theorem was posed before black hole mechanics [7].

Table 2. Comparison table for different versions of the laws of thermodynamics. The (+) denotes the possibility of additional terms. Italics in the first row are own additions.

#	Observation thermodynamics	Classical thermodynamics [2]	Black hole thermodynamics [6]	Ginsberg’s theorem [5]
-	An equibinomial observation distribution is maximally statistically characterized by its state variables (m,n,$ϵ$) and the number of observations N.	A system in thermodynamic equilibrium is maximally statistically characterized by its state variables.	No-hair theorem.1	There’s just a few pieces.
T0	Transitivity of the variance: ${\sigma }_{n,ϵ}=$ constant, irrespective of N.	Transitivity of thermal equilibrium: $T_S=$ constant.	The surface gravity of a black hole is constant over the event horizon: ${\kappa }_A=$ constant.	There is a game.
T1	$\delta \left(m^2\right)=\delta {\left(nϵ\right)}^2$	$\delta E=T\delta S+{\mu }_E\delta N_E$ (+)	$\delta M=(\kappa /8\pi )\delta A$ (+)	You can’t win.
T2	If two observation distributions are combined, the joint d.o.f. at least equal the sum of the initial d.o.f.: $n_{+}\ge n_1+n_2$ or $\partial n/\partial t\ge 0$.	If two thermodynamic systems are combined, the joint entropy at least equals the sum of the initial entropies: $S_{+}\ge S_1+S_2$ or $\partial S/\partial t\ge 0$.	If two black holes coalesce, the area of the final event horizon is greater than the sum of the areas of the initial horizons: $A_{+}>A_1+A_2$ or $\partial A/\partial t\ge 0$.	You can’t break even.
T3	$n,ϵ>0$	$T>0$	$\kappa >0$	You have to play.

¹ The no-hair theorem was posed before black hole mechanics [7].

5. Demonstrative Applications

In practice, an observation distribution always has the number of observations N as a distribution scaling factor, but not necessarily $2^n$, as that would require fixing N beforehand while the degrees of freedom n are typically unknown a-priori. Nevertheless, it is obviously the (effective) noise error $ϵ$ within the observations that determines the variance of the Gaussian approximation as $n{ϵ}^2$, independent of N. Moreover, as an under-sampling or over-sampling affects all result bins by the same sampling factor, the ratio of the number of observations in the bin of width $2ϵ$ centered around m, denoted $N_m$, and the total number of observations N (actually within $2nϵ$, hence providing a means for outlier screening) is fixed:1

$${\displaystyle \frac{N_m}{N}}={\displaystyle \frac{\left(\genfrac{}{}{0pt}{}{n}{n/2}\right)}{2^n}}$$

(9)

from Equation (2) with $k=n/2$. Alternatively, the ratio between the fitting window width $2nϵ$ and the central bin width $2ϵ$ equals n directly. In combination with ${\sigma }^2=n{ϵ}^2$ from Gaussian fitting, this allows determining n and $ϵ$ for a distribution of repeated observations: One can update the bin width $2ϵ$ and bin number $B=n+1$ until $n{ϵ}^2$ complies with the variance ${\sigma }^2$ from Gaussian fitting. Given an observation distribution, however, one can in principle not distinguish between error contributions originating from the measurand or from the observation system consisting of measurement equipment, its operator(s), and observer(s). These contributions are dubbed (measurand) variability and (measurement) precision, respectively [10]. Consequently, each observation distribution is expected to combine (at least) two distinct (equi)binomial distributions—here assuming well-defined state variables [11]—although these can often be disentangled in practice, as demonstrated below.

Fully characterized distribution (precision assessment): Consider the (repeated) measurement of the length of a rod with a ruler that is longer than the rod. Each length measurement results from two independent readings from the ruler, one at each end of the rod, so $n=2$ as the number of independent error contributions to the overall assessment. Each reading introduces an error that amounts to half the ruler’s scale ${\Delta }_L$ at maximum, or $ϵ\equiv {\Delta }_L/2$. The measurement distribution will therefore have three bins of size ${\Delta }_L$ and a variance ${\sigma }^2=n{ϵ}^2={\Delta }_L^2/2$, irrespective of the amount of measurements made. Now imagine the ruler breaks into two parts that are just shorter than the rod. One hence has to put a mark on the rod, and perform $n=4$ readings (from the mark to each of the rod’s ends) for each rod length measurement (cf. Figure 1). Although the ensemble’s expectation value and random error remain the same, the new observation distribution will have an increased variance ${\sigma }^2={\Delta }_L^2$. Note that this example neglects other random uncertainty contributions (terms in the second law of thermodynamics), like thermal noise (which here cannot be distinguished between the rod and the ruler), ruler scaling imprecision, reading and recording errors, etc. If the observed variance is significantly larger than just described, one must conclude that these other uncertainty contributions cannot be neglected.

Archetype statistics (variability assessment): The Belgian polymath Adolphe Quételet introduced statistical methods to the social sciences in the 19th century. An archetype normal distribution from his hand covers the chest circumference of 5758 “Scottish militiamen” (1846). This distribution consists of $B=16$ bins, ranging from 33 to 48 inch, that contain 3, 18, 81, 185, 420, 749, 1073, 1079, 954, 658, 370, 92, 50, 21, 4, and 1 count(s), respectively. Fitting a Gaussian function yields ${\sigma }^2=n{ϵ}^2\approx 4.2$ (square inch), while $2nϵ=15$ (inch). Solving this set of equations results in $ϵ\approx 0.56$ (inch) and $n\approx 13.4$. Taking $n\equiv 13$ gives $ϵ\approx 0.57$ (inch). So by selecting one-inch ($2ϵ$) bins, Quételet came close to the exact binomial distribution, although it is under-sampled by about 30 % (as $1-5758/2^{13}$). Note that if he had selected eight two-inch bins containing 21, 266, 1169, 2152, 1612, 462, 71, and 5 counts, respectively, this under-resolving would indeed have been detected, as Gaussian fitting hence yields $ϵ\approx 0.59$ (inch) and $n\approx 12.8$, or essentially again $n\equiv 13$. In any case, for 1 inch result bins, the measurement precision, which is assumed to be of the order of 0.1 inch, can easily be neglected, so the distribution can be considered to be due to `natural variability’ only.

Under-constrained observations (removing precision): In under-constrained observations, one attempts to retrieve—usually as an ill-posed inverse problem—more measurands about a system than there are independent measurements of that system available. This can only be achieved by making prior assumptions about the system that add information to the measurement(s). Each measurand m as such is effectively a function $\mathcal{F}$ of both the measurement(s) of the true state t of the system and the prior assumption(s) p: $m\equiv \mathcal{F}(t,p)$, which combines the degrees of freedom n and error $ϵ$ originating from the measurement, prior, and retrieval function $\mathcal{F}$. Knowledge of $\mathcal{F}$, e.g., as in remote sounding experiments, enables a posterior assessment of the prior-free observation error, originating from the measurement(s) and retrieval only: Applying $\mathcal{F}$ to an independent reference observation r of the system and taking the difference yields a new measurand (i.e., the expectation value of the difference between m and the reference as observed by the retrieval system) $m^{\prime }=\mathcal{F}(t,p)-\mathcal{F}(r,p)\equiv {\mathcal{F}}^{\prime }(t-r)$ that is effectively independent of p [12]. The availability of an uncertainty assessment of the reference(s) hence permits determining the degrees of freedom and error of the prior-free observation.

Comparing observations (removing variability): Measurements of natural variables can most often not be repeated, think of the atmospheric composition or astronomical events, but can be simultaneously (coincident in time, collocated in space) reproduced by several instruments instead. Taking the first of two independent observation ensembles as a `known’ reference $(m_1,{\sigma }_1)$, the bias $\Delta m$ and variance ${\sigma }_2^2={\sigma }_{1,2}^2-{\sigma }_1^2$ of the second can be obtained straightforwardly, with ${\sigma }_{1,2}^2$ the variance of the difference distribution that lacks measurand variability (approximately, cf. the previous case). Once three (or more) independent observation ensembles of multiple system states are available, triple comparisons allow an exact assessment of the measurement variances involved [10]: ${\sigma }_{1,2}^2={\sigma }_1^2+{\sigma }_2^2$, ${\sigma }_{2,3}^2={\sigma }_2^2+{\sigma }_3^2$, and ${\sigma }_{3,1}^2={\sigma }_3^2+{\sigma }_1^2$, resulting in three identical equations for three unknowns, called a three-cornered hat, here in its most simple form. Whenever relevant, however, imperfect coincidences and collocations between the observation systems, called spatiotemporal representativeness differences, have to be taken into account in these expressions as well [13].

Temperature measurements (as variability or not): The temperature or variance of a distribution can be a physical measurand under study as such, with $T=2n{ϵ}^2/k_B$. A temperature measurement can therefore be obtained using two distinct approaches: One either performs highly-resolved (kinetic) energy measurements on a system and fits a variance to their distribution, or one calibrates a system that changes with temperature using predefined temperature values. In the first approach, one has to sum the (kinetic) energy of all n particles involved, which hence equal the system’s degrees of freedom. The variability or `error’ $ϵ$ is determined by the Maxwell–Boltzmann distribution of the particles’ velocities (which actually is a ${\chi }^2$ distribution of three normally distributed variables). The most well-known objectification of the second approach consists of putting a liquid that linearly expands with temperature into a transparent tube, and marking an equidistant scale on the tube between two calibration levels, e.g., of freezing and cooking water (defined as 0 °C and 100 °C, respectively, for the Celsius scale). The measurand as such becomes a scale reading with a distribution shape that is independent of the temperature quantity under study! Rather, the above notes on the fully characterized distribution of ruler readings apply. It is clear that in practice this second approach is preferred, although it critically depends on the preceding calibration operation(s).

6. Discussion

We have demonstrated that the common four laws of classical thermodynamics apply to equibinomial observation statistics, with the partition function straightforwardly representing the number of independent observations N as the distribution integral. Generalizations, especially of the partition function definition, to quantum-mechanical observed or observation systems should apply, but require more study, both in terms of mathematical rigor and physical interpretation. Besides, one can expect non-equilibrium thermodynamics to relate with shifting state variables, while asymmetric or generally non-Gaussian observation distributions result from errors mixing both random and systematic components (unofficially dubbed “headache errors” for obvious reasons [3]). E.g., apart from a constant offset as an additive bias, the ensemble mean m can also be shifted by a multiplicative bias component that is equivalent to $ϵ\propto k$.

We have proposed a reformulation of the laws of thermodynamics that summarizes the definition and constraints of equibinomial observation ensembles, matching them with classical thermodynamics terminology. As such, this work provides a direct microscopic interpretation for the entropy, which is different from what has been proposed earlier [14]. Moreover, each term in the first law of thermodynamics is shown to be a variation on $\delta m^2=\delta {\left(nϵ\right)}^2$ for both canonical and macro-canonical observation ensembles, while micro-canonical ensembles correspond to a single observation result bin having $\delta m^2=0$. The three state variables m, n, and $ϵ$ maximally statistically characterize the observation ensemble, and correspond to its expectation value (including systematic error as constant offset), degrees of freedom, and random error, respectively. A perfect equibinomial sampling is only obtained for $N\equiv 2^n$. Expanding on Ginsberg’s theorem, we could conclude that “with the existence of only few observable pieces, nature plays a hiding game that you just cannot get out of, and also cannot win.”