A New Index for Measuring the Non‐Uniformity of a Probability Distribution

2. The Proposed Distribution Non-Uniformity Index (DNUI)

2.1. Discrete Cases

Consider a discrete random variable X with probability mass function (PMF)$P\left(x\right)$ and n possible outcomes. Let $X_U$ denote the uniform distribution with the same possible outcomes, so that $P_U\left(x\right)$ $={\displaystyle \frac{1}{n}}$ for all x. We use this uniform distribution as the baseline for measuring the non-uniformity of the distribution of X.

The difference between $P\left(x\right)$ and $P_U\left(x\right)$ is given by

$${\Delta }_{P(x)}=P(x)-P_U(x)=P(x)-{\displaystyle \frac{1}{n}}.$$

(4)

Thus, $P\left(x\right)$can be written as

$$P(x)={\Delta }_{P\left(x\right)}+{\displaystyle \frac{1}{n}}.$$

(5)

Taking squares on both sides of Eq. (5) yields

$${P\left(x\right)}^2={\Delta }_{P\left(x\right)}^2+{\displaystyle \frac{2}{n}}{\Delta }_{P\left(x\right)}+{\displaystyle \frac{1}{n^2}}.$$

(6)

Then taking the expectation on both sides of Eq. (6) yields

$${\mathrm{E}\left[P\right(x)}^2]=\mathrm{E}({\Delta }_{P\left(x\right)}^2)+{\displaystyle \frac{2}{n}}\mathrm{E}({\Delta }_{P\left(x\right)})+{\displaystyle \frac{1}{n^2}}={\mathsf{\omega }}_{P\left(x\right)}^2+{\displaystyle \frac{1}{n^2}},$$

(7)

where ${\mathsf{\omega }}_{P\left(x\right)}^2$ is called the total variance and ${\omega }_{P\left(x\right)}$ is called the total deviation

$${\omega }_{P\left(x\right)}=\sqrt{\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)+{\displaystyle \frac{2}{n}}\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)}.$$

(8)

where $\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)$ is the variance of $P\left(x\right)$ relative to $P_U\left(x\right)$, given by

$$\mathrm{E}({\Delta }_{P\left(x\right)}^2)=\mathrm{E}\{{\left[P\right(x)-P_U(x\left)\right]}^2\}={\sum }_{i=1}^n{P\left(x_i\right)[P\left(x_i\right)-{\displaystyle \frac{1}{n}}]}^2$$

(9)

$\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)$ is the bias of $P\left(x\right)$ relative to $P_U\left(x\right)$, given by

$$\mathrm{E}({\Delta }_{P\left(x\right)})=\mathrm{E}[P(x)-P_U(x)]={\sum }_{i=1}^n{P\left(x_i\right)}^2-{\displaystyle \frac{1}{n}}=\beta \left(X\right)-{\displaystyle \frac{1}{n}}$$

(10)

where $\beta \left(X\right)$ is the informity of X. The concept of informity is introduced in Huang (2025). The informity of a discrete random variable is defined as the expectation of its PMF. The informity of the baseline uniform distribution $X_U$, is $\beta \left(X_U\right)=P_U\left(x\right)={\displaystyle \frac{1}{n}}$. Thus, $\mathrm{E}(\Delta )$ is the seen as the informity bias.

Definition 1.

The proposed distribution non-uniformity index (DNUI) (denoted by $\rho \left(X\right)$) is given by

$$\rho \left(X\right)={\displaystyle \frac{{\omega }_{P\left(x\right)}}{\sqrt{{\mathrm{E}\left[P\right(x)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}\left[P\right(x)}^2]-\frac{1}{n^2}}{{\mathrm{E}\left[P\right(x)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)+\frac{2}{n}\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)}{\mathrm{E}\left({\Delta }_{P\left(x\right)}^2\right)+\frac{2}{n}\mathrm{E}\left({\Delta }_{P\left(x\right)}\right)+\frac{1}{n^2}}}} .$$

(11)

$\sqrt{{\mathrm{E}\left[P\right(x)}^2]}$ is the root mean square (RMS) of $P\left(x\right)$ and ${\mathrm{E}\left[P\right(x)}^2]$ is the second moment of the probability $P\left(x\right)$, given by

$${\mathrm{E}\left[P\right(x)}^2]={\sum }_{i=1}^n{P\left(x_i\right)P\left(x_i\right)}^2={\sum }_{i=1}^n{P\left(x_i\right)}^3.$$

(12)

2.2. Continuous Cases

Consider the continuous random variable Y with probability density function (PDF) $p\left(y\right)$ defined on an unbounded support (e.g., $(-\infty ,\infty$)). Since no baseline uniform distribution exists for an unbounded support, we cannot measure the non-uniformity of the entire distribution of Y. Instead, we focus on a partial distribution of Y on a fixed interval ${[y}_1,y_2]$. According to Rajaram et al. (2024a), the PDF of the partial distribution is given by renormalization of the original PDF

$$p\text{'}\left(y\right)={\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}},$$

(13)

where $P_{{(y}_1,y_2)}=\underset{y_1}{\overset{y_2}{\int }}p\left(y\right)dy$, which is the coverage probability of the interval ${[y}_1,y_2]$.

Let $Y_U$ denote the uniform distribution on ${[y}_1,y_2]$ with density $p_U\left(y\right)$ $={\displaystyle \frac{1}{{{(y}_2-y}_1)}}$. We use this uniform distribution as the baseline for measuring the non-uniformity of the partial distribution.

Similar to the discrete cases, the difference between $p\text{'}\left(y\right)$ and $p_U\left(y\right)$ is given by

$${\Delta }_{p\text{'}\left(y\right)}=p\text{'}\left(y\right)-p_U(y)=p\text{'}\left(y\right)-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}.$$

(14)

Thus, $p\text{'}\left(y\right)$ can be written as

$$p\text{'}\left(y\right)={\Delta }_{p\text{'}\left(y\right)}+{\displaystyle \frac{1}{{{(y}_2-y}_1)}}.$$

(15)

Taking squares on both sides of Eq. (15) yields

$${p\text{'}\left(y\right)}^2={\Delta }_{p\text{'}\left(y\right)}^2+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}{\Delta }_{p\text{'}\left(y\right)}+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}.$$

(16)

Then taking the expectation on both sides of Eq. (16) yields

$${\mathrm{E}[p\text{'}\left(y\right)}^2]=\mathrm{E}({\Delta }_{p\text{'}\left(y\right)}^2)+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}\mathrm{E}({\Delta }_{p\text{'}\left(y\right)})+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}={\mathsf{\omega }}_{p\text{'}\left(y\right)}^2+{\displaystyle \frac{1}{{{{(y}_2-y}_1)}^2}}.$$

(17)

The total deviation ${\omega }_{p\text{'}\left(y\right)}$ is given by

$${\omega }_{p\text{'}\left(y\right)}=\sqrt{\mathrm{E}({\Delta }_{p\text{'}\left(y\right)}^2)+{\displaystyle \frac{2}{{{(y}_2-y}_1)}}\mathrm{E}({\Delta }_{p\text{'}\left(y\right)})}.$$

(18)

where $\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}^2\right)$ is the variance of $p\left(y\right)$ relative to $p_U\left(y\right)$ on ${[y}_1,y_2]$, given by

$$\mathrm{E}({\Delta }_{p\text{'}\left(y\right)}^2)=\mathrm{E}\{{[p\text{'}\left(y\right)-p_U(y\left)\right]}^2\}=\underset{y_1}{\overset{y_2}{\int }}{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}{\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}\right]}^{2}dy,$$

(19)

$\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}\right)$ is the bias of $p\text{'}\left(y\right)$ relative to $p_U\left(y\right)$, given by

$$\mathrm{E}\left({\Delta }_{p^{\text{'}\left(y\right)}}\right)=\mathrm{E}\left[p^{\text{'}\left(y\right)}-p_U\left(y\right)\right]=\underset{y_1}{\overset{y_2}{\int }}{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}-{\displaystyle \frac{1}{{{(y}_2-y}_1)}}\right]dy.$$

(20)

Definition 2.

The proposed DNUI for the partial distribution on ${[y}_1,y_2]$ (denoted by $\rho (y_1,y_2)$) is given by

$$\rho \left(y_1,y_2\right)={\displaystyle \frac{{\omega }_{p\text{'}\left(y\right)}}{\sqrt{{\mathrm{E}[p\text{'}\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}[p\text{'}\left(y\right)}^2]-\frac{1}{{{{(y}_2-y}_1)}^2}}{{\mathrm{E}[p\text{'}\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}^2\right)+\frac{2}{{{(y}_2-y}_1)}\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}\right)}{\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}^2\right)+\frac{2}{{{(y}_2-y}_1)}\mathrm{E}\left({\Delta }_{p\text{'}\left(y\right)}\right)+\frac{1}{{{{(y}_2-y}_1)}^2}}}}.$$

(21)

where ${\mathrm{E}[p\text{'}\left(y\right)}^2]$ is the second moment of the probability density $p\text{'}\left(y\right)$, given by

$${\mathrm{E}[p\text{'}\left(y\right)}^2]=\underset{y_1}{\overset{y_2}{\int }}{\left[{\displaystyle \frac{p\left(y\right)}{P_{{(y}_1,y_2)}}}\right]}^{3}dy.$$

(22)

Definition 3.

If the continuous distribution is defined on the fixed support $[-a,a]$, $P_{(-a,a)}=1$ and ${{(y}_2-y}_1)=2a$, the proposed DNUI for the entire distribution of Y (denoted by $\rho \left(Y\right)$ is given by

$$\rho \left(Y\right)={\displaystyle \frac{{\omega }_{p\left(y\right)}}{\sqrt{{\mathrm{E}[p\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{{\mathrm{E}[p\left(y\right)}^2]-\frac{1}{4a^2}}{{\mathrm{E}[p\left(y\right)}^2]}}}=\sqrt{{\displaystyle \frac{\mathrm{E}\left({\Delta }_{p\left(y\right)}^2\right)+\frac{1}{a}\mathrm{E}\left({\Delta }_{p\left(y\right)}\right)}{\mathrm{E}\left({\Delta }_{p\left(y\right)}^2\right)+\frac{1}{a}\mathrm{E}\left({\Delta }_{p\left(y\right)}\right)+\frac{1}{4a^2}}}},$$

(23)

where ${\mathrm{E}[p\left(y\right)}^2]$ is the second moment of the probability density $p\left(y\right)$, given by

$${\mathrm{E}[p\left(y\right)}^2]=\underset{-a}{\overset{a}{\int }}{p\left(y\right)}^{3}dy,$$

(24)

the variance $\mathrm{E}\left({\Delta }_{p\left(y\right)}^2\right)$ is given by

$$\begin{array}{c}\mathrm{E}({\Delta }_{p\left(y\right)}^2)=\underset{-a}{\overset{a}{\int }}{p\left(y\right)\left[p\left(y\right)-{\displaystyle \frac{1}{2a}}\right]}^{2}dy\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}={\int }_{-a}^a{p\left(y\right)}^{3}dy-{\displaystyle \frac{1}{a}}{\int }_{-a}^a{p\left(y\right)}^{2}dy+{\displaystyle \frac{1}{{4a}^2}}\end{array},$$

(25)

and the bias $\mathrm{E}\left({\Delta }_{p\left(y\right)}\right)$ is given by

$$\mathrm{E}({\Delta }_{p\left(y\right)})=\underset{-a}{\overset{a}{\int }}p\left(y\right)[p\left(y\right)-{\displaystyle \frac{1}{2a}}]dy={\int }_{-a}^a{p\left(y\right)}^{2}dy-{\displaystyle \frac{1}{2a}}.$$

(26)

3. Examples

3.1. Coin-Tossing

Consider tossing a coin, which is a simplest two-state probability system: {X; P(x)}={head, tail; P(head), P(tail)}, where $P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)=1-P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)$. The DNUI for this system is given by

$$\rho \left(X\right)=\sqrt{{\displaystyle \frac{{\mathrm{E}\left[P\right(x)}^2]-\frac{1}{2^2}}{{\mathrm{E}\left[P\right(x)}^2]}}} .$$

(27)

where the second moment ${\mathrm{E}\left[P\right(x)}^2]$ can be calculated as

$${\mathrm{E}\left[P\right(x)}^2]={\left[P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)\right]}^3+{\left[P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)\right]}^3.$$

(28)

Figure 1 shows the DNUI of the coin-tossing system as a function of the bias represented by $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)$. The two evenness measures: Simpson’s evenness $E_{S2}$ and Buzas & Gibson’s evenness $E_{BG}$ are also shown in the figure for comparison.

As shown in Figure 1, when the coin is fair, i.e., $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)=P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)=0.5$, the DNUI is 0, and Simpson’s evenness $E_{S2}$ and Buzas & Gibson’s evenness $E_{BG}$ are both 1, indicating perfect uniformity. When a coin is highly biased towards the tail or head, the DNUI increases, while both Simpson’s evenness $E_{S2}$ and Buzas & Gibson’s evenness $E_{BG}$ decrease. In the most extreme case where $P\left(\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}\right)=1$ or $P\left(\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}\right)=1$, the DNUI is the highest: $\rho \left(X\right)=0.866$, indicating the highest non-uniformity or unevenness. However, although $E_{S2}$ and $E_{BG}$ are the lowest, i.e., $E_{S2}=E_{BG}=0.5$, this value does not reflect the high degree of unevenness. This confirms the argument of Gregorius and Gillet (2021): “… measures of diversity generally do not produce characteristic values that are associated with states of complete unevenness.”

3.2. Three Frequency Data Series

JJC (2024) posted a question on Cross Validated about quantifying distribution non-uniformity. He supplied three frequency datasets (Series A, B, and C), each containing 10 values (Table 1). Visually, Series A is almost perfectly uniform, Series B is nearly uniform, and Series C is heavily skewed by a single outlier (0.6). Table 1 lists these datasets alongside their corresponding DNUI, $E_{S2}$, and $E_{BG}$ values.

From Table 1, we can see that the DNUI value of Series A is 0.1864, confirming its high uniformity, and the DNUI value of Series B is 0.2499, indicating its close uniformity. In contrast, the DNUI value of Series C is 0.9767 (close to 1), indicating its extreme non-uniformity. On the other hand, the $E_{S2}$ and $E_{BG}$ values for Series A and Series B are close to 1, indicating high uniformity. The $E_{S2}$ value for Series C is 0.2625, indicating the high degree of unevenness. However, the $E_{BG}$ value for Series C is 0.4545, not reflecting the high degree of unevenness.

3.3. Five Continuous Distributions with Fixed Support $[-a,a]$

Consider five continuous distributions with fixed support $[-a,a]$: uniform, triangular, quadratic, raised cosine, and half-cosine. Table 2 summarizes their PDFs, variances, biases, second moments, and DNUIs.

As can be seen from Table 2, the DNUI is independent of the scale parameter a, which is a desirable property for the measures of distribution non-uniformity. The DNUI value for the uniform distribution is 0, which is by definition. The DNUI for the other four distribution are all greater than 0.9, indicating these four distributions are highly non-uniform. This result is consistent with our common sense. In addition, for the raised cosine distribution, the DNUI is the highest among the five distributions, indicating that it is the greatest non-uniformity.

3.4. Exponential Distribution

The PDF of the exponential distribution with support $[0,\infty )$ is

$$p\left(y\right)=\lambda e^{-\lambda y},$$

(29)

where $\lambda$ is the shape parameter.

We consider the partial exponential distribution on the interval [$0,b]$ (i.e., $y_1=0$ and $y_2=b$), where b is the length of the interval. The informity of the baseline uniform distribution on [$0,b]$ is 1/$b$. Thus, the DNUI for the partial exponential distribution on [$0,b]$ is given by

$$\rho \left(0,b\right)=\sqrt{{\displaystyle \frac{{\mathrm{E}[p\text{'}\left(y\right)}^2]-\frac{1}{b^2}}{{\mathrm{E}[p\text{'}\left(y\right)}^2]}}},$$

(30)

where the second moment ${\mathrm{E}[p\text{'}\left(y\right)}^2]$ is given by

$${\mathrm{E}[p\text{'}\left(y\right)}^2]={\displaystyle \frac{1}{{{[P}_{(0,b)}]}^3}}\underset{0}{\overset{b}{\int }}{p\left(y\right)}^{3}dy.$$

(31)

The coverage probability of the interval [$0,b]$ is given by

$$P_{(0,b)}=\underset{0}{\overset{b}{\int }}\lambda \mathrm{e}\mathrm{x}\mathrm{p}(-\lambda y)dy=1-e^{-\lambda b}.$$

(32)

The integral $\underset{0}{\overset{b}{\int }}{p\left(y\right)}^{3}dy$ can be solved as

$$\underset{0}{\overset{b}{\int }}{p\left(y\right)}^{3}dy=\underset{0}{\overset{b}{\int }}{\left\{\lambda e^{-\lambda y})\right\}}^{3}dy={\lambda }^3\underset{0}{\overset{b}{\int }}{e}^{-3\lambda y}dy={\displaystyle \frac{{\lambda }^2}{3}}(1-e^{-3\lambda b}).$$

(33)

Figure 2 shows the plots of the DNUI for the partial exponential distribution with $\lambda =1$ as a function of the interval length b. As the interval length b increases, the DNUI increases, indicating that the non-uniformity of the partial distribution increases with a longer interval.

4. Discussion and Conclusions

Unlike the degree of inequality (DOI) proposed by Rajaram et al. (2024a, b) or the diversity-based evenness measures $E_{S2}$ and $E_{BG}$ (which are not distance measures), the proposed DNUI is a standardized distance measure based on the total deviation defined by Eq. (8). It is important to note that the total deviation consists of two components: variance and bias, both of which are relative to the baseline uniform distribution.

The proposed DNUI ranges from 0 to 1, where 0 indicates perfect uniformity and 1 indicates extreme non-uniformity. Lower DNUI values (close to 0) indicate a lower degree of non-uniformity or a flatter distribution, while larger DNUI (close to 1) values correspond to a higher degree of non-uniformity or a more uneven distribution. However, there is no universal standard for defining the level (or degree) of non-uniformity of a distribution. Based on the examples presented, we tentatively propose to use DNUI values of 0.4, 0.7, and 0.85 (corresponding to low, moderate, and high non-uniformity, respectively) to assess the levels of non-uniformity of a distribution.

It is important to note that the DNUI depends solely on the probabilities and not on the outcomes (or scores) or the specific order of outcomes. This property can be demonstrated with the frequency data from Series C in subsection 3.1: {0.03, 0.02, 0.6, 0.02, 0.03, 0.07, 0.06, 0.05, 0.05, 0.07}. If we swap the order of the second and third frequency values, the DNUI values remains unchanged. This invariance means that different distributions can yield the same DNUI value. In other words, the DNUI is not a one-to-one function of the distribution; it can “collapses” different distributions into the same DNUI value. This is similar to how different distributions can have the same mean or variance.

In summary, the proposed distribution non-uniformity index (DNUI) provides effective measures for assessing the non-uniformity of a probability distribution. It is applicable to any distribution with fixed support and can also be used for a partial distribution defined on a fixed interval, even when the entire distribution has unbounded support. The presented examples have demonstrated the effectiveness of the proposed DNUI in capturing and quantifying distribution non-uniformity.