Inforpower: Quantifying the Informational Power of Probability Distributions

1. Introduction

Consider a continuous random variable Y with probability density function (PDF) $p\left(Y\right)$. The mean of Y is

$${\mu }_M={\mathbb{E}}_p\left[Y\right]=\int yp\left(y\right)dy,$$

(1)

which is the expected value under $p\left(Y\right)$. That is, ${\mu }_M$ is the weighted average of all possible values the random variable Y can take, where the weights are determined by the PDF $p\left(Y\right)$. It is often interpreted as the center of mass of the distribution.

The variance of Y is

$$Var\left(Y\right)={\mathbb{E}}_p\left[{\left(Y-{\mu }_M\right)}^2\right]=\int {\left(y-{\mu }_M\right)}^{2}p\left(y\right)dy={\mathbb{E}}_p\left[Y^2\right]-{\mu }_M^2,$$

(2)

which quantifies the spread or dispersion of Y about its mean.

In information theory, Shannon entropy measures the uncertainty represented by a probability distribution. The continuous (i.e., differential) entropy is defined as [1]

$$H\left(p\right)=-\int p\left(y\right)\log p\left(y\right)dy=-{\mathbb{E}}_p\left[\log p\left(Y\right)\right].$$

(3)

Recently, Huang [2] introduced informity theory, in which, informity measures the certainty represented by a probability distribution and acts as the complementary dual of entropy [3]. The continuous informity is defined as [2]

$$\beta \left(p\right)=\int {p\left(y\right)}^{2}dy=\int \left[p\left(y\right)\right]p\left(y\right)dy={\mathbb{E}}_p\left[p\left(Y\right)\right].$$

(4)

In many practical applications, a PDF describes a system comprising a signal component and a noise component. The mean ${\mu }_M$ naturally quantifies signal strength (or systematic effect), whereas variance $Var\left(Y\right)$, entropy $H\left(p\right)$, and informity $\beta \left(p\right)$ each characterize different aspects of noise or uncertainty. Apparently, these measures quantifying the signal and noise separately. Although these quantities separately capture signal and noise, they do not account for their interaction, which is often critical when assessing the overall performance or informational power of a system.

This paper introduces a new measure termed inforpower that explicitly incorporates the interplay between signal and noise, thereby providing a more comprehensive quantification of a distribution’s informational power. The remainder of the paper is organized as follows: Section 2 formally defines inforpower; Section 3 interprets its meaning and properties; Section 4 presents inforpower for ten well-known continuous distributions; Section 5 develops an inforpower-based metric and illustrates its use in practical applications; Section 6 offers discussion and conclusions.

2. Definition and Basic Properties of Inforpower

From an information-theoretic perspective known as the information-weight interpretation of probability [4], a realized value y of a random variable represents information (a score or signal), while the probability P represents the information weight or the probability density $p\left(y\right)$ represents the information density, attached to that value. Thus, in continuous distributions, $p\left(y\right)$ quantifies the certainty or noise level associated with the signal y. The product $yp\left(y\right)$ therefore captures the effective information: the original signal (or score) weighted (and potentially diluted) by the degree of certainty in the distribution.

Definition 1 (Effective Information)

Let Z be the random variable representing effective information

$$Z=Yp\left(Y\right).$$

(5)

For any realized value y, the corresponding effective information is $z=yp\left(y\right)$.

Definition 2 (Inforpower)

Let $\varphi$ denote inforpower of a continuous probability distribution. It is defined as the expected value of the effective information Z

$$\varphi ={\mathbb{E}}_p\left[Z\right]=\int zp\left(y\right)dy=\int {yp\left(y\right)}^{2}dy.$$

(6)

Thus, inforpower is the first moment of the density-weighted signal. The following theorems reveal important structural properties of inforpower.

Theorem 1. If $p\left(Y\right)$ is symmetric about its location parameter a, then,

$$\varphi =a\beta \left(p\right)$$

(7)

where $\beta \left(p\right)=\int {p\left(y\right)}^{2}dy$ is the continuous informity.

Theorem 2. If $p\left(Y\right)$ is supported on [0, ∞) with no explicit location parameter (i.e., location fixed at zero) and belongs to a scale family, then $\varphi$ is scale-invariant.

Proof of Theorems 1 and 2. Consider a generalized location-scale family of probability distributions with PDF

$$p\left(y\right)={\displaystyle \frac{1}{b}}f\left({\displaystyle \frac{y-a}{b}},c\right), b>0,$$

(8)

where a is the location parameter, b is the scale parameter, and c is the shape parameter.

The inforpower is:

$$\varphi =\int yp{\left(y\right)}^{2}dy=\int y{\displaystyle \frac{1}{b^2}}f{\left({\displaystyle \frac{y-a}{b}},c\right)}^{2}dy.$$

(9)

Make the change of variable $u=(y-a)/b$, then $dy=bdu$. Substituting yields

$$\varphi ={\displaystyle \frac{1}{b}}\int \left(bu+a\right)f{\left(u,c\right)}^{2}du=\int uf{\left(u,c\right)}^{2}du+{\displaystyle \frac{a}{b}}\int f{\left(u,c\right)}^{2}du.$$

(10)

Case 1 (Symmetric distributions, proving Theorem 1): When the distribution is symmetric about a, the standardized density $f\left(u,c\right)$ (or simply $f\left(u\right)$ if no shape parameter is present) is symmetric about zero. Consequently, $f{\left(u,c\right)}^2$ is even, and $uf{\left(u,c\right)}^2$ is odd, so

$$\int uf{\left(u,c\right)}^{2}du=0.$$

(11)

Thus,

$$\varphi ={\displaystyle \frac{a}{b}}\int f{\left(u,c\right)}^{2}du=a\int {p\left(y\right)}^{2}dy=a\beta \left(p\right).$$

(12)

This establishes Theorem 1.

Case 2 (Nonnegative support, no location shift, proving Theorem 2): When the support is $b\ge 0$ and the location is fixed at a = 0 (common for many positive distributions such as exponential, gamma, lognormal, Pareto, etc.), the term involving a/b vanishes. Then,

$$\varphi =\int uf{\left(u,c\right)}^{2}du,$$

(13)

which depends only on the shape parameter c and is independent of the scale parameter b. If the distribution has no shape parameter (e.g., standard exponential distribution), $\varphi$ is an absolute constant. This proves scale invariance and establishes Theorem 2. ∎

3. Interpretation of Inforpower

3.1. Inforpower as a Measure of Informational Power

Inforpower $\varphi$ is the expected value of the effective information $Z=Yp\left(Y\right)$. Therefore, it quantifies the informational power (effective signal or performance) of the system described by the distribution $p$. To illustrate this, consider the normal distribution $Y~N\left(\mu ,{\sigma }^2\right)$. Its inforpower is

$$\varphi ={\displaystyle \frac{\mu }{2\sigma \sqrt{\pi }}}\propto {\displaystyle \frac{\mu }{\sigma }}.$$

(14)

Here $\mu$ represents the signal strength and $\sigma$ represents the noise level. Therefore, $\mu /\sigma$ is the classical signal-to-noise ratio (SNR), a standard performance metric in signal processing, detection theory, and measurement science. Thus, inforpower is proportional to SNR and directly measures the degree to which the original signal is preserved after being degraded by noise. In this sense, $\varphi$ serves as a natural and distribution-based indicator of a system’s informational power.

3.2. Inforpower as the Standardized Center of Information Energy

Inforpower admits an elegant energy-based reinterpretation. Rewrite $\varphi$ as

$$\varphi =\beta \left(p\right)\int y{\displaystyle \frac{{p\left(y\right)}^2}{\beta \left(p\right)}}dy=\beta \left(p\right)\int y{\displaystyle \frac{{p\left(y\right)}^2}{\int {p\left(y\right)}^{2}dy}}dy,$$

(15)

Define the normalized function

$$g\left(y\right)={\displaystyle \frac{{p\left(y\right)}^2}{\int {p\left(y\right)}^{2}dy}}={\displaystyle \frac{{p\left(y\right)}^2}{\beta \left(p\right)}}.$$

(16)

The function $g\left(y\right)$ is a valid probability density function, called the information-energy density function. This naming is consistent with Onicescu [5], who named the quantity $\int {p\left(y\right)}^{2}dy$ as the information energy.

In physical analogy, on one hand, $p\left(y\right)$ behaves like mass density (amplitude), $p\left(y\right)dy$ is the differential mass element, and the ordinary mean ${\mu }_M={\mathbb{E}}_p\left(Y\right)$ is the center of mass. On the other hand, ${p\left(y\right)}^2$ behaves like energy density (energy ∝ amplitude²), and $g\left(y\right)dy$ is the differential information-energy element. Thus, the expected value of Y under $g\left(y\right)$ is the center of information energy

$${{\mu }_E =\mathbb{E}}_g\left(Y\right)=\int yg\left(y\right)dy={\displaystyle \frac{\varphi }{\beta \left(p\right)}}.$$

(17)

Consequently, inforpower can be expressed as the product of informity and information-energy center

$$\varphi =\beta \left(p\right) {\mu }_E$$

(18)

Dimensional analysis confirms the following consistency: $\beta \left(p\right)$ has dimension [y]^-1, ${\mu }_E$ has dimension [y], and their product $\varphi$ is dimensionless, which matches its interpretation as a pure performance index. Thus, inforpower $\varphi$ is the standardized center of information energy. This dual view (the SNR interpretation in Section 3.1 and the energy-center interpretation here) reinforces $\varphi$ as a powerful, unified measure of informational power or performance of the system.

For symmetric distributions, ${\mu }_E={\mu }_M$ (the ordinary mean). For skewed distributions, however, ${\mu }_E$ deviates from ${\mu }_M$, typically shifting toward the higher-density regions that contribute disproportionately to the energy. Because high-density regions reflect greater certainty, ${\mu }_E$ can be interpreted as the effective signal strength, i.e., the location where the signal is most reliably concentrated under the energy perspective.

The concept of information-energy center extends naturally to discrete distributions; details are provided in the Appendix.

4. Inforpower for Ten Well-Known Continuous Distributions

Table 1 presents the probability density function (PDF), informity $\beta \left(p\right)$, inforpower $\varphi$, ordinary mean (center of mass) ${\mu }_M$, and center of information energy ${\mu }_E$ for ten widely used continuous distributions. The first four (normal, Laplace, logistic, and Cauchy) have support on (-∞, ∞). The next five (exponential, Gamma, Rayleigh, Weibull, and lognormal) have support on [0, ∞). The Pareto distribution has support $\left[y_m,\infty \right)$, where $y_m$ > 0 is the minimum possible value.

As Table 1 shows, for the symmetric normal, Laplace, and logistic distributions, $\varphi =\mu \beta \left(p\right)$. For the symmetric Cauchy distribution (location parameter $y_0$), $\varphi =y_0\beta \left(p\right)$. These results confirm Theorem 1: when the distribution is symmetric about its location parameter a, inforpower reduces to $\varphi =a\beta \left(p\right)$.

For the remaining six distributions, which are asymmetric and supported on [0, ∞) or $\left[y_m,\infty \right)$, the expressions for $\varphi$ contain no scale parameter. Specifically, for the exponential and Rayleigh distributions (one-parameter families with fixed shape), $\varphi$ is a numerical constant; for the Gamma, Weibull, lognormal, and Pareto distributions, $\varphi$ depends solely on the shape parameter and is invariant to scale parameter. These observations fully confirm Theorem 2.

Figure 1, Figure 2, Figure 3, Figure 4 and Figure 5 display, for the exponential, Gamma, Rayleigh, Weibull, and lognormal distributions respectively, both the probability density function $p\left(y\right)$ and the corresponding information-energy density function $g\left(y\right)={p\left(y\right)}^2/\beta \left(p\right)$. The figures also mark the mode, median, ordinary mean ${\mu }_M$ (center of mass), and energy center ${\mu }_E$.

As expected, the information-energy density function $g\left(y\right)$ is visibly sharper and more concentrated around regions of high probability density than $p\left(y\right)$ itself, because squaring the density amplifies peaks and suppresses tails.

In all five right-skewed distributions, the locations satisfy the following consistent ordering:

mode < ${\mu }_E$ (energy center) < median < ${\mu }_M$ (mass center)

This systematic shift demonstrates that the center of information energy ${\mu }_E$ lies closer to the region of highest certainty (the mode) than does the ordinary mean. This provides further intuitive support for interpreting ${\mu }_E$ as the effective, certainty-weighted signal location and $\varphi$ as a robust measure of the system’s informational power.

5. The Inforpower Metric and Application Examples

Inforpower provides a metric to compare and select probability distributions that model the same data.

Proposition. Given a dataset and M candidate distributions $p_k\left(y\right)$ (k = 1, …, M) from the same family (i.e., sharing the same support), the preferred model is the one that maximizes inforpower

$$\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t} \mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{b}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}=\underset{k}{\mathrm{arg max}} \left[{\varphi }_k\right], k=1, 2, 3\dots M,$$

(19)

where ${\varphi }_k$ is the inforpower of the k-th candidate distribution. We refer to this Proposition as the maximum inforpower criterion.

Example 1: Synthetic Weibull distribution data [6]

A sample of n = 200 observations was generated from a Weibull distribution with true parameters $k=1.8$ and $\lambda =1$. Xie [6] fitted exponential, Weibull, Gamma, and normal distributions and compared them using the Akaike information criterion (AIC) and Pearson’s Q-statistic. Huang [2, 7] compared these candidate distributions using the maximum informity criterion and the minimum entropy criterion, respectively. The normal distribution was excluded from our analysis because its support differs from the other three (positive-support) candidates. The estimated parameters are as follows [6]: exponential distribution: $\widehat{\lambda }=1.167$, Weibull distribution: $\widehat{k}=1.662$ and $\widehat{\lambda }=0.9588$, Gamma distribution: $\widehat{\alpha }=2.255$ and $\widehat{\beta }=2.6309$. Table 2 reports the performance of each distribution under five criteria.

As shown in Table 2, the Weibull distribution ranks first under the AIC, minimum entropy, and maximum inforpower criteria, confirming that the true generating distribution is correctly identified by the inforpower metric. Although the Q-statistic and maximum informity criteria favor the Gamma distribution, Xie [6] noted that ΔAIC = 4.7 between the Weibull and Gamma distributions is small and that the AIC “selects a good model which is not necessarily the true model.” Overall, the five criteria yield highly consistent conclusions.

Example 2: Annual maximum streamflow of the Tana River, Kenya (1941–2016)

Langat et al. [8] analyzed the historical flow data collected from the Tana River (one of the longest rivers in Kenya) between 1941 and 2016. One of their goals was to find the best statistical distribution to model annual maximum streamflow. They considered six candidate distributions and used the Akaike information criterion (AIC) for selection. We focused on the following three best-performing candidates they identified: the Gamma (Pearson Type III), lognormal, and Weibull distributions. Table 3 shows the estimated parameters and AIC values, and our analysis results for the entropy, informity, and inforpower metrics.

As shown in Table 3, among these three distributions, the lognormal distribution is the best according to the all four goodness-of-fit metrics (minimum AIC, minimum entropy, maximum informity, and maximum inforpower). The Gamma distribution is the second best, while the Weibull is the worst.

Table 4 shows the percentage differences in metrics between the Weibull and lognormal distributions, computed as (Weibull – Lognormal)/Lognormal × 100%.

Although the AIC and entropy metric detect only tiny differences (0.157% and 0.369%), the informity and inforpower metrics reveal substantially larger separations (–6.730% and –1.968%, respectively). This demonstrates that inforpower (and informity) can be considerably more sensitive than the traditional information criteria when distinguishing competing heavy-tailed distributions in hydrological and other positive-skewed applications.

These examples illustrate that the maximum inforpower criterion is intuitive, easy to compute, and either agrees with or usefully complements established model-selection tools such as the AIC, minimum entropy criterion, and maximum informity criterion.

6. Discussion and Conclusions

A distinctive and desirable feature of inforpower $\varphi$ is its sensitivity to location shifts. Adding a constant a to the random variable $Y\to a+Y$ changes $\varphi$, reflecting an actual change in signal strength or systematic effect. In contrast, variance, differential entropy $H\left(p\right)$, and informity $\beta \left(p\right)$ are all translation-invariant. For example, in the normal distribution $N\left(\mu ,{\sigma }^2\right)$, the entropy ${\displaystyle \frac{1}{2}}\mathrm{l}\mathrm{o}\mathrm{g}\left(2\pi e{\sigma }^2\right)$ and informity ${\displaystyle \frac{1}{2\sigma \sqrt{\pi }}}$ depend only on σ, not on the location μ. While translation invariance is useful when one wishes to measure pure spread or uncertainty, it is considered as a drawback in applications where it is necessary to take into account the values of the random variable along with their probabilities (or densities) [9].

Attempts to remedy the translation-invariance problem in related quantities have appeared in the literature. Belis and Guiasu [10] introduced weighted entropy:$H^w\left(p\right)=-\int yp\left(y\right)\log p\left(y\right)dy.$ More recently, several authors [9, 11, 12] defined extropy for continuous distributions as $J\left(p\right)=-{\displaystyle \frac{1}{2}}\int {p\left(y\right)}^{2}dy=-{\displaystyle \frac{1}{2}}\beta \left(p\right)$ and proposed a weighted extropy (restricted to nonnegative random variables) $J^w\left(p\right)=-{\displaystyle \frac{1}{2}}{\int }_0^{\infty }{yp\left(y\right)}^{2}dy=-{\displaystyle \frac{1}{2}}\varphi .$

Although weighted extropy is formally −½ times informity, the two measures arise from fundamentally different conceptual foundations. Extropy was originally motivated as a “complementary dual” of entropy, a claim that has been critiqued on both theoretical and interpretative grounds [4]. Consequently, weighted extropy inherits these conceptual difficulties, whereas inforpower—defined directly as the expected effective information ${\mathbb{E}}_p\left[Yp\left(Y\right)\right]$—possesses clear probabilistic, informational, and physical interpretations (Section 3 and Section 4) without requiring an intermediate appeal to extropy.

Inforpower $\varphi$ and the information-energy center ${\mu }_E=\varphi /\beta \left(p\right)$ are both robust statistics. They exist even when ordinary moments fail: for the Cauchy distribution, the mean and variance are undefined, but $\varphi$ and ${\mu }_E$ remain well-defined, and $\varphi$ equals the location parameter times ${\mu }_E$. For right-skewed distributions on [0, ∞), empirical evidence (Section 4) consistently places the energy center in the order: mode < ${\mu }_E$ < median < mean, highlighting ${\mu }_E$ as an effective signal strength that lies closer to the region of highest probability density.

Computation of $\varphi$ and ${\mu }_E$ requires an explicit PDF. In empirical studies, this is not a significant limitation: simply fit candidate distributions to the observed data (using maximum likelihood estimation or other estimation methods), and then evaluate $\varphi$ on the fitted models. The maximum inforpower criterion (Section 5) thereby provides a simple model-selection rule that complements or even outperforms traditional tools (such as AIC) in applications where signal magnitude, peak sharpness, and tail behavior matter (e.g., hydrology, finance, reliability engineering, and extreme-value analysis).

For symmetric location-scale families (normal, Laplace, logistic, Student’s t, etc.), $\varphi =\mu \beta \left(p\right)$. When comparing distributions with the same estimated location $\widehat{\mu }$, the maximum inforpower criterion collapses to the maximum informity criterion. In all other cases, especially positive-support and skewed data, it offers additional discrimination power, as demonstrated by the Tana River streamflow example.

In summary, inforpower $\varphi$ constitutes a principled, robust, and effective measure of informational power that naturally integrates signal strength and noise level. The maximum inforpower criterion is easy to implement and provides a valuable new tool for distribution selection and performance assessment. Future research should explore its behavior in multivariate settings, its sampling properties, and its applications to machine learning, finance, and information fusion.