A Note on the Relationship of Normative Principles Between Decision Under Risk and Over Time

1. Introduction

Behavioral economic studies have explored various “anomalies” in human judgment and decision making. For instance, people’s decision under risk and uncertainty often violates von Neuman and Morgenstern [1]’s expected utility theory based on the independence axiom (see Allais, 1953 [2] for critical examination of this axiom), and people usually discount future outcomes (gains and losses) in a hyperbolic manners [3] rather than rational (dynamically consistent), exponential manner. On the other hand, in the field of mathematical modelling of human and animal behaviors, behavioral psychologists have established “hyperbolic” delay [4] and probability [5] discounting models for decision over time and under uncertainty, respectively. In this note, I demonstrate that normative principles of decision under risk and over time in neoclassical economic theory are incompatible, by utilizing the mathematical models of delay (time) and probability discounting.

1.1. von Neumann and Morgenstern’s Expected Utility Theory and Behavioral Economics

In order to mathematically model human decision making under risk, von Neumann and Morgenstern proposed the expected utility theory in their work in game theory [1], which has often been employed studies in neoclassical economics. This theory is a modern axiomatization of Bernoulli (1738)’s theory which was proposed to solve St. Petersburg’s paradox [6]. For establishing the expected utility theory, they assumed the independence axiom, which is formally represented as:

for all p,q,r∈∆(X) and for all λ∈[0,1],

p⪰q⇔λp+(1−λ)r⪰λq+(1−λ)r,

where we represent the set of lotteries with prizes in X by ∆(X), with weak preferences ⪰ defined over ∆(X). We denote generic prizes in X by x,y,z, and denote generic lotteries in ∆(X) by p,q,r,s. The probability of receiving prize x under lottery p is denoted p(x). By assuming this independent axiom (and other axioms of, e.g., continuity) von Neumann and Morgenster’s expected theory states agent’s risk preference can be represented as concavity (Arrow-Pratt measure [7]) of its utility function, irrespective of psychological effects of risk and uncertainty on the agent’s information processing. However, economist Maurice Allais (1953)[2] pointed out that people often violate the independence axiom. Later, behavioral economist Kahneman and Tversky proposed their prospect theory to model nonlinear probability weighting in human decision making under risk and uncertainty [8]. Furthermore, behavioral economist Drazen Prelec [9] axiomatically derived the functional form of the probability weighting w(p), which is regressive (first w(p)>p, then w(p)<p as probabilities p increases), s-shaped (first concave, then convex), and asymmetric, and with an invariant fixed point and inflection point at 1/e = .37. Takahashi (2011) [10] derived the probability weighting function, by employing psychophysical laws of time perception, indicating that nonlinearity of probability weighting and psychophysics of time are inter-related. It is to be noted that in von Neumann and Morgenstern’s theory, probabilities are frequentionistic, rather than Bayesian subjective ones, it was extended to “subjective” expected utility theory, by incorporating subjective (personal) Bayesian probabilities based on Savage’s axioms [11].

1.2. Probability Discounting Models and Behavioral Economics and Psychology

In contrast to models of nonlinearity of information processing of probabilities in human judgement and decision making, behavioral psychologist Rachlin and colleagues [5] proposed that information processing of probabilities is transformation of probabilities into time (delay of a reward until its receipt). Rachlin and colleagues laid the groundwork for treating the valuation of probabilistic goods as a variant of the valuation (discounting) of delayed goods. This approach in behavioral psychology was seminal for a large body of subsequent research. Howard Rachlin et al. (1991) [5] explicated and applied the treatment of gambles-as-delays created by Rachlin et al. (1986) [12]. The theory was established a hypothetical equivalence of risk and uncertainty to time, in the manner like this: The average waiting time to reinforcement, d, is

$$d={\displaystyle \frac{t_{ITI}+t_T}{p}}-t_{ITI}$$

where$t_{ITI}$is the intertrial time, $t_T$ is the trial time, and p is the probability of obtaining the payoff on any trial, p > 0. It will require 1/p trials on the average to obtain the reward, with each trial (except the first, see Killeen 2023 [13] for reevaluations of this issue) requiring the time in the numerator. The expected average delay on all trials (except the first) is given by the fraction. The first trial is assumed to begin with the trial. The lower the probability of an outcome, the greater the expected delay until its receipt, suggesting that risk and uncertainty are equivalent to time, under the assumption that the system is ergodic. Rachlin et al. (1991) [5] then developed the model in the following way. They first assumed that trial time while making the choice, $t_T$, was negligible so that they could drop it, and then they rearranged the equation:

$$d\approx {\displaystyle \frac{t_{ITI}}{p}}-t_{ITI}={\displaystyle \frac{t_{ITI}-{pt}_{ITI}}{p}}={\displaystyle \frac{t_{ITI}(1-p)}{p}}=t_{ITI}O$$

where the odds - against winning on any trial are

$$O={\displaystyle \frac{(1-p)}{p}},\mathrm{with}0<p<1.$$

Thus, in behavioral psychology, in contrast to economic theory, Rachlin and colleagues’ approach attributes risk aversion to devaluation of delayed rewards under a repeated, rather than one-shot, gambling condition with ergodicity. For instance, if the probability of winning is 0.25, i.e., one winning out of four trials, the average delay until winning is $t_{ITI}$ (1/0.25-1)=$3t_{ITI}$. It is to be underscored that in neoclassical economic theory, the subjective expected utility theory (by assuming von Neumann and Morgenstern’s independence axiom [1] and Savage’s sure-thing principle [11]) has been often utilized in which probabilities are Bayesian subjective probabilities and decision could be one-shot, rather than repeated. Rachlin’s probability discounting theory is distinct, from the subjective expected utility theory, in that probabilities are, even in one-shot risky decision making, defined in virtually repeated gambles, in terms of delay until winning. Takahashi generalized Rachlin’s probability discounting model by utilizing the q-exponential function developed in Tsallis thermodynamics and demonstrated that the q-exponential probability discounting model well fit human behavioral data under probabilistic risk for gains and losses [14] even in one-shot risky decision making, supporting the general validity of the probability discounting theory.

1.3. Dynamic Consistency in Intertemporal Choices and Behavioral Economics

A long time ago, economist Paul Samuelson [15] proposed exponential discounting model for human intertemporal choice behavior, which assumed that humans devalue (discount) future delayed reward exponentially. Strotz [16] theoretically revealed that non-exponential temporal discounting behavior leads humans to dynamic inconsistency (irrationality) in intertemporal choice, which is referred to as myopia concurrent with preference reversal over time. Behavioral economist Richard Thaler [3] observed that humans (university students) make dynamically inconsistent intertemporal choices for delayed monetary gains and losses. This observation indicates the necessity interventional economic policies what we call “nudge” in order to correct human inconsistent choices and other self-destructive and anti-societal choices. On the other hand, in the domains of behavioral psychology, Mazur [17] proposed and Ainslie [18] employed the hyperbolic discounting function, for modelling agent’s delay discounting behavior. Taiki Takahashi (2005) [19] proposed that the dynamically inconsistent hyperbolic discounting behavior is due to nonlinear (logarithmic) time perception, which has been studied in psychophysics, in humans waiting for delayed rewards, which was later supported by behavioral experiments [20].

Taken together, it can be stated that in economic theory, there are two separate principles of rationality between risk and time, i.e., the independence axiom (the expected utility theory) and dynamic consistency (exponential discounting), although vast amount of evidence in behavioral psychology and psychophysics indicates that both risk and time are related to each other in information processing by humans and animals. With respect to information processing in the human brain, high intelligence and cognitive abilities are related to reduced risk aversion (risk tolerance) in decision under probabilistic uncertainty [21] and impatience in intertemporal choice [22], indicating common neural information processes between risk (uncertainty) and time. As noted earlier, behavioral economic studies have been examining the discrepancies between normative and descriptive models of human decisions, only little attention has been paid to discrepancies between normative principles between risk and time domains. In this study, I examined this issue by combining models of risky decisions in both economics and psychology to find that normative principles of risk and time are incompatible. This incompatibility potentially has important implications for behavioral and neuro- economics, as well as behavioral psychology. For instance, several studies [23] indicated that problematic gamblers have greater delay discounting, although they are considered to weaker probability discounting under the assumption of equivalence of risk and delay. Also, the effects of the size (magnitude) of rewards on delay and probability discounting are opposite: small rewards are strongly delay-discounted, but weakly probability-discounted [24]. These incongruent findings on decision making between time and risk could be illuminated by considering the present analysis.

Recent advances quantum information theory [25,26] have extended supports for “QBism” which employs the frameworks of optimal Bayesian betting strategies for interpreting quantum probabilities. Since QBism is distinct in terms the interpretations of probabilities from conventional frequentionistic interpretation, the present study may also help understanding of the roles of probability concepts in mathematical physics.

2. Analysis of Mathematical Models for Decion Under Risk

In von Neumann and Morgenstern’s expected utility theory (derived from the independence axiom and often adopted in neoclassical economics), it is assumed that the subjective value $U\left(x,p\right)$ of a risky outcome $x$ obtaining at the probability $p$ is expressed as:

$$U\left(x,p\right)=pu\left(x\right),$$

(1)

where $u\left(x\right)$ is the subjective value (utility) of certain outcome $x$. Regarding decision over time (intertemporal choice) in neoclassical economics, it has been proposed that humans discount the value of a future outcome $x$ in an exponential manner[6]:

$$U\left(x,t\right)=u\left(x\right)\mathrm{e}\mathrm{x}\mathrm{p}(-kt),$$

(2)

where $u\left(x\right)$ is the subjective value (utility) of immediate outcome $x$.

In behavioral psychology, in order to describe human and animals’ (not necessarily rational/normative) choice under risk, the following exponential (equation 3) and “hyperbolic” (equation 4) probability discounting models have been utilized under the assumption of ergodicity, in other words, waiting time until winning is proportional to “odds-against” $O\left(p\right)=1/p-1$, see Rachlin et al.,1991 [5] for details):

$${exponential:U}_e(x,O(p\left)\right)=u\left(x\right)exp(-k_{e}O(p\left)\right),$$

(3)

and

$${hyperbolic:U}_h\left(x,O\left(p\right)\right)={\displaystyle \frac{u\left(x\right)}{1+k_{h}O\left(p\right)}}.$$

(4)

It is to be noted that, from the view point of ergodic assumption that odds-against corresponds to waiting time, equation 3 is rational (exponential) dynamically consistent discounting model in terms of neoclassical economics (equation 2); while equation 4 is irrational (non-normative) hyperbolic (non-exponential) model.

Now we are prepared to utilize the probability discounting theory for analyzing relationship of normative principles of decision under risk between neoclassical economics and behavioral psychology. The question is “is expected utility compatible with exponential discounting?”. Von Neumann and Morgenstern’s expected utility theory (neoclassical economics, equation 1) corresponds to behavioral psychological model’s equation 4 (hyperbolic), in the case of $k_h=1$ (${\displaystyle \frac{u\left(x\right)}{1+k_{h}O\left(p\right)}}={\displaystyle \frac{u\left(x\right)}{1+1(\frac{1}{p}-1)}}=pu(x)).$ This indicates that “rational” model (equation 1) in neoclassical economics corresponds to the special case of “irrational” (non-exponential, hyperbolic) model (equation 4). Therefore, von Neumann and Morgenstern’s expected utility theory (equation 1) is not normative (rational) when the system is ergodic. As far as I know, this study is the first to demonstrate that von Neumann and Morgenstern’s theory is not normative when considering risk in terms of time under ergodicity.

3. Discussions

The present analysis indicates von Neumann and Morgenstern’s expected utility theory (equation 1) is not necessarily normative, particularly when assuming the system is ergodic and normative principle for decision over time is exponential discounting (equation 2). The finding indicates that von Neumann and Morgenstern’s expected utility theory and Samuelson’s exponential discounting are incompatible with respect to the equivalence of risk and time. Future behavioral economic and econophysical studies should take into account this incompatibility of normative principles between risky and intertemporal decision making. I here discuss limitations of the present analysis. The augment depends entirely on the probability-time equivalence proposed Rachlin’s model. However, we previously demonstrated, in behavioral experimental study, that probabilities are, even in one-shot choice of risky lotteries, transformed into subjective waiting time following Rachlin’s assumption [14]. This finding may supports the present approach. Also, behavioral psychologist Killeen [13] recently raised some questions about assumptions in the probability discounting theory as to whether we can ignore the trial time, within one-shot gambling, in repeated gambling. This issue should be examined in future studies.

3.1. Implications for Neural Information Processing

I here note several applications of the present findings to future possible studies. Regarding information processing in the human brain, temporal discounting [27] and probability weighting in decision under risk [28] are both, at least partially, mediated by neurotransmitter dopaminergic systems. Hence, how rational, intelligent decision making is modulated by the dopaminergic systems should be examined. Neuropsychologically, high cognitive abilities and intelligence are shown to commonly related to reductions in risk aversion [21] and impatience in intertemporal choice [22], suggesting that neurocomputational processes underlying rational decisions both under risk and over time share common mechanism. Therefore, future studies are expected to examine how the present revealed incompatibility of normative principles on rationality between risk and time are resolved in the human brain. Takahashi and Han (2013) [29] reported that psychophysical time in decision under risk is also logarithmic similarly in physical time, indicating that humans commonly transform delay time and Rachlin’s odds against [5]. Therefore, it may be important to examine how rationality is related between risk and time in psychophysical manners.

3.2. Implications for Behavioral and Psychiatric Neuroeconomics

The present study also has several implications for neuroeconomics and computational psychiatry. For instance, Kyonka and Schutte (2018) [30] hypothesized that shallower (weaker) probability discounting (risk tolerance in decision under risk) is associated with problematic gambling, by conducting meta-analysis on the relationships between probability discounting. Shallow probability discounting can be viewed as a cognitive bias or distortion (i.e., violation of normative principle of rationality in decision making) that results in assigning low probability gains or high probability losses more subjective value than is warranted in terms of probability theory. A model of neurocomputational underpinnings of cognitive distortions associated with reward expectation and outcome evaluation holds that these violations of rationality principles may lead to pathological gambling behavior. The present findings regarding rationality in decision under risk (probability discounting) and over time (time discounting) support the premise that cognitive distortions relating to both uncertain and delayed rewards are linked to maladaptive behaviors associated with gambling. the results of the present meta-analysis suggest that probabilistic cognitive biases relating to gambling outcomes may underlie gambling problems. As maladaptive gambling behavior can cause significant distress and has extensive societal costs, a more extensive understanding of cognitive biases and violations of normative principles (either hyperbolic/exponential probability discounting or von Neumann and Morgenstern’s independence axioms) underlying gambling can lead to investigation of more effective behavioral economic intervention efforts. The probability discounting framework is also limited in its scope, which does not allow for the clear dissociation of neuropsychological constructs that are thought to underlie behavioral and neuro- economic parameters (e.g., risk aversion, loss aversion in prospect theory) and that may differ across tasks or measurement modalities. Additionally, it may be important to investigate decisions on non-monetary outcomes under risk and over time. More examinations are broadly needed to continue exploring the extent of decision task reliability achieved in probability and temporal discounting on nonmonetary commodities. Although previous findings support the use of either discounting type (delay or probability) and administration across commodities, money remains the predominant medium for discount-like patterns of choice, although choices in daily life often involve nonmonetary outcomes (e.g., food, drink, hobby time). Most studies in behavioral economics examined discounting in a commodity other than money and only few studies explicitly focused on a nonmonetary commodity in probability discounting, although nonmonetary rewards have been more extensively studies in temporal discounting [31]. Future studies should continue to examine changes in discounting across commodities to better inform the extent to which commodity- and domain-specific discounting more precisely describe relevant choice and how domain-specific violations of rationality principles appear in human decision making.

With respect to effects of contextuality on human decision making, Recent meta-analytic findings observed in discounting as a predictor of clinical outcomes (i.e., cannabis use; Strickland et al., 2021 [31]) reported a statistically significant moderator outcome for the experiential nature of the task and thus a notable difference in predictive outcome between task iterations. Also, it was suggested that an important role of contextual cues for modulating decision making in a manner consistent with that experienced outside the laboratory (similar to the use of the relevant commodity instead of money; e.g., Acker & MacKillop, 2013 [32]). Therefore, future behavioral economic studies should study whether decisions under risk are influenced by the context of risk, i.e., in cases when probabilities of obtaining rewards are presented as of either one-shot or repeated gambling. This type of examinations may help understanding human mental representations of uncertainty; whether uncertainty and probabilities are represented Bayesian-like subjective beliefs, or frequentionistic psychological quantities.

3.3. Implications for Quantum Epistemics

As stated earlier, recent advances in quantum information theory continue gathering supporters of “QBism”, which is a Bayesian subjective (personal) interpretation of quantum probabilities (also called quantum epistemics, in which quantum theory is treated as theory of knowledge stats of observers of quantum systems). The Born’s rule of quantum probability has also been derived from normative principles of rational decisions under uncertainty [25]. This is distinct from more traditional interpretation of quantum probabilities in which probabilities are viewed in terms of frequencies of obtaining certain experimental values of physical quantities such as directions of spin, locations of electrons. The probability concept in QBism is in sharp contrast to Rachlin’s probability discounting concept in which probabilities are thought to be equivalent to delay time (akin to frequentionistic interpretations of probabilities), not subjective beliefs of decision makers. Although von Neumann explored the mathematical foundations of quantum mechanics [33], he did not investigate the compatibility of his interpretations of quantum mechanics and rationality principles in decision theory. It is again to be stressed that, in QBism, similarly to de Finetti’s subjective interpretation of probabilities, quantum probabilities are derived from normative principles of decision under risk and uncertainty [25]. The core idea of personalist Bayesian probability theory is that it is a manner of rational calculus of consistency (or “coherence” as the practitioners call it), but in this theory for one’s decision-making degrees of belief. Probability theory can only say if various degrees of belief are consistent or inconsistent with each other [33]. However, as demonstrated in the present study, widely-utilized normative decision principles of decision under risk and uncertainty are incompatible with the rationality principle in intertemporal choice (i.e., exponential discounting). Therefore, future studies in quantum epistemics and quantum information theory should investigate how to reconcile the two principles (von Neumann and Morgenstern’s independence axiom and Samuelson’s exponential discounting) in the domain of quantum theory.

4. Conclusions

von Neumann and Morgenstern’s expected utility theory (and the independence axiom) is not necessarily normative, particularly when assuming the system is ergodic and normative principle for decision over time is exponential discounting (dynamic consistency), from the standpoint of the probability discounting theory. The analysis indicates that von Neumann and Morgenstern’s expected utility theory and Samuelson’s exponential discounting are incompatible with respect to the equivalence of risk and time when the system is ergodic. Future studies in behavioral economics and quantum epistemics should consider this issue.