Innate perception of risk: probability ratio or difference?

In public health literature the risk of death or disease associated with a dietary, environmental of behavioral factor is most commonly denoted by odds ratio (OR), hazard ratio (HR) or risk ratio (RR). The ratio indices have several desirable statistical properties. However, the most important question is whether there are some evolved innate norms of perception of risk that people use and what they are. We conducted a simple one question survey of 98 individuals with different age, sex, educational and professional backgrounds. The respondents were asked to judge the relative perceived risk of four different hypothetical habits for which data on the percentage of people affected by the disease with and without the habit was given. They were asked to rank the risks for the four habits. Results showed that the habits that had the highest difference between probability of acquiring the disease were ranked high on risk perception. The probability ratios did not affect risk perception significantly. Further age, sex, profession or formal training in statistics did not affect the response significantly. Even individuals that were formally trained to use OR and HR as risk indicators, preferred using probability differences over ratios for judging their own risk in the perceived context. This preliminary inquiry into intuitive statistical perception suggests that designing statistical indices based on people’s innate perception may be a better strategy than trying to train people to understand the indices designed by expert statisticians. Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 8 December 2020 doi:10.20944/preprints202012.0200.v1

Abstract: In public health literature the risk of death or disease associated with a dietary, environmental of behavioral factor is most commonly denoted by odds ratio (OR), hazard ratio (HR) or risk ratio (RR). The ratio indices have several desirable statistical properties. However, the most important question is whether there are some evolved innate norms of perception of risk that people use and what they are. We conducted a simple one question survey of 98 individuals with different age, sex, educational and professional backgrounds. The respondents were asked to judge the relative perceived risk of four different hypothetical habits for which data on the percentage of people affected by the disease with and without the habit was given. They were asked to rank the risks for the four habits. Results showed that the habits that had the highest difference between probability of acquiring the disease were ranked high on risk perception. The probability ratios did not affect risk perception significantly. Further age, sex, profession or formal training in statistics did not affect the response significantly. Even individuals that were formally trained to use OR and HR as risk indicators, preferred using probability differences over ratios for judging their own risk in the perceived context. This preliminary inquiry into intuitive statistical perception suggests that designing statistical indices based on people's innate perception may be a better strategy than trying to train people to understand the indices designed by expert statisticians.
Sampling to judge the context quantitatively, analyzing in some way or the other to take a quantitative behavioral decisions is an innate ability demonstrated in many animal behavior studies (1,2,3). Therefore it is a fair assumption that humans too have some evolved innate algorithms for sampling their surroundings and drawing useful inferences (4,5). One important function of this ability is to assess risks of various kinds to avoid them or to optimize behavioral strategies accordingly.
Risk assessment is also an area extensively researched in formal statistical literature, and used widely in public health literature. The commonly used indices include risk ratio (RR), odds ratio (OR) or hazard ratio (HR) (6). Although differing in some details, they depict the fold increase in risk. Some indices based on probability difference are also described in literature (7) but are not commonly used in public health practice. We ask a single relevant question here whether people have any innate tendency to prefer probability ratio over difference or difference over ratio in intuitively judging risk.
We gave an imaginary task to respondents in which they had to rank the risk of an imaginary disease from four hypothetical habits. Data on the percentage of people contracting the disease with and without the hypothetical habit was provided. Among the four habits in the question, two had the same probability ratio but different probability difference. Two had the same probability difference but different ratio. Maintaining these features, the actual numbers and sequence was reshuffled and three different versions of the questionnaire were used. The master questionnaire is given in the supplementary information, to which the shuffled versions were realigned after receiving the responses.
Respondents were asked to give ranks, 1 denoting highest risk and 4 denoting lowest. Tied ranking was permitted. Potential respondents were contacted through personal links online and offline. No randomization procedure was followed for sampling. Nevertheless the samples represented a wide variety of age groups, educational and professional backgrounds. The minimum limit for education was secondary school, since it was important to understand the question by reading it. Any queries about clarity of the question were appropriately replied to.
The data provided in the questionnaire realigned to the master copy and the corresponding response summary are given in A and C as well as B and D have the same difference but different ratios. If the responses were random, we would have expected equal mean scores for all the habits. The 98 completely filled questionnaires revealed that the responses were non-random and concordant. The mean risk perception scores differed significantly with B and D being perceived as high risk habits and A and C with low risk (One way ANOVA F=77.53, p = 2.6x 10 -39 ). The mean scores of B and D that had the same difference (40%) but different ratios (3 versus 401) did not differ significantly (F=1.26, p=0.26). A and C also had the same difference (40%) but different ratios (3 versus 1.01), here C was perceived to have a marginally greater risk (F= 4.18, p=0.04). The difference in ratios between B and D was much greater than that in A and C, but risk perception in the former was non-significant whereas the latter was, so ratios do not seem to explain the perception of risk significantly. In contrast, A and B that had the same ratio (3) but different difference (0.2 versus 40) the one with higher difference was perceived as greater risk (F=143.19, p < 0.001) with a substantial difference in perception. Taken together, people seemed to judge the risk primarily by probability difference rather than by probability ratio.
When age, sex, education and specifically training in medical statistics was included as a second factor in two-way Anova analyses, none of them showed any significant effects on risk perception scores. The last one is particularly interesting because use of ratio based indices such as OR and HR is commonly taught in public health statistics to reflect risk, whereas probability difference based indices may not receive even a mention. It is interesting that even after undergoing training with ratio based indices, when one makes a decision for oneself, probability difference seems to be preferred over probability ratio.
Although the experiment was simple and the results could be treated only as preliminary and exploratory, the finding that human intuitive statistics may work differently than mainstream academic statistics, suggests a rethinking of indices of risk assessment. It would be advisable to use indices that resonate better with people's innate perception rather than indices which may have better mathematical properties but have a greater mismatch with people's perception. This is particularly relevant to public health statistics and may have important implications for policy making in the interest of people.