Psychometric Properties of the ACE-IQ Questionnaire’s Binary and Frequency Scoring in a Chilean Community Sample

1. Introduction

1.1. Impact of Adverse Childhood Experiences (ACEs) on Health

Over 20 years ago, Felitti et al. (1998) published a groundbreaking study on the long-term impact of Adverse Childhood Experiences [ACEs] on health risk behaviors in adulthood [1]. Over time, this study has evolved from an epidemiological scientific article to being widely recognized as a critical comprehensive framework in public health [2]. Felitti et al. demonstrated that ACEs are much more prevalent than previously believed, using a sample of over 17,000 individuals from middle-class backgrounds [1]. Additionally, the research confirmed a robust relationship between adverse experiences during childhood and adult emotional health, physical health, and leading causes of mortality in the United States. The risk increased as ACEs increased in a dose-response relationship [1,3].

Subsequent studies have also established a relationship between these experiences and a higher prevalence of common health risk factors and long-term disease causes in adulthood, including cardiovascular diseases, diabetes, and cancer, among others [4]. Furthermore, there is a higher prevalence of subsequent psychopathology, with the number of ACE serving as a predictor of mental illnesses such as depression, bipolar disorder, substance abuse, and post-traumatic stress disorder, along with other conditions [5].

The mechanism explaining this dose-response relationship involves brain-level changes. Childhood exposure to adversities can lead to toxic stress, resulting in disruptions in children’s neurodevelopment [6]. This manifests as dysregulation in stress response, impairment in executive functioning, alterations in the endocrine and immune systems, genetic regulatory mechanisms, and other related effects. These alterations consequently lead to social, cognitive, and emotional dysfunction, an increase in risky behaviors, difficulty in forming healthy relationships, early onset of diseases, and socio-emotional problems, and could ultimately result in premature death [4,7]. Therefore, childhood exposure to experiences such as abuse, neglect, domestic violence, and other stressors within the home significantly impacts individuals’ developmental trajectories, with consequences throughout the lifespan.

Most ACE studies have continued to use the ten adversities assessed by Felitti and colleagues in the original questionnaire (physical abuse, emotional abuse, physical neglect, emotional neglect, sexual abuse, household substance abuse, household incarceration, domestic violence, parental mental illness, and separation/divorce/loss of a parent) [1]. However, it has been noted that these 10 ACEs are not the only forms of adverse experiences that can occur in childhood, causing negative effects on developmental trajectories [8]. Therefore, to significantly improve the prediction of physical and mental health problems, it has been recommended to include other adverse experiences present at different levels of an individual’s ecology, such as peer violence and community violence [8,9].

1.2. Development of the International Questionnaire of Adverse Childhood Experiences (ACE-IQ)

In 2011, recognizing the significant interest in developing policies informed by the ACEs framework, the World Health Organization [WHO] expanded the original ACE survey and developed the Adverse Childhood Experiences International Questionnaire [10]. This questionnaire covers, in addition to the original ten adverse experiences, a broader range of adverse events that individuals may encounter during childhood. The overarching objective behind developing ACE-IQ was to provide an instrument capable of evaluating childhood adversities worldwide, studying their potential implications in different countries, and enabling international comparisons of ACEs’ prevalence and consequences, including collective violence [10]. ACE-IQ assesses the occurrence of 13 categories of ACEs in adults, grouped into four dimensions. Each category of adversity occurrence is scored as 1, providing an overall score ranging from 0 to 13. The evaluated dimensions are:

• Parents/caregivers’ physical and emotional neglect.
• Abuse, including physical and emotional abuse experienced at home and sexual abuse.
• Family dysfunction, encompassing having one or more household members with alcohol or drug abuse; a family member incarcerated; a household member with mental illness; a household member subjected to violence; single or absent parents, parental separation, or divorce.
• Violence outside the home, including bullying, community violence (e.g., witnessing someone being assaulted/killed, fights), and collective violence (e.g., exposure to wars, terrorism, police, or gang fights).

The WHO has also encouraged the translation and inclusion of ACE-IQ in comprehensive health surveys with the additional goal of testing the reliability and validity of this questionnaire in different countries [11]. This endeavor has been successfully accomplished; indeed; a recent systematic review reports that the instrument has been utilized in community samples from more than 29 countries across various income levels, including India, Turkey, Pakistan, China, Germany, the United States, Canada, the United Kingdom, Egypt, Kenya, Malawi, Sierra Leone, Tanzania, and Israel, revealing different prevalences in the sub-dimensions depending on the country [12]. This systematic review, encompassing 63 articles, concluded that, on average, 75% of respondents in community samples experienced ACEs, with an average of three adversities per person. Emotional abuse and bullying had higher prevalence rates [12]. Additionally, it was observed that, in general, men experienced more ACEs but were underrepresented in the samples [12].

Nonetheless there is a difficulty in the comparison of the prevalences between countries because the WHO proposed two scoring methods to calculate ACEs exposure (i.e binary and frequency scoring) and suggested comparing the results between both methods to determine “the most appropriate approach to determine an accurate overall ACE score for a participant” [13] (p.1). Both scoring methods involve dichotomizing the 13 ACE categories into “non-exposure or 0” and “exposure or 1” leading to a total score range from 0 to 13 [13]. However, these methods differ in their approach to quantifying exposure levels within these categories. The frequency scoring method considers the intensity or frequency of exposure and varies between the different kinds of ACE. For instance, for sexual abuse, a single incident constitutes exposure. Conversely, for physical abuse, repetitive incidents are required to constitute exposure. However, the binary scoring method adopts a lower threshold for identifying ACE exposure. Under this method, any experience of adversity is regarded as exposure. For instance, being yelled at or sworn at even once is categorized as emotional abuse. However, challenges arise because most of the research that uses the instrument doesn’t report or compare both coding systems. For example, none of the studies included in the most recent systematic review on the use of the ACE-IQ conducted a comparison between these two scoring systems [12]. Moreover, some studies even did not specify the scoring method they employed. Consequently, comparing prevalences becomes problematic. One recent study directly compared prevalences between the frequency and binary versions, revealing substantial differences in total ACE scores based on the chosen scoring method [14]. Also, Kidman et al. coded the instrument with both scoring methods, and they saw that the frequency scoring yielded a much lower total ACE scores but that the association with depressive symptoms was similar in magnitude to the binary version [15].

1.3. Psychometric Properties and International Validation of the ACE-IQ

Regarding the instrument’s psychometric properties, there are numerous validation studies in different countries [15,16,17,18,19]. Kazeem showed in a Nigerian sample that the instrument had good internal validity with a Cronbach’s alpha of .80 and showed that the ACE-IQ total score correlated with the CTQ [16]. In this study, six subscales were considered: marriage, relationship with parents/guardians, family environment, peer violence, witnessing community violence, and exposure to war/collective violence. Kidman et al. also showed that the ACE-IQ had good predictive validity by examining the association between overall experience of adverse events and depression measured by BDI score [15]. In the validation with a sample from Hong Kong, Ho et al. demonstrated that the instrument showed good content validity and internal validity with a Cronbach’s alpha of .80 [17]. They considered three subscales: “childhood maltreatment,” “family/ household dysfunction,” and “violence outside the home” with Cronbach’s alpha of 0.74, 0.62, and 0.60, respectively [17]. In the French validation, Tarquino and colleagues considered the following dimensions that emerge from the structure of the original instrument: childhood maltreatment, family dysfunction, and violence outside the home, with Cronbach’s alphas ranging from 0.77 for child maltreatment to 0.62 for household disruption, and 0.41 for violence outside the home [19].

Regarding the structure of the instrument, Kidman et al. tested the instrument on a Malawian sample using principal components analysis [15]. They found a three-component solution comprised of “household dysfunction,” “abuse,” and “neglect” components, but this solution did not include three of the thirteen ACEs. In the Mexican validation of the instrument, Téllez et al. conducted an exploratory factor analysis, finding five factors with adequate internal consistency: household violence, sexual abuse, family dysfunction, peer violence, and community violence with Cronbach’s alpha values for reliability of .86, .90, .72, .69, and .69 respectively [18].

Seeing the differences in the categories reported by all the previous studies is clear that there is not a standard definition of the categories that comprise the ACE-IQ. This lack of clarity regarding the dimensions was also reported by Pace et al. after conducting the systematic review [12]. So, in this study we will test the factor structure of the ACE-IQ with the aim to compare which solution is a best fit: a three-factor solution (household dysfunction, childhood abuse, external violence) or a four- factor solution (household dysfunction, childhood abuse, external violence, childhood neglect). The rationale for the three dimensions model is based on a two-factor solution for the ten ACE questionnaire plus a third dimension of violence outside the home and is consistent with the one reported by Tarquinio et al. [19]. On the other hand, the four-dimension solution is based on the structure reported by Pace et al. in their systematic review [12].

This study addresses the need to compare the psychometric properties of the ACE-IQ scale for both scoring systems and for the three- or four-dimension models to offer guidance for its application. Furthermore, psychometric data for this instrument in Latin America is currently lacking, with just one study on the psychometric properties that considered specific adaptations for the Mexican culture [18]. Consequently, no information is available on the instrument’s psychometric properties for its use in the Chilean context. This research seeks to provide reliable information for using the ACE-IQ in Chile and other South American countries. The results will facilitate international comparisons and address the knowledge gap of ACE studies in Latin America, providing the necessary data to guide public policies that address early adversities and their consequences.

Given the context above, the primary objective of this research was to validate the ACE-IQ scale in a Chilean community sample, analyzing the prevalence of adverse childhood experiences and the psychometric properties of the instrument in terms of (1) structural validity, through confirmatory factor analysis; (2) internal consistency, using Cronbach’s alpha and Categorical omega as an index; and (3) concurrent validity, using the Marshall Scale as a comparison instrument. Additionally, this investigation entailed a comparative analysis of both scoring methodologies.

2. Materials and Methods

2.1. Design

This study has a cross-sectional design nested in a prospective cohort study and was approved by the ethical committee of Pontifical Catholic University of Chile. The sample studied came from the “Maule Cohort (MAUCO) of chronic diseases in Chile 2014-2024” project, belonging to the Advance Center for Chronic Disease [ACCDiS]. The MAUCO project protocol was approved by the ethics committee of the Pontifical Catholic University and by the Maule Regional Health Service [20].

2.2. Participants

Regarding the composition of the sample, it was a randomized sample stratified by sex, and it consisted of 705 participants- enrolled in MAUCO before the year 2020, whose inclusion criteria were: (a) being 65 years at enrollment in our study; (b) have complete questionnaires and medical examinations; (c) have -at least- elementary education and can read -autonomously –the survey and record their answers in a tablet.

2.3. Instruments

a) Adverse experiences in childhood, the Adverse Childhood Experiences - International Questionnaire [ACE_IQ] was used; a retrospective self-report instrument for adults that investigates the occurrence of 13 categories of adverse experiences experienced before the age of 18. ACE-IQ was created by the World Health Organization [WHO] [10]. The experiences included in the questionnaire are: Emotional neglect, Physical neglect, Emotional abuse, Physical abuse, Sexual assault, Caregiver used drugs/alcohol, Caregiver presented psychopathology, Caregiver deprived of liberty, Parents separated or divorced or death of caregiver, Witness of domestic violence, Bullying/Aggression by peers, Witness of community violence, Exposure to political, collective violence or war. Exposure to each specific category of childhood adversity was coded as a binary ACE score (presence or absence of that type of experience) such that participants’ total ACEs scores indicate how many types of adversities they were exposed to (0 to 13 ACEs). In the application of the instrument, two questions were added to differentiate the type of exposure to collective violence. Specifically, differentiate violence caused by the military and police, in contrast to that caused by terrorism, militias, or gangs, given that political violence and public order forces have marked the history of Chile is type of adjustment of the instrument has also been carried out by other studies, such as Saudi Arabia, China and Korea and these adjustments are promoted by the WHO to adjust the instrument to the realities of each country [21,22,23]. The instrument was scored using both methods to calculate ACEs exposure (binary and frequency). Both scoring methods involve dichotomizing the 13 ACE categories into “non-exposure or 0” and “exposure or 1” leading to a total score range from 0 to 13. The frequency scoring method considers the intensity or frequency of exposure and varies between the various kinds of ACE [13]. However, the binary scoring method adopts a lower threshold for identifying ACE exposure. Under this method, any experience of adversity is regarded as exposure [13].

b) Marshall Trauma Scale. The Marshall Scale is a questionnaire that determines the memory of the occurrence of maltreatment during childhood [24]. This scale is a brief instrument, whose external validity has been confirmed in Chile obtaining a Pearson correlation coefficient of 0.88 [25]. The scale evaluates the presence or absence of early adverse experiences through seven items: 1) Traumatic separation from the father, mother or caregiver for more than one month 2) Experience of having suffered significant physical punishment 3) Being left with physical damage after having been punished 4) Having witnessed physical violence between parents or caregiver 5) Alcohol or drug abuse by a family member 6) Forced sexual contact by a relative 7) Forced sexual contact with a non-family member. This scale was recently used in Chile for the initial validation of the Brief Childhood Trauma Questionnaire [CTQ-SF] [26].

2.4. Translation of the ACE-IQ

The questionnaire is freely available and was translated and adapted for the Chilean study following the World Health Organization guidelines. Given the above, the first step consisted of the translation of the instrument into Spanish by two psychologists with professional competence in English and in the subject that addresses the questionnaire. In the second step, the translation was reviewed by a group of experts in the area. The third step was a back-translation process developed by two bilingual members of the Translation Review Committee of the National Child Traumatic Stress Network (NCTSN, U.S.) to review the adequacy of the translation of the original instrument into Spanish. The fourth step of the process was reviewing the translation with a professional team working on the interviews and home visits in the MAUCO cohort, to adjust the instrument’s language to the population to which it is directed. The fifth step was a pilot application of the survey to 12 people, with similar sociodemographic characteristics as the study sample was developed, which allowed evaluating the implementation of the instrument. This pilot application was carried out to test the instrument’s application digitally and explore the possible emotional impact that people who answered the questionnaires could experience. Through this translation process, the understanding of the items was checked and corrected, and during the content review and piloting, the Action Protocol was adjusted against possible emotional impact on participants due to the application of the questionnaire.

2.5. Statistical Analysis

For data analysis, 54 participants of the 705 initial sample were excluded due to missing information in any of the ACE-IQ questions. Thus, the analysis was carried out on a sample of 651 participants with complete information. To evaluate potential biases due to sample reduction, the characteristics of both subsamples were analyzed (that is, with and without missing information in the ACE-IQ), and no significant differences were found between them in the variables analyzed (age, sex, education, marital status) as shown in Table 1.

To examine the structural validity of the ACE-IQ, a confirmatory factor analysis (CFA) was conducted. This analysis allows to assess the goodness of fit for the structure of the questionnaire developed or found in the previous literature. In this case, models of three and four factors were tested. Simultaneously, these models were tested for binary and frequency corrections. The factor models were specified using for each model a polychoric matrix, as this is the recommended procedure when analyzing categorical variables [27,28]. The estimator used for these purposes is WLMSV, which does not require a distributional assumption check [29]. To assess model fit, fit indices including the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA) and the standardized root mean squared residual (SRMR) were analyzed [30]. A good model is one with SRMR close to or less than 0.08, RMSEA close to or less than 0.06, and CFI and TLI close to or greater than 0.95 [29].

Cronbach’s alpha and categorical omega indices were used to estimate internal consistency. Cronbach’s alpha allows to quantify the consistency of the variation of the items of a scale [31]. On the other hand, the categorical omega index is a robust estimate of internal consistency, because it admits the non-fulfilment of assumptions (equality of factor loadings and that the variables are continuous, in the case of Cronbach’s alpha) [32]. In addition, it uses polychoric correlations at the base, which, as mentioned above, is more appropriate when analyzing dichotomous variables.

While categorical omega is a more appropriate indicator of internal consistency in this case, Cronbach’s alpha was still calculated. This was done to compare the results with other studies of the psychometric properties of the ACE-IQ previously carried out, in which Cronbach’s alpha was used to estimate the internal consistency of the scale.

Finally, a concurrent validity analysis was conducted with the Marshall Trauma Scale (MTS), using Spearman correlation analysis and simple linear regression. According to Cohen, effect sizes (the correlation coefficients) are described as small (0.10), medium (0.30) and large (0.50) [33]. In this case, it is hypothesized that the correlation between ACE-IQ and MTS scores will be significant (p < 0.05), positive, and will have a large effect size.

For the simple linear regression, bootstrapping was performed, which is a method that allows estimating confidence intervals from the sample and assessing the quality of the estimate when assumptions are not met [34].

The analyses were conducted using RStudio 2022.07.2 Build 576 and the packages “psych” Version 2.3.12, “corrplot” Version 0.92, “lavaan” Version 0.6-17, “semTools” Version 0.4-14, “MASS” Version 7.3-60 and “boot” Version 1.3-28.1 [35,36,37,38,39,40].

3. Results

3.1. Descriptive Analysis

Among participants 54% were women, mean age was 48 years, most lived with a partner, and mostly had high school education. Table 2 shows the main characteristics of the participants and Table 3 indicated ACEs prevalence in the ACE-IQ binary and frequency.

3.2. Factorial Analysis

In total, four factor models were estimated: two for three dimensions with binary and frequency scorings, and two for four dimensions with the aforementioned scorings. For the three-factor and four-factor models, confirmatory factor models were estimated by clustering as presented in Table 4. The (robust) fit indices show that only the frequency scored models (M2 and M4) fit adequately.

Regarding the factor loadings of the model, most are close to .50, which is considered good. Despite that, there are some factor loadings considered reasonable to poor < .4 [30] as illustrated in Table 5.

3.3. Internal Consistency

The internal consistency for the overall scale is adequate (> 0.7). However, the dimensions of childhood negligence and violence outside the home show lower internal consistency (Table 5). This may be due to the number of items in each category. Both coefficients (categorical omega and Cronbach’s alpha) are known to be biased by the number of items (both require the number of items for their computation). For example, if the domain of violence outside the home were to maintain the average correlation between its items, but only increase the number of items in the domain, it would considerably increase its internal consistency. It is also important to note that internal consistency will also be affected by how the participant perceives the content of each item [41] as evidenced in Table 6.

3.4. Concurrent Validity

To conduct the concurrent validity analysis, the magnitude of the relationship between the total score of the ACE-IQ (binary and frequency scorings) and the Marshall Trauma Scale (MTS) was first evaluated. Spearman correlations were used for this purpose. The results indicated that the ACE-IQ score (binary and frequency scoring methods) correlated significantly and positively with the MTS (ρ(651) = 0.73, p < 0.001; and ρ(651) = 0.75, p < 0.001; respectively). Table 7 shows the results of the regressions between the ACE-IQ (binary and frequency scorings) and the MTS.

Therefore, both models are adequate (when using binary or frequency scoring methods for the ACE-IQ) and the best model is the second one (frequency scoring method). In this model, for each point obtained in the ACE-IQ, the MTS score increases significantly by 0.47 points. The R² indicates that the ACE-IQ explains 60% of the variance of the level of trauma measured with the MTS.

It is important to mention that each estimated model obtained a significant fit to the data (significant ANOVA F-value) and that, due to the non-fulfilment of the normality assumption, confidence intervals for the intercept and predictor were estimated by bootstrapping (resampling performed 1000 times). The estimates of the bootstrapped confidence intervals were the same as the estimates made by the regressions, so it can be confirmed that the results are adequate.

4. Discussion

Our study is the first in Chile to examine the psychometric properties of the ACE-IQ in South America and it intends to provide a measure for future studies in Chile. This country has a high prevalence of ACEs during childhood, a national survey shows that around 40% of the urban population has experienced four or more ACEs in their lifetime [42]. In the past, Chilean studies included a fraction of ACEs in their studies and the availability of an instrument like the ACE-IQ with a wider range of ACEs, provides the opportunity to assess a more realistic prevalence of ACEs in the Chilean context. Given the association of ACEs and later life consequences, is crucial to have a measure for the studies that need to explore this association in Chile.

This study embraced the ACE-IQ developers’ advocacy for utilizing both scoring methods in research settings to enable cross-study comparisons.

The results of this study suggest that the ACE-IQ, translated into Spanish and adapted to the Chilean context, has an appropriate full-scale reliability (using binary and frequency scorings), with evidence of adequate construct validity (for frequency scoring) and concurrent validity (using both methods of scoring).

This is in line with other studies [19,43]. However, it is important to note that in studies such as Kidman and Swinge studies, the analyses were performed using the score for the totality of the items, obtaining a comparatively higher estimate of internal consistency [15,43]. For this reason, the coefficients obtained in the present study are more similar to the Tarquinio et al. work, in which the score was calculated for 13 categories, considering the corresponding correction methods [19]. In addition, the dimensions were grouped according to the WHO (for three domains) and Pace et al. (for four domains) guidelines [12,44]. In this sense, as can be observed in the present study, the internal consistencies for the domains: household dysfunction, external violence, and child neglect, are rather low and inadequate. This same scenario is presented in the Tarquinio paper (for three dimensions) [19].

There are some reasons why this result might have occurred. For example:

a. The number of items affects the estimates of internal consistency [45]. In the Kidman and Swingen works, by using the score of all items, without employing any prior method of scoring, the number of items per domain increases, which could be affecting and improving the estimates of the internal consistency coefficients [15,43].

b. The context can influence the perception of some items, and therefore, the scores obtained on these items [41].In this case, as can be seen in Table 3, there are two items with factor loadings below 0.4. One corresponds to the category one or no parents, parental separation or divorce, and the other to the category collective violence. In the first category, which belongs to the domain dysfunction in the home, to obtain the score we include questions such as: ‘Were your parents ever separated or divorced? Did your mother, father or guardian die? Where you separated from your parents for a long period of time?’ These questions could be perceived differently from each other, which causes a decrease in the consistency of the answers, affecting the total internal consistency of the domain.

On the other hand, the second item with a low factor loading corresponds to the category collective violence, which belongs to the domain of external violence. In this case, as noted by Tarquinio et al., except for the fact that the categories bullying, community violence and collective violence are related to psychosocial violence, there is no real coherence between these categories [19]. Using the low factor loading criterion, if the collective violence category is eliminated, and only the internal consistency between the bullying and community violence categories is observed, the categorical omega value increases to 0.49 and 0.48 (binary and frequency correction, respectively).

For the reasons mentioned above, it is important in the future to review the validity of the instrument in detail. This could involve studying the instrument in other contexts and/or using the full item scores when assessing its psychometric properties.

Nevertheless, the initial validity evidence obtained is adequate, and it is expected that the use of the ACE-IQ will be an important contribution to the study of adverse childhood experiences in the Chilean population.