Exploratory Factor Analysis of a French Adapted Version of the Substance Abuse Attitude Survey Among Medical Students in Belgium

0. Introduction

As part of the implementation of a new optional substance use disorder (SUD) training program for final year medical students, we wanted to assess its impact on students in terms of skills development (pre and post-test evaluation). To evaluate the impact of the educational program of trainee’s representations and attitudinal skills development we looked for a questionnaire to assess their attitudes towards SUD adapted to our criteria. Various instruments were found in the literature in French and English, but they had a number of limitations for our study. The first was that most of them focused on either illicit drugs or alcohol and/or perceptions of professional attitudes (e.g. Drug and Drug Problems Perception Questionnaires and Alcohol and Alcohol Problems Perception Questionnaire [1,2] and we wanted to be able to assess both. Others were not culturally transferable to our context (e.g. Attitudes Mental Illness Questionnaire [3], some were limited in the dimensions they explored (e.g. The Addiction Beliefs Scale [4], Short Understanding of Substance Abuse Scale [5], Attitudes and Opinions Survey [6], or were difficult to use on a large scale given their qualitative or vignette analysis approach. Many also had limited «validity» and «reliability», not having been used beyond the study itself. On this basis, we selected the Substance Abuse Attitude Survey (SAAS) developed by Chappel et al. in 1985 [7]. Even though this questionnaire was developed more than thirty years ago, it was still relevant in our context. Indeed, it was designed to evaluate educational programs in the framework of initial or continuing education [8] and allows to explore 5 dimensions (also called factors). These are “Permissiveness” (implies accepting substance use within a continuum of normal human behaviour), “Treatment intervention” (relates to an individual’s orientation toward perceiving substance use/misuse in the context of treatment and intervention), “Non-stereotypism” (relates to a person’s non-reliance on popular societal stereotypes of substance use and substance users), “Treatment optimism” (relates to an optimistic perception of treatment and the possibility of a successful outcome) and “Non-moralism” (linked to an individual’s absence or avoidance of moralistic perspective when considering substance use and substance users). This questionnaire has been validated and used in many previous studies [9,10,11,12,13]. In our study, we selected its short version, the “Brief SAAS” [14] to maximise its completion since in our program it was administered in parallel with a knowledge test. Since an adaptation to our cultural and societal context was necessary, we named this new questionnaire the “bSAAS”( See Appendix A). The first objective of this study was to carry out an exploratory factor analysis of our bSAAS questionnaire in view of its adaptation to ensure its good internal consistency and structure. The second was to identify whether student’s representations were influenced by certain socio-demographic characteristics.

1. Materials and Methods

Questionnaire

The SAAS questionnaire validated widely by the literature consists of 50 items. For our study, we preferred its brief version, the Brief SAAS, which is used by Yale School of Medicine to evaluate their SBIRT (Screening, Brief Intervention, and Referral to Treatment) training programs and which is limited to 25 items. We adapted it to the Belgian context through consultation with various experts (including 3 general practitioners working in addiction medicine, 2 medical researchers, a SUD expert psychiatrist, 2 health sociologists and a professor of psychology). We removed the questions about marijuana experimentation among young people and Alcoholics Anonymous, which are less present in Belgium than in the U.S. We added a question on paramedical professionals who are much more involved than para-professional counsellors in our context. After consensus, we split the questions on alcohol and drugs to be able to assess whether there were different attitudes according to the substances consumed. This resulted in a questionnaire with 29 items (Table 1 and Table S1). We assumed that perceptions of legal and illegal substances might differ. Indeed, a common perception of the Belgian population is that the term “drugs” refers de facto to illegal drugs. This difference has been highlighted in studies conducted among Belgian doctors [15,16]. Each item was coded according to a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). To ensure the cross-cultural validity of the questionnaire, we used bilateral translation by 2 certified translators and pretested the questionnaire with trainees and medical doctors from different professional backgrounds and with lay people.

In order to characterise our population and to better understand the factors favoring the students’ representations, we included several socio-demographic data (sex, age, origin); data related to the personal and professional background linked to substance use; data related to the type of professional orientation (choice of specialty); as well as data related to the perception of their own health (we wanted to evaluate whether this influenced the way in which addicts are perceived). This questionnaire, which was called “bSAAS” by our research team, has already been used in a previous study to evaluate the attitudes of medical students towards substance use in pregnancy and showed a good internal consistency (Cronbach’s alpha = 0.77) [17].

Data Collection

The questionnaire was presented to 923 final year medical students in three consecutive years (2019, 2020 and 2021). It was administered face-to-face to final year medical students in 2019 and online in 2020 and 2021 given the context of the Sars-Cov-2 crisis. A total of 657 students completed the questionnaire with an average response rate of 71.1%. 80 students completed the questionnaire at the time of enrollment in the theoretical addictology course. We considered questionnaires with fewer than 10 responses, those with responses for only one of the two substances and those with no socio-demographic data to be invalid. On this basis, 615 questionnaires were retained.

Statistical Methodology

First, we analysed the correlations between the 29 items and performed Bartlett’s test of sphericity to ensure that a factor analysis was appropriate. Several items were highly correlated (r > 0.30) and the Bartlett test was significant (p <0.001). The Kaiser-Mayer-Olkin (KMO) measure was 0.71 and, our sample including 615 subject was large enough to perform factor analyses. The factor analysis was carried out in 3 steps and we decided to retain the factors with eigenvalues > 1 and with kat least 3 items with a weighting > |0.40|, while keeping a sufficiently high % of variance explained (75%) [18,19]. Five items (Table S1) and one item (Table S1) were not retained after the first and second factor analysis respectively. The final factor analysis included 23 items and three factors were retained. Factor scores were obtained for each of these factors, retaining for each factor only those items with a weight > |0.30| and taking the mean of these items. Five items were found in two factors, in this case the item was assigned to the factor with the highest loading or if the loadings were nearly equal, to the one that most closely matched the content of the factor. The mean scores were analysed according subject’s characteristics using Student’s t-tests or one-way Analysis of Variance (ANOVA). Where appropriate, ANOVA was followed by multiple comparison tests with Bonferroni’s correction or a linear trend test. Cronbach’s alpha was calculated to measure the consistency of the 23 items included in the analysis as well as the consistency of the items selected for each factor separately. All analyses were performed with STATA SE v16.1 software.

Ethics Committee

The research protocol was approved by the local ethics committee (Ethics Committee of ERASME-ULB) on February 25, 2019, ref: P2019/156. Informed consent was obtained from all participants who filled out and returned the questionnaire.

2. Results

Cronbach’s alpha was 0.80 for all 23 items. Considering the items with a weighting > |0.30|, the Cronbach’s alphas were 0.8, 0.70 and 0.59, respectively for the 10 items of factor I, the 9 items of factor II and the 4 items of factor III. We kept the same factor name for Factor I as mentioned by Jenkins et al. [9], i.e. “Stereotypes and moralism”, because the items correlated with this factor reflected value judgments. As in that study conducted on secondary school students, which retained only three factors (stereotypes and moralism, treatment and permissiveness), the students in our study did not seem to differentiate between stereotype and moralism items.

The name “Treatment optimism” for Factor II found in Chappel et al., 1985 and Jenkins et al., 1990, has also been retained. As in the first study, we find here negative correlations in relation to this factor since we have kept these items stricto senso (not recoded in the other direction). This also enabled us to define this factor well with items correlated negatively and therefore opposed to optimism, such as the item “An alcoholic who has had several relapses is unlikely to be treated”.

Factor III differed from the baseline studies [7,9] and seemed to be clearly marked by the specialised domain, with items correlated to hospital and specialist management in the field. We therefore named it “Specialised treatment”. These different factors and the corresponding items are shown in Table 1.

Concerning the 3 dimensions or factors retained in our final analysis: “Stereotypes and moralism”, “ Treatment optimism” and “Specialised treatment”, we separated them into 2 axes based on the 3 major learning domains conceptualised by Benjamin Bloom in 1956 (cognitive, affective and psychomotor) and further developed thereafter [20,21]. On one hand, we identified a psycho-affective axis, based on moral judgments, and on the other, a cognitive axis, linked to the perception of health care skills. The latter includes Factor II, which evaluates a dimension linked to results (the perception of the effectiveness of the treatment, “successful outcome”) and Factor III concerning the perception of the needed resources (material and human resources). The correlations between the scores derived from these factors (r = -0.32, 0.23 and -0.17 for score 1 and score 2, score 1 and score 3 and score 2 and score 3 respectively) were weak to moderate, which is an additional argument for maintaining a three-dimensional questionnaire. The score “Stereotypes and moralism” was positively correlated with the “Specialised treatment” score and negatively correlated with the “Treatment optimism” score. The score “Specialised treatment” was positively correlated with “Stereotypes and moralism” score and negatively correlated with “Treatment optimism” score (see Figure 1).

To be noted that the items dealing with the same content for drugs and alcohol had good or excellent correlations (r =0.55-0.85).

Secondly, we analysed the factor scores according to the different socio-demographic data collected (Table 2). There was a significant increase in the mean score for “Stereotypes and moralism” (Factor I) with age. This mean score was also statistically significantly higher for male respondents (P=0.037). The differences in the average score “stereotypes and moralism” according to “subject-parent” origin were significant (p=0.005), among people of non-European origin, we observed a higher average score than that of people of Belgian or Belgian-European origin. Subjects who didn’t consume substance had a higher “Stereotypes and moralism” mean score than people who had consumed but the differences were at the limit of statistical significance (p=0.065). Conversely, the average score of those who had ever used cannabis was low. This was also the case for those who had been in contact with patients with SUD in specific centers (e.g. addiction centers, harm reduction centers, prisons, etc.) or who had enrolled in the optional SUD training, who differed significantly from the other respondents in having very low average scores on “Stereotypes and moralism”. There was no statistically significant difference in the average factorial I scores according to the perception of one’s own health and SUD in the environment. The same was true for the choice of specialty, although students who wanted to go into gynecology had a lower average score. There was no statistically significant difference in the mother’s level of education either, although for this last variable, the average score was higher for those whose mother’s level of education was low.

For Factor II, “Treatment optimism”, the average score remained close to the mean of the total sample (2.94) across the different socio-demographic characteristics. This means that there was a tendency, independent of subject characteristics, to be optimist towards treatment and possible interventions. However, the 23 respondents who had no contact with people with SUD at work showed a clear tendency towards low optimism about treatment (average score lower than the others, NS).

In relation to Factor III, which assessed attitudes towards specialised treatment of SUD, male respondent had a significantly lower mean score (p=0.035). People whose origin was outside Europe had a significantly higher mean score than Belgian-European (p=0.019) or European (p=0.049). We also found a tendency for people with no contact with SUD patients, or with contact in hospitals, to be in favor of “Specialised treatment” (NS). On the other hand, those who went to general practice did not seem to be in favor of “Specialised treatment”. The same was true for respondents enrolled in addiction training (p=0.031). Details of these results can be found in Table 2.

3. Discussion

Our study, through the adaptation of a short version of the Substance Abuse Attitude Survey questionnaire administered to medical students in Belgium, allowed us to select a 23-item questionnaire with good face validity and content validity (via experts panel and pre-test), construct validity (via exploratory factor analysis, convergent correlations between alcohol and drugs items and discriminatory correlations between factor scores) and very good internal consistency (Cronbach’s alpha = 0.80). That allowed us also to highlight interesting results concerning influencing factors of medical students attitudes regarding SUD. The three dimensions selected: “Stereotypes and moralism”, “Treatment optimism” and “Specialised treatment”, which we have divided into two axes (affective and cognitive), are useful for exploring the learning objectives pursued in our educational system. These results can also guide us in the methodological choices to be made to achieve these objectives.

We can see some differences with the reference studies [7,9]. As the latter pointed out, the factor structure logically changes according to the characteristics of the people completing it. Our target population and questionnaire were somewhat different from those of the first study, conducted on different profiles of already experienced professionals (non-clinicians, clinicians not in the field, and clinicians in the field) and those of the second study conducted on college students (first degree of university). This may partly explain the difference in the factors observed.

In our sample, which also seems to be representative of our target population with a good response rate (71.1%) given the circumstances (Sars-Cov-2 crisis) and the survey carried out partly online, we find an over-representation of women, which corresponds to the feminisation of medicine over the last few years and is in line with the latest gender report from our university [22], which reports 62% female students in medicine. We also see that 72.7% of students have a mother with a high level of education. These attitudes are therefore reflecting a selected population of future caregivers who are very likely to differ, also by the number of years between these studies, from the populations studied in previous articles and from the general population at the socio-demographic level (gender imbalance and particularly high educated population).

With regard to the factors, it is firstly interesting to note that the dominant factor in our study (explaining 40.3% of the variance) was, like that of the college students, the factor relating to “Stereotypes and moralism”. This differs from the case of Chappel et al., 1985, where it was the factor relating to permissiveness. The predominance of this affective dimension in a medical student audience can be explained in different ways. Firstly, we hypothesise, as the SUD topic being little taught in the formal curriculum, that their representations are at this stage still based on representations conveyed by society or by their community. These representations are probably also influenced by their internship experience, where the impact of role modelling (defined here as “a teaching by example and influencing students in an unintentional, unaware, informal and episodic manner” [23] is documented on students [23,24]. Students may integrate both positive and negative behaviors of their role models and mentors, perhaps more so in the context of choosing a career path, where they are looking for identifying models. The informal and hidden curricula play a key role in the transmission of attitudes and values [24]. This is consistent with the study of Kidd et al. in 2020 [25], which justified the fact that students tended to adopt more negative attitudes over time by the potential impact of the «hidden curriculum» through which students internalise the “negative” attitudes of their supervisors. The hidden curriculum includes “a set of values, behavioral norms, attitudes, skills, and knowledge that medical students learn implicitly” [26]. In our study, the (non) place of contact seemed to play an important role on stereotypes and moralism, the latter being particularly marked if there was no contact or if it took place only in the hospital system. Conversely, scores were low if students had had multiple contacts or in specific settings for patients with SUD such as SUD centres. This demonstrates the importance of varied and supervised experiences to reduce stereotypes. The fact of never having used substances on one’s own also seemed to encourage these stereotypes. All these elements are in line with the literature [17,27,28].

The difference between the factors emerging from our study and our reference articles can also be explained by the fact that in our final analysis we no longer had any of the items correlated with the “Permissiveness” factor initially present in the study by Chappel et al. Some of those items were not included in the “Brief SAAS”, some of which differed from the initial questionnaire, and other items were removed from the “bSAAS” because they were not adapted to our context, such as experimental marijuana use among young people, or were not retained during the 2^nd or 3^rd stage of the factor analysis leading to the “beSAAS” (See Appendix B).

In relation to the 3^rd factor that we have named “Specialised treatment”, it seems to be well characterised in our study by the 4 items correlated with it (referring to care by specialists and in hospital) and thus seems to be better defined here rather than in previous studies where they were associated with the notions of stereotypes and moralism among other items. Nevertheless it does seem to be stereotypical to believe that the complexity of the situations should be dealt with by the second line of care. Recent studies on opiate use disorders show, on the other hand, that primary care setting is the most appropriate [29,30], although this is more controversial for alcohol, even if its role of early detection and intervention is essential [31]. The fact that in our study this emerges as a factor in its own, in contrast to the two studies mentioned above, is indicative of our target population. Here we can also see the impact of the “hidden curriculum” on medical students, as mentioned by Sc. Mahood in 2011 [32]. Indeed, in the medical curriculum of our Faculty, the importance and primacy of specialties is regularly emphasised outside the formal curriculum. The training being mainly hospital-centered as well. The fact that general medicine is not sufficiently valued in the learning process is certainly internalised by the students, especially a few months before the final exams and selection for career choice, and may influence these stereotypical representations (these two scores being positively correlated in our study, albeit moderately so).

As for the results of the factor scores, we can see that the characteristics of the subjects with higher scores for stereotype and moralism are similar to those who were in favor of punishing substance use during pregnancy (alcohol/drugs) in our previous study [17], these 2 items being the most «weighted» in this first factor.

In conclusion, it is interesting to note that the people who enrolled in the SUD training scored lower on stereotypes and moralism compared to those who did not decide to attend, but there wasn’t a statically significant difference in terms of optimism about treatment between the two groups which was a tendency to be in favour. This leads us to the conclusion that in this optional training, we will especially focus on improving their perception of treatment outcomes through the sharing of positive experiences and teaching a vision of recovery and long-term support for this chronic pathology (“care vision”) rather than focusing on curing the individual (“cure vision”) as is commonly taught in medicine. This study also reveals the interest and necessity of making a SUD training program compulsory for all medical students, in addition to the basic SUD education, to improve attitudes, access and quality of care for people with SUD, which all these future caregivers will have to face [27,33,34]. It should be noted, however, that the fact that the people taking part in the training were rather not in favour of specialised treatment for this group can be explained by the way in which this training is offered by the Department of General Medicine and students that are heading for general medicine are the more likely to take part.

Although this study made it possible to validate at least a questionnaire for evaluating representations in French that could be used to evaluate educational systems, it does have certain methodological limitations. Indeed, we were not able to ensure its external validity by measuring its correlation with the basic instrument or other related instruments. Nor have we assessed its reliability in terms of repeatability (test-retest) and reproducibility on other profiles. It would also be useful to conduct a confirmatory factor analysis on the basis of our new 23-item questionnaire (bSAAS) to validate its construct. Further studies could be conducted in the future to do this. It is also a questionnaire filled by the students themselves which has its limits in terms of assessment of students’ attitudes in real practice and may have social desirability bias.

4. Conclusions

The adaptation of the Substance Abuse Attitude Survey to our cultural context and our target population of medical students seems to have good validity and internal consistency. The questionnaire selected, beSAAS with 23 items and its three-dimensional interpretation through 2 axes: affective and cognitive, seems useful and relevant to evaluate the impact of a pedagogical program on students’ representations. This study was able to highlight certain factors influencing stereotypical representations such as age, gender, personal or professional experience with substance use. The factor evaluating the interest of “Specialised treatment” clearly emerged in our study and seems to be explained by our target population and its representations influenced by the formal, informal and hidden curriculum. The study highligthed also interesting findings for improving medical education to provide better care for SUD people. Further studies are needed to complete the validity and reliability of the questionnaire.