Application of the Delphi Method for the Design and Content Validity Analysis of a Questionnaire to Determine the Risk Factors of the Chemsex

1. Introduction

Chemsex understood as "the intentional use of stimulant drugs to have sex for a long period of time among gay, bisexual and other men who have sex with men" is a public health problem because of the increased incidence of cases [1,2] and because of the consequences on the physical and mental health of those who practice it [3,4,5,6]. According to Madrid's recent 2022-2026 Addictions Plan, care for people who practice chemsex has increased from 50 in 2017 to 351 in 2021, an increase of 602% in the number of people served in recent years [7].

It should also be noted that the increased use of geolocation applications has led to an increase in unprotected sexual encounters, which is associated with increased risk behaviors for sexually transmitted infections (STIs) [4,8]. Previous studies indicate association of chemsex with the priority consumption of substances such as mephredrone, methamphetamine, cocaine, GHB/GHL [9] and the search for achievement of emotions, pleasurable sensations, and management of negative symptoms [10]. Other studies indicate that individuals with substance use-related problems used in chemsex may have experienced early adverse events and may have an avoidant-insecure attachment style. In addition, those who have been diagnosed with HIV may show greater emotional dysregulation and worse self-care patterns. These variables should be routinely assessed in this population [11].

The assessment of mood disorders and addiction linked to the practice of chemsex is of interest given the psychoactive substances used [11]. The practice of chemsex has been linked to increased suicide sexually transmitted infections (STIs) [12,13,14,15] increase in psychosis [16] mental problems [9] and decreased adherence to pre-exposure prophylaxis treatment (PrEP) among others [17].

It is of interest to raise awareness of chemsex as a public health problem among gay, bisexual and men who have sex with men (GBMSM). Specific identification, education [18] and prevention programs need to be strengthened to reduce the incidence of the most undesirable implications of sexualized drug use (USD) among GBMSM [10].

The literature indicates the importance of self-monitoring for the reduction of harm from chemsex use [19] as well as the development of different programs that allow through computer applications [18] applications (app) [20,21] to support and inform participants, reduce the negative impacts associated with chemsex and encourage more reasoned participation. On the other hand, the lack of knowledge of professionals in our country [22] regarding a growing problem should be considered.

Thus, studies such as Nagington's in 2022, indicate that we suggest that medicalized forms of chemsex support could benefit from more rigorous and rapid forms of assessment for problematic chemsex, and provide infrastructure and training for peer support initiatives. We also suggest that medical services can learn from patients and their peers about support needs that professional services continue to miss and engage in collaborative approaches to practice development [23]. Early detection and knowledge of risk factors can contribute to the reinforcement of accessible, non-judgmental, and well-informed prevention and harm reduction activities to support MSM who engage in slamsex [24]. In addition, an equity-oriented approach should be adopted to facilitate unbiased care opportunities [25].

However, to date, the risk factors for chemsex have not been specifically addressed, nor is there currently any instrument available to assess them. All of the above can slow down the detection of these warning signs by the health professionals who work with them, who care for them and understand that the quality of life of these people is fundamental. Therefore, to improve their care and holistic well-being, it is necessary to know the predictive behaviors. To address and respond to the needs, the objective of this study was to design a questionnaire to detect risk factors for chemsex.

2. Materials and Methods

2.1. Desing

The study carried out was of the instrumental type, which, according to Montero and León (2005), is research that develops tests and devices, comprising both the design or adaptation and the study of their psychometric properties. The scale is literally translated from Spanish into English and will therefore be subject to validation in Spanish.

2.2. Item Bank Construction

The questionnaire was developed between July and August 2022. A literature search was conducted, and reference studies related to the research topic were analyzed, verifying the absence of instruments available for the study. The initial version of the questionnaire entitled "Chem-Sex Inventory" (CSI) was organized into six sections consisting of 114 items from various validated scales. The first block related to anxiety to 7 items (from 1-7) (GAD 7) [26]; the second block related to depression (PHQ-9) [27]; consisting of 9 items (from 8-16 items), the third block related to the risk of psycho (PQ-B) [28]; Scale of Corrigan [29]; and CAPE-15 scale [30]; consisting of 21 items (from 17-37), 14 ítems (from 88-99) and 15 ítems (from 100-114) respectively, the fourth block related to impulsivity (BIS-11) [31]; consisting of 30 items (from 38-67), the fifth block related to body perception (PHQ-15) [32]; consisting of 15 items (from 68-82) and finally the sixth block related to suicide risk and consisting of 5 items (from 83-87) (Paykel Scale) [33]. From the Corrigan Scale, the group of researchers initially eliminated two items associated with hospital admission.

Each member of the research committee, including all the authors of the study, collaborated in the development of the questionnaire, defining its structure and main characteristics, selecting the items and reviewing them.

2.3. Selection of Experts

Fifty experts on the subject were contacted, understanding as experts those professionals who had more than 5 years of experience in their field and professional trajectory and at least two years of experience in the management of users who practice Chemsex, had knowledge of the Chemsex phenomenon, were active specialists in their field and had a direct relationship with users who practice Chemsex. The 50 experts were made up of: 10 LGTBI+ individuals, 10 mental health professionals, 10 emergency and urgent care professionals, 10 primary care professionals and 10 professionals from infectious disease and/or sexually transmitted infection (STI) units.

The experts were invited to participate in the study directly by e-mail. Together with the e-mail, an information sheet was sent describing the objectives of the research, as well as the selection criteria, the confidentiality of the data and the voluntary nature of the study. The participation of this group of experts was carried out voluntarily, anonymously, and confidentially by means of a questionnaire through the Microsoft Forms platform. Before disseminating the first survey, the experts identified were asked to accept the Declaration of Consent if they were interested in participating in the study according to the Data Protection Law in force in Spain.

2.4. Delphi Method

The conventional Delphi method was used, through an iterative process in which experts are consulted in two rounds [34].; Linstone and Turoff (1975) consider that two rounds are sufficient to reach a consensus, which would allow adequate reflection on the group's responses [35].

This process was developed during the months of September and October through different phases. Each of the experts was asked to evaluate both the relevance and comprehensibility of each of the items using a Likert-type scale between 1 (strongly disagree) and 5 (strongly agree). A qualitative question on the relevance and clarity of the sections was also added. In addition to criteria of completeness, wording and structuring of each item. Secondly, the responses of the group of experts were received. Subsequently, the discussion group was held, where suggestions were taken into account. Finally, the experts' responses were collected, integrating the pertinent modifications, and the final version of the questionnaire was defined.

2.4.1. Round 1. Content Validity / Linguistic Validity and Loss of Experts are Evaluated.

The first round of consultation was used to evaluate both the content validity (appropriateness) and linguistic validity (comprehensibility) of each of the items. After this first round, the number of items to be evaluated in the second round was significantly reduced.

2.4.2. Round 2. Content Validity Assessed.

In the second round, the content validity of the items was evaluated (although in some cases there were still some items with adequate content validity but low scores in comprehensibility that were re-evaluated for comprehensibility).

2.5. Content Validity Analysis

The content validity of the questionnaire was analyzed by calculating the content validity index and Aiken's V value for each of the items. A minimum CVI and Aiken's V value of 0.6 was established to include the items in the questionnaire, this being the criterion used to select the items. Based on the experts' scores, the indicators were calculated. Following the methodology described by Polit and Beck [36]; and used by other authors [37,38,39]; the content validity index of each of the items (IVC and Aiken's V) was calculated individually, based on the ratings made by the group of experts, using the following equation:

CVI = number of experts who evaluated the item with 3 or 4 / Total of experts

The equation, algebraically modified by Penfield and Giacobbi (2004), is:

V=X−l k

X is the mean of the expert ratings in the sample, l is the lowest possible rating, and k is the range of possible values of the Likert scale used. For example, if l = 1 and k = 5, then k = 5 - 1= 4. The estimation of the confidence intervals for Aiken's V coefficient will use the score method [40].

The CVI and Aiken V indices were calculated with a database created in Excel 2013, using the assessments of the Expert Group and according to their respective formula.

2.6. Comprehensibility Analysis / Linguistic Validation

To obtain the validity of comprehension. The experts were asked to evaluate the degree of understanding of each of the items in the first round and whether they considered that any of the items in the questionnaire should be reformulated. It calculated the average score of such items. Items with scores above 4 were considered to be of high comprehensibility; those with scores between 3.5 and 4 were considered to be of medium comprehensibility; and those with scores below 3.5 were considered to be of low comprehensibility. The items that obtained lower scores in the first round and were selected for the second round because of their content validity were reformulated and their comprehensibility was re-evaluated in the second round.

3. Results

3.1. Characteristics of Experts

Of the 50 experts initially considered, 72% (36 experts) agreed to participate and completed the first round. The second and final round was completed by 30 experts (60%), 70% of whom were men and 30% women (Table 1). Other noteworthy characteristics of the group of experts are: eight mental health professionals, five emergency professionals, six primary care professionals, seven infectious disease professionals and four LGTBI+ professionals. They came from seven autonomous communities: Aragón (n=1); Extremadura (n=2); Canary Oslands (n=2); Catalonia (n=5); Valencian Community (n=1); La Rioja (n=3); Madrid (n=16). In relation to profession, 56.66% (n=17) were nurses, 33.33% (n=10) were physicians, 6.66 (n=2) were psychologists, and 3.33% (n=1) were social workers and sexologists.

3.2. Results of the Delphi Method

Figure 1 shows the questionnaire development process and the content validity index (I-CVI) of each of the 114 items from the Delphi rounds. Of these 114 items, 56 (49.1%) had an I-CVI >0.6 after the first round and 58 items that had an I-CVI <0.6 were eliminated. After the second round, 52 items (92.8%) were above the cutoff value, eliminating the remaining 4 items.

3.3. Content Validity Analysis

The criterion used to select the items that made up the final questionnaire was that the CVI value or the Aiken V test score was greater than 0.6.

The first round was completed by 36 experts. After review of the first round, those items that had obtained a minimum CVI and VdA value of 0.6 were selected. Of the 114 items that made up the questionnaire (Annex 1), 61 items met the criteria of permanence and relevance. Of these 61 items, we proceeded to the analysis of the experts' observations for reformulation and understanding of items 43, 44, 63, 65, 65, 73, 91, 92 and 97. The rest of the experts' suggestions were not taken into account because they were related to items that did not obtain a value higher than 0.6 and were therefore eliminated from the questionnaire. In addition, the group of experts considered that questions 11 and 81, 16 and 84, 10 and 82, 65 and 67, 9 and 83 (which, in principle, all met the criteria for permanence) were similar, so 5 of these 10 questions were eliminated, leaving 56 items finally selected after the first round.

The second round was completed by 30 experts out of the 36 who initially completed the first round. After review of the second round, the same criteria as in the first round were maintained, selecting those items that had obtained a minimum CVI and VdA value of 0.6. Of the 56 items in the questionnaire, the items that did not obtain a minimum CVI and VdA value of 0.6 were eliminated, leaving 54 items that met the criteria. According to the experts' observations, questions 11 "Have you had difficulty concentrating when doing everyday things, such as reading the newspaper or watching television?" and 26 "Are you a person who has difficulty concentrating?" were similar, so item 26 was eliminated because the question was less complete. Subsequently, after review by the research group, two similar items were observed, item 55 "Have you ever heard voices when you were alone?" and item 56 "Have you ever heard voices talking to each other when you were alone?" and the second item was removed. Finally, a questionnaire with 52 items was obtained.

When analyzing the content validity of the 52 items through the CVI and Aiken's V, the researchers eliminated 10 items for not being relevant to the topic even though they had values above 0.6. The values of CVI and Aiken's V for each of the items that make up the questionnaire are shown in Table A1 (Appendix A). Analyzing the CVI and Aiken's V for each item, we see that 97.6% of them reached the value of 0.6 considered acceptable. Only the following two items: "Have you ever seen things that other people cannot see?" and "Have you ever felt as if you were under the control of any external force or power?" scored below 0.6 in one of the two indicators but the other indicator was above 0.6. Finally, the questionnaire consisted of 42 items (Table 2).

3.4. Comprehensibility or Linguistic Validation Analysis

The reports on the items were minimal, some spelling and grammatical changes and syntax structure, for example, item 13 requested to reformulate the question "has felt bad about himself, that he has failed or has failed". The rest of the items were eliminated and were not reformulated. Items 2, 15, 40, 84, 93 and 95 were eliminated from the questionnaire because they presented a low level of comprehension with a value lower than 4. Item 13 was modified because it had values lower than 4 in comprehension; however, after its reformulation, it was considered important to keep it in the instrument.

All the active experts who participated responded that they understood, without difficulty, the content of the final questionnaire designed, the concepts and the answers to each of the items in terms of adequacy and comprehension. Finally, no expert detailed any doubts regarding the completion of the questionnaire.

4. Discussion

Given the importance regarding the increase in the prevalence and public health problem of Chemsex consumption, we proposed at the beginning of the work the design and validation of a scale entitled "Chem-Sex Inventory" (CSI) to reach a consensus on the content in the development of a questionnaire [42,43], based on the Delphi method and analyzing the content validity of the questionnaire to be able to know the behaviors that predict consumption, allowing a better approach to people who practice chemsex and an advance in research on this type of addictions [44].

As a strength of the study, this instrument will facilitate the development of future studies with the aim of analyzing and relating its construct to different variables such as substance use, impulsivity, altered body perception, risk of psychosis, risk of suicide, anxiety and depression. Its application will allow multidisciplinary teams of professionals to plan and develop a better approach to chemsex patients, as well as to know their behavior and conduct. In order to improve personal outcomes related to quality of life.

It used the Delphi method by being a relevant and useful method in the health area [45], characterized to get a reasoned opinion, consensual and individualized in relation to the analysis and reflection on a given research objective [46,47].

First, professionals who had practice-based knowledge and experience in the context of Chemsex and therefore could make valid contributions to the study were chosen as experts. Validation aims to ensure that the questionnaire documents what it is intended to measure and that its design and validation are of rigorous scientific quality. Such validation was carried out with experts to achieve an optimal level of validity, defined as the degree to which all representative indicators intended to be assessed are included in the questionnaire [44].

Furthermore, the group of experts consisted of a heterogeneous group and belonged to different regions and work services, which is fundamental because it allowed a different view from different points of view. Different studies indicate the strengths of using the modified Delphi technique as each round is conducted anonymously and independently, with the advantage of opportunity for participation in the study and reducing response bias that can appear in group settings [41]. Thus, all expert group participants had equal opportunity to participate in the study reducing the risk of response bias that can arise in group settings [41,48].

Secondly, the methodology used was the delphi technique, a consensus technique that allows quantitative estimators to be obtained through the degree of agreement among the participants. This is an effective method for building and creating consensus in a group without the group of experts having to meet in person [43], but rather contacting each of the group members via e-mail. With all of the above, it is necessary to have validated tools adapted to our environment that allow us to know the behavior and conduct of people who practice Chemsex.

Regarding the characteristics of the present study, we can assure that the content validity is high. As we have seen, the decision-making of the group of experts has become increasingly uniform in the different Delphi stages, which is reflected in the significant increase in the congruence index of the items included [44]. The group of experts considered that the 42 items of the “Chem-Sex Inventory” (CSI) questionnaire, having a high level of content validity, capture aspects related to the predictive behavior of Chemsex, and therefore present a high level of content validity [41]. Being a first version of the questionnaire, the research group eliminated second-round items with a CVI and Aiken's V higher than 0.6 as they were not relevant to assess the risk behaviors of Chemsex phenomenon, and kept items with a CVI and Aiken's V lower than 0.6 that were considered relevant to the study.

By means of the Delphi methodology, a high degree of agreement of the group of experts in the elaboration of the questionnaire was observed [44]. In the absence of content validation previously on the Chemsex phenomenon in the literature reviewed as in other situations (UPP, emergency department, etc.) and in the absence of tools to predict risk the development of new tools is necessary [49]. This study can be used in the future to inform other researchers in their own efforts to validate the content of behaviors predictive of the Chemsex phenomenon [43]. Following this it is planned to conduct a study of the LGTBI+ population that performs Chemsex in order to be able, after the resulting analysis, to assess additional aspects of its validity and reliability, and, based on the results, to perform item reduction, and subsequently establish a cut-off point to discriminate risk and allow for group stratification.

In this sense, the panel of experts made a quantitative and qualitative contribution that allowed improving the tool [50], obtaining very positive values in all dimensions and their assessment category, namely, the relevance of the reference question and its response category: Relevance to the object of research, clarity, adequacy and comprehension of the wording, structure and sequence of dimensions and questions. After content validation by experts, a final questionnaire was developed, which is in the process of validation of other psychometric properties.

For future studies, some limitations should be kept in mind; first, a diverse sample of experts working in different systems and in different autonomous communities was selected to obtain a broader perspective on the chemsex phenomenon. Although previous research identified snowball sampling as an effective method for identifying expert groups in Delphi studies [51], it was found that the application of this sampling method in the present study may have resulted in the inclusion of some non-experts in the sample.

This may indicate that those with less experience may have limited exposure to some aspects of chemsex practice. As this was a study based on expert opinion, and although the team of experts was sufficiently broad and knowledgeable about the phenomenon, there is always the possibility that not all aspects or dimensions of the phenomenon were addressed. Another of the limitations encountered is that the resulting questionnaire is relatively long in our opinion, which may lead to abandonment in the responses of users in future research. Since this is a first version of the "Chem-Sex Inventory" (CSI) questionnaire, it is the aim of the research team to shorten the questionnaire in successive versions while maintaining its properties. Por todo lo anterior, investigaciones futuras que utilicen muestras de expertos deben considerar hasta qué punto las experiencias de los participantes se ajustan a los objetivos de la investigación.

5. Conclusions

In conclusion, the results of the present study allow us to conclude that the questionnaire designed to determine the risk factors for the Chemsex phenomenon has a high level of content validity and can therefore be used in the different emergency departments, primary care, psychiatry and infectious disease departments for this purpose.