Development and Content Validation of a Clinical Screening Scale to Identify Attention Deficit Hyperactivity Disorder Considering the Gender Perspective: An eDelphi Study

1. Introduction

Attention Deficit Hyperactivity Disorder (ADHD) has been widely documented in the literature, which supports its existence, although it has been criticised as a diagnostic category. Its inclusion as a disorder in the Diagnostic and Statistical Manual of Mental Disorders III (DSM-III) in 1983 generated controversy as it was initially defined by the American Psychological Association (APA) in 1980 as a list of generic behaviours. In 1998, the American Medical Association noted that ADHD was one of the most studied disorders in medicine, recognising its significant prevalence and chronic nature as a public health problem [1].

Since then, it has maintained its status as a diagnosis and has become one of the most common childhood disorders, with a prevalence of 8.8% of the world's population according to the World Health Organisation (WHO) report of June 2022 [2]. It is present in 3.1% of adolescents aged 10-14 years and 2.4% of those aged 15 years and older [3]. ADHD is one of the most common chronic disorders in childhood, the effects of which can persist into adolescence and adulthood. With a neurobiological aetiology, the core symptoms manifest in childhood before the age of 12, and its course is chronic, evolutionarily symptomatic, and with a high rate of heritability [4]. In Spain, the prevalence is between 5% and 7% in the school population [5]. Many children and adolescents with ADHD face difficulties in regulating their behaviour, which can result in adjustment problems in family, school, and social settings. They often perform below their ability and may experience emotional and behavioural disturbances [6].

Research has found that children diagnosed with ADHD show comorbidity with other disorders, difficulties in interpersonal relationships, and behavioural problems that directly affect their academic performance, family, and relationships in adulthood [7,8,9,10,11,12,13].

In recent years, there has been an in-depth analysis of how gender influences ADHD symptomatology, clinical presentation, diagnosis, and treatment [14,15,16,17,18,19]. Several studies indicate that the symptomatic basis of ADHD differs between boys and girls, with a greater tendency in boys towards hyperactive and impulsive symptoms, while in girls, more inattentive and anxious symptomatology is observed [14,20,21,22,23]. This appears to relate to the perceived stereotype for parents and teachers of ADHD, which tends to be associated with the 'disruptive child' in line with perceived DSM-IV diagnostic criteria [24,25,26].

Current scientific literature has documented gender differences in psychosocial, cognitive, and psychiatric functioning in boys and girls with ADHD [27,28,29,30,31,32,33,34]. In addition, under-diagnosis or under-identification has been observed in girls with ADHD, with studies suggesting a ratio of diagnosis at younger ages ranging from 2:1 to 10:1, with higher ratios in clinical settings [35,36,37,38,39,40,41].

According to the scientific literature, in the assessment of ADHD, instruments to measure symptoms and executive functions are essential; interviews with the patient and family members, as well as self-report scales and questionnaires, are important to understand the intensity of symptoms and their specificities [42,43]. However, many of the instruments used lack a gender perspective, which highlights the need for tools that are sensitive to the symptomatology presented in girls, as demonstrated by the extensive literature [44]. Some of the most widely used and specific scales for the detection and diagnosis of ADHD do not differentiate between boys and girls [14] or existing instruments are analysed according to gender to make them more reliable in their discriminative function [45,46].

This study aims to design an instrument for the early detection of ADHD that considers the gender perspective and is sensitive to the detection of girls with ADHD. This instrument must have sufficient content validity and be able to differentiate ADHD from other disorders with which it could be confused, as well as being sensitive to the symptomatology of ADHD in girls.

2. Methods

2.1. Study Design

For the construction of the ADHD detection scale from a gender perspective, the Delphi technique was used in its electronic form, e-Delphi [47]. Figure 1 shows the procedure followed for the e-Delphi method [48,49].

The Delphi technique has been widely used in health research [47,50,51,52,53] to establish consensus and convergence of opinions in a structured way among a group of experts on different types of issues and topics.

This anonymous process makes it possible to ascertain the assessments and degree of agreement of the group of experts about different questions that are posed through multiple rounds of surveys distributed by e-mail in this case. Anonymity among the expert professionals themselves allows them to express their opinions freely.

In the same way, it prevents influential participants from directing or guiding the answers, and this has an important impact on the results. In this case, the telematic method has been used as it allows the research team to recruit professional experts from different origins and geographical dispersion. Thus, each professional expert receives the information and responds autonomously and individually.

The e-Delphi was structured in three rounds; for each round, three specific forms were elaborated and sent telematically to the panel of experts.

In the first round, experts were asked to assess which DSM-5-TR symptoms of inattention and hyperactivity are more frequent in the symptomatological expression of ADHD in girls, and then to identify which symptomatology would be specific only to girls.

The second round consisted of assessing the symptoms identified in the first round specific to ADHD in girls. Given that the symptomatology of Inattention and Hyperactivity/Impulsivity are validated by the DSM-5-TR; the aim of the second round was for the experts to rate the new items that would make up the scale and that would be sensitive to the symptomatology of ADHD in girls.

The third round the research team transformed the symptomatology into a broad pull of items to be included in the scale. The items distributed in the three dimensions, Inattention, Hyperactivity/Impulsivity and "Internalisation" made up the first version of the scale that was submitted to expert opinion, in which each item was re-evaluated to be included or not in the final version and the most suitable format for the scale.

The final proposal of the scale met the criteria of relevance, coherence, and clarity in its formulation to determine the degree of consensus and convergence of opinion.

2.2. Theoretical Structure of the Scale

For this study, we have used the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders-5-TR (DSM-5-TR) as the basis for developing the scale. These criteria are widely accepted internationally for diagnosing ADHD. The diagnosis of ADHD is based on whether or not a person meets these criteria, which describe the main symptoms of the disorder.

Most of the instruments used to diagnose ADHD are based on the perceptions of parents and teachers, but some have been adapted to be completed by the children themselves. After reviewing the literature, it was decided to create a self-report scale that is gender-sensitive and includes internalised symptomatology. Although the scientific literature describes differential symptomatology between boys and girls with ADHD, no internationally validated instrument has addressed this specific dimension.

2.3. Questionnaire Design

To ascertain the opinions of the experts and reach a consensus, three online forms were created on the Google Forms platform [54]. The forms were sent with a brief description of the project, the objectives, and the corresponding instructions. As they were online forms, they had the advantage that the expert professionals could access them from any personal device and at the time that best suited their schedule. Therefore, by allowing a reasonable response time, it was easy to collect and compile the responses.

The forms were designed in such a way that all questions had to be answered by all experts; they were mandatory. This ensured that there were no missed questions or omitted data, which would have complicated the analysis of the responses.

2.4. Data Collection

The first round of the e-Delphi method was launched electronically in May 2023, the second round in October 2023, and the third round in January 2024. The forms were organised coherently with the development of the scale, i.e. an orderly timeline was followed that accompanied the whole process of creating the scale. There were three rounds: the first, in which experts were asked to identify, based on the DSM-5-TR criteria, key symptomatology for the early detection of ADHD, classifying them according to the differential symptomatology between boys and girls aged between 6 and 16 years.

The second round consisted of assessing the symptomatology identified in the first round that would help to specifically identify ADHD in girls. Each identified symptom was individually rated on a numerical scale from 1 to 4 (1 being not relevant at all and 4 being very relevant). In addition, experts in this round could suggest items that had not been previously considered.

Finally, the third round consisted of transforming the symptomatology into a broad pool of items, which were distributed in the three dimensions (Inattention, Hyperactivity/Impulsivity, and "Internalisation"). The items distributed in the three dimensions made up the first version of the scale, which was submitted to expert opinion, where each item was re-evaluated to be included or not in the final version and the most suitable format for the scale.

Finally, the final version of the scale was pilot-tested on a group of 20 children aged 6-12 years to assess the comprehension of each item and the appropriateness of the format.

2.5. Participants and Recruitment

To conduct this study, it was deemed essential to consult with leading health professionals involved in the field of mental health and child patients, both in the private and public sectors. Hence, a purposive sampling approach was undertaken.

Communication was established with experts possessing extensive experience in the primary topic of the study: the approach and diagnosis of ADHD in childhood and adolescence. These professionals were selected based on their track record in child mental health, as well as their contributions and publications in mental health and gender studies. They were required to have a minimum of 10 years of clinical experience in the study's field. Direct email invitations were sent to recruit the experts.

The professional panel comprised a total of 10 experts, including a neuro-paediatrician, two child psychiatrists, two clinical psychologists, two neuropsychologists (from both public and private sectors), a general health psychologist, a psychologist researcher in gender studies, and a nurse and researcher specializing in mental health. Regarding the sample size, the total number of professionals participating in this study falls within an acceptable range. The Rand Corporation, an international research and development organization, suggests that with a minimum of 7 experts, the margin of error decreases significantly for each additional expert, and it is not advisable to exceed 30 experts [50,52].

2.6. Data Analysis

With the expert panel set up and the three rounds of the e-Delphi completed, the results of the e-Delphi were coded and analysed descriptively as responses were generated, taking into account the general patterns of response and their frequency and percentages, indicating convergence of opinions, consensus, and achievement of the objective.

3. Results

The final e-Delphi results have been analysed to directly achieve the purpose of the study, to obtain a gender-sensitive ADHD screening scale for boys and girls.

3.1. Round 1

Table 1 shows the frequency of Inattention and Hyperactivity/Impulsivity symptoms assessed by the expert panel. Symptoms related to Inattention had the highest frequency and agreement compared to the symptoms associated with Hyperactivity.

The expert panel identified 7 possible specific symptoms for the detection of ADHD in girls. These symptoms were grouped under the criterion "Internalising". Table 2 shows these symptoms.

3.2. Round 2

In this second round, the experts assessed the relevance of the symptomatology identified in the previous round as specific symptomatology for detecting ADHD in girls. Four of the seven symptoms generated were rated as relevant (3) or very relevant (4) by the expert panel (Table 3). No expert considered adding any additional symptoms to the proposal.

3.3. Round 3

Following the assessment of items in the previous rounds and considering all contributions from the panel of experts, the research team compiled a list of 62 items distributed across three dimensions. The first dimension focused on Inattention with 21 items, the second on Hyperactivity/Impulsivity with 20 items, and the third on a differential symptomatology termed "Internalisation" with 21 items.

Each expert evaluated every item to determine its relevance for inclusion in the scale. Out of the 21 items proposed for the Inattention dimension, 10 received full consensus from the experts for inclusion in the final scale. Similarly, for the Hyperactivity/Impulsivity dimension, 10 out of the 20 items proposed were agreed upon, and for the Internalisation dimension, 11 out of 21 items gained expert consensus. Consequently, a final list of 31 items was established.

In the latter part of the third phase, the panel was asked whether they would propose additions or alterations to the scale's content and format. Based on the expert input, 3 items were reformulated, and an additional 3 items were included, deemed significant for identifying ADHD in girls.

Requests were also made regarding the standardization, adaptation, and expansion of the scale to include versions for parents, schools, and adults. It was suggested that in the final version, items should not be distributed by dimensions to prevent response selection bias.

The final scale comprised 34 items, with 11 items corresponding to the Inattention dimension, 10 to the Hyperactivity/Impulsivity dimension, and 13 to the Internalisation dimension. Each item on the scale is assessed using four possible responses: 1 for NEVER or ALMOST NEVER, 2 for FEW TIMES, 3 for MANY TIMES, and 4 for ALWAYS or ALMOST ALWAYS.

The summary table of the proposed scale is shown in the table below: Table 4.

The scale is presented in Appendix 1: Model instructions: ADHD SCALE FOR BOYS AND GIRLS - SELF REPORT-6 to 16 years; and Appendix 2: ADHD SCALE FOR BOYS AND GIRLS - SELF REPORT-6 to 16 years.

Finally, a pilot test was conducted with children to assess their comprehension of the items and the scale format. This pilot test involved a small sample of 20 Spanish-speaking boys and girls from different schools and towns in Catalonia aged between 6 and 16 years old, as older children are expected to encounter fewer difficulties in comprehending the items. For the pilot test, the corresponding informed consents were signed.

During the review conducted by the children, no significant changes to the scale were deemed necessary, except for adjustments in vocabulary and the adoption of less formal and more accessible language. For instance, "I feel overwhelmed" was changed to "I find it difficult to control," and "to decide between two options" was modified to "to decide between two things."

4. Discussion

This study aimed to develop an instrument for the early detection of ADHD that considers the gender perspective and is sensitive to identifying girls with ADHD. The goal was to address a recognized need observed by many professionals in their daily clinical practice. Specialized professionals in neuropsychological assessment, learning difficulties evaluation, and child and adolescent clinical practice have noted for years that girls often do not fit neatly into certain diagnostic categories, yet their functional difficulties persist without adequate diagnosis and treatment.

Numerous studies over recent decades support gender differences in ADHD [19,20,21,25,43,44,56,65,66]. However, there is ongoing controversy regarding how to detect gender differences and their clinical manifestation in boys and girls. Studies analyzing ADHD assessment based on the DSM criteria and its diagnosis report that DSM criteria analyses show symptom invariance regardless of the gender of the children assessed [53,54]. Yet, some authors suggest deeper analyses using clinical population samples to assess whether symptom invariance holds under these conditions. Other studies question the adequacy of DSM criteria for detecting ADHD in girls [58].

Therefore, in this study, we initially consulted experts from various disciplines involved in ADHD to understand how gender manifests in girls and how this symptomatology can be addressed. The first two rounds yielded broad consensus on the most relevant symptomatology to consider. Both the Inattention and Hyperactivity/Impulsivity dimensions, along with more internalized symptomatology, were grouped under the "Internalisation" dimension.

Despite existing controversies, gender differences have been analyzed and evidenced. For instance, significant interactions between gender diagnosis and anxiety symptomatology have been reported in girls [28,30,43], along with lower rates of externalizing behaviors and hyperactivity, and higher rates of internalizing symptomatology and inattention in girls [21,32]. Consistently, our study's analysis of expert opinions regarding classic ADHD dimensions indicated that inattention symptomatology carries more weight for detecting ADHD in girls than hyperactivity/impulsivity, although the latter should still be included in the scale.

Regarding internalizing symptomatology in girls, some authors like Skogli [43] and his research team recommend integrating self-reports of internalizing symptomatology with ADHD rating and detection scales, given the lack of specific instruments including this symptomatology for ADHD detection and diagnosis. Hence, the internalizing symptomatology identified and agreed upon by experts, as expressed in girls, has been categorized under the dimension termed "Internalisation".

Another crucial aspect considered in this study is the assessment of ADHD. Historically, ADHD diagnosis has relied on various informants, particularly reports from third parties (primary caregivers, teachers, etc.), which are susceptible to multiple biases [57]; even professionals themselves can be subject to bias [58]. Guided by this understanding, the design of this scale has consistently prioritized self-reporting by the child, giving children a voice regarding their symptoms and avoiding instruments solely based on third-party behavior observation. However, it is not precluded that in subsequent studies, this scale could be complemented with others incorporating input from other informants.

Numerous studies have addressed ADHD assessment and detection instruments

[59,60,61,62,63]. However, most instruments have focused on behavioral aspects reported by children or their informants rather than the children's internal symptoms. Currently, several studies support that children can be reliable informants of their difficulties, or no less valid than third parties [64,65,66,67]. Thus, due to the more internal basis of differential symptomatology between boys and girls [64,65,66,67,68], the decision was made to develop a self-report scale for boys and girls aged 6 to 12 years. Moreover, the study demonstrated significant consensus among the expert panel regarding this internalizing symptomatology.

Given the differential presentation of ADHD according to gender, early detection of this symptomatology in girls facilitates better disorder management, prevents long-term comorbidities, and reduces the risk of low self-esteem and/or suicidal behavior in girls. It also lowers healthcare costs associated with inadequate or failed treatments [67,69]. Consequently, our research team opted for an early detection instrument capable of identifying ADHD-compatible symptoms in boys and sensitive to early detection in girls.

In light of this review, the development of ADHD screening instruments that consider the gender perspective and transcend differentiated normative groups was imperative; a factor that guided this study. In the literature review of ADHD assessment instruments according to gender, no validated scales were found. However, a scale design, exclusively for girls, the ADHD Self-rating Scale for Girls (+ Teenage Girls), developed by Patricia Quinn, Kathleen Nadeau, and Ellen Littman and published in "Understanding Girls with ADHD" in 2005, has not been validated or adapted to date.

The scale, finally composed of 34 items, with 11 corresponding to the Inattention dimension, 10 to the Hyperactivity/Impulsivity dimension, and 13 to the "Internalisation" dimension, has been developed under the consensus of professionals with over 10 years of experience in the field. It adheres to standards established by the scientific literature on ADHD symptoms and gender, current diagnostic tools, and the reliability of minors' reports.

The limitations of this study are mainly centred on the fact that the scale was developed in the Spanish context, with Spanish professionals, and in a specific language, Spanish. This does not favour the generalisation of results, but as a research team, we hope that the scale can eventually be adapted to English and other languages.

5. Conclusions

Using the e-Delphi method, developed to achieve consensus in health research, and involving experts with over a decade of clinical experience, a scale for Attention Deficit Hyperactivity Disorder (ADHD) has been crafted. This scale, grounded in a robust theory of ADHD symptomatology, is tailored to accommodate the symptomatic variations between boys and girls, aiming to mitigate gender bias in diagnosis. It encompasses both traditional dimensions of the disorder and internalized symptoms, addressing challenges in emotional self-regulation. The panel of experts exhibited a high level of concurrence throughout the Delphi process rounds, thus endorsing the content of the developed scale. This concise, self-administered tool is designed for children aged 6 to 16 years and represents a significant stride towards crafting gender-neutral psychological instruments for ADHD. However, further research is warranted to assess its psychometric properties, as the scale will require an exhaustive study of its reliability and validity.