Attitudes Toward the Use of Artificial Intelligence Chatbots for Mental Health Support: Artificial Intelligence in Mental Health Scale

1. Introduction

Artificial Intelligence (AI), in its broadest sense, refers to the field of computer science focused on creating systems capable of performing tasks that typically require human intelligence. These tasks include learning from experience, reasoning, problem-solving, understanding natural language, and perceiving and interacting with the environment [1]. As AI technologies continue to proliferate across diverse sectors, their integration into mental health services has emerged as a significant area of inquiry [2,3].

Recent research has increasingly focused on the use of AI chatbots as innovative tools for digital mental health interventions. AI chatbots are increasingly being used in mental health support as complementary tools to traditional therapy [4]. These chatbots utilize machine learning algorithms and natural language processing to interpret user input and generate contextually appropriate responses, thereby enabling interactions that closely resemble human conversation [5]. AI chatbots offer 24/7 access to mental health resources, providing instant responses without waiting for appointments. Additionally, AI chatbots are useful for people in remote or underserved areas where mental health professionals are scarce. Thus, AI chatbots offer a promising solution to several barriers commonly associated with traditional psychotherapy, including financial cost, limited availability of mental health professionals, and geographic or logistical constraints. Moreover, the anonymity and privacy afforded by chatbot platforms may reduce the stigma often linked to seeking mental health support [6,7].

Empirical studies investigating the effectiveness of AI-based interventions have yielded encouraging findings. For instance, a systematic review of AI-assisted psychotherapy for adults experiencing symptoms of depression and anxiety revealed significant improvements in depressive outcomes [8]. Specific applications such as Tess (Text-based Emotional Supporter by X2AI), which employs natural language processing to respond to emotional distress, have demonstrated reductions in both depression and anxiety symptoms [9]. Similarly, Greer et al. reported notable decreases in anxiety among users of “Vivibot”, an AI chatbot designed to provide mental health support [10]. Further evidence from Drouin et al. showed that participants who interacted with the AI chatbot “Replika” experienced lower levels of conversational anxiety compared to those who engaged with human counterparts, either face-to-face or online [11]. In another study, De Gennaro et al. found that individuals who received empathetic responses from an AI chatbot following a social exclusion task reported improved mood [12]. Additionally, usability testing and expert evaluations of an AI-based emotion chatbot indicated its particular usefulness for individuals who struggle with emotional expression in face-to-face interactions [13]. Moreover, users who regularly interacted with the “Wysa” chatbot reported significantly lower levels of depressive symptoms compared to those with minimal engagement [14]. Similarly, individuals using “Woebot” experienced notable reductions in depressive symptoms over a two-week period, whereas participants in a psychoeducational control group showed no significant change [15]. AI chatbots may be particularly well-suited for young adults, a demographic that frequently utilizes online messaging platforms with interfaces resembling those of chatbot applications [7]. Moreover, young adults have demonstrated high levels of satisfaction with digital mental health resources, suggesting that chatbot-based interventions may be both accessible and appealing to this population [16].

Although preliminary findings support the effectiveness of AI chatbots and highlight their potential utility, empirical research investigating individuals’ attitudes toward these technologies remains limited. Attitude is commonly defined as a psychological predisposition expressed through evaluative judgments—ranging from favorable to unfavorable—toward a specific entity [17]. It may also be conceptualized as an affective orientation toward or against a psychological object. Attitudes serve adaptive functions and are instrumental in shaping behavior [18]. Given that technological innovations often elicit distinct attitudinal responses, it is imperative to understand public perceptions of AI-based applications, particularly in domains where such technologies are actively being implemented [19]. Consequently, the accurate assessment of attitudes through psychometrically sound instruments is essential for predicting behavioral responses and informing the future trajectory of technological developments, such as AI-driven mental health interventions.

Until now, several scales have developed to measure attitudes towards AI as a more overarching concept, such as the Attitudes Towards Artificial Intelligence Scale (ATTARI--12) [20], the Attitudes Towards Artificial Intelligence Scale (ATAI) [21], the General Attitudes Towards Artificial Intelligence Scale (GAAIS) [22], the Concerns About Autonomous Technology questionnaire [23], the Artificial Intelligence Attitude Scale (AIAS-4) [24], and the Artificial Intelligence Attitudes Inventory (AIAI) [25]. Additionally, several other scales have produced to assess attitudes towards specific domains of AI such as the Attitudes Towards Artificial Intelligence at Work (AAAW) [26], the Treats of Artificial Intelligence Scale (TAI) [27], the Artificial Intelligence Anxiety Scale (AIAS) [28], the Short Trust in Automation Scale (S-TIAS) [29], and the Attitude Scale Towards the Use of Artificial Intelligence Technologies in Nursing (ASUAITIN) [30].

Evaluating individuals’ attitudes toward applications of AI in mental health has become increasingly relevant for both research and practice. Given the well-established relationship between attitudinal dispositions and help-seeking intentions [31,32,33], rigorous investigation of user perceptions toward chatbot-assisted psychotherapy is critical for evaluating its viability as an empirically supported mental health treatment modality. In this context, measuring attitudes toward the use of AI-based chatbots for mental health support is highly important, especially as these technologies become more integrated into clinical and therapeutic settings.

However, to the best of our knowledge, there is no scale that measures individuals’ attitudes toward the use of AI-based chatbots for mental health support. The development of a psychometrically robust, concise measurement scale to assess attitudes toward AI-enabled chatbots in mental health applications would significantly advance empirical research and applied practices in evaluating and addressing public perceptions of artificial intelligence. Therefore, the aim of our study was to develop and validate a scale to measure attitudes toward the use of AI-based chatbots for mental health support, i.e., the Artificial Intelligence in Mental Health Scale.

2. Materials and Methods

2.1. Development of the Scale

AI chatbots for mental health support present distinct technical advantages that enhance their applicability in clinical and research contexts. These systems employ advanced natural language processing techniques to interpret user input and generate contextually appropriate responses, facilitating a conversational interface that mimics human interaction. They utilize machine learning algorithms to adapt responses based on user behavior and historical interactions, improving accuracy and relevance over time. Their continuous availability ensures uninterrupted support, overcoming temporal and geographical barriers inherent in traditional mental health services. Furthermore, through machine learning algorithms, chatbots dynamically adapt to individual user patterns, improving predictive accuracy over time. Integration with data analytics frameworks allows for real-time monitoring of behavioral indicators and early detection of psychological distress, thereby supporting preventive care strategies. Finally, their cloud-based infrastructure ensures scalability and compliance with data protection regulations, while maintaining secure storage and interoperability with electronic health record systems [19,34,35,36]. Thus, we considered that technical advantages could be a factor in our scale. In this context, we created items that refer to technical advantages of the AI chatbots such as “Artificial intelligence chatbots can demonstrate better problem-solving skills compared to a human therapist” and “Artificial intelligence chatbots can offer up to date information on mental health issues”.

Additionally, AI chatbots for mental health support offer significant personal benefits that enhance accessibility and user engagement. They provide immediate, on-demand support, reducing barriers such as scheduling constraints and geographic limitations commonly associated with traditional therapy. Their anonymity and privacy features foster a safe environment for individuals who may experience stigma or discomfort in seeking face-to-face care, thereby encouraging disclosure and emotional expression. Moreover, chatbots deliver personalized interventions by analyzing user input and tailoring responses to individual needs, which promotes a sense of relevance and trust. The non-judgmental nature of AI interactions further alleviates anxiety associated with human evaluation, making them particularly valuable for individuals with social apprehension [37,38,39,40,41]. Thus, we considered that personal advantages could be a factor in our scale. In this context, we created items that refer to personal advantages of the AI chatbots such as “Artificial intelligence chatbots can expand access to mental health care by reducing geographic barriers” and “Artificial intelligence chatbots can expand access to mental health care by reducing financial barriers”.

After all, we expected a two-factor model for our scale (i.e., technical and personal advantages).

Figure 1 illustrates the process undertaken to develop and validate the Artificial Intelligence in Mental Health Scale (AIMHS). We followed the suggested guidelines [42,43,44] to develop and validate the AIMHS. Initially, a comprehensive literature review was conducted to identify existing instruments, scales and tools that assess attitudes towards AI [e.g., 20–22,24,26,28,29,45,46]. The relevance and applicability of items from these instruments were critically evaluated to inform the development of the AIMHS. Moreover, we examined instruments, scales and tools that measure mental health issues [e.g., 47–54]. After all, an initial pool of 24 items was generated to capture attitudes towards the use of AI chatbots for mental health support. The initial item pool was structured around the two factors (i.e., technical and personal advantages) that we explain in detail above.

Subsequently, content validity was assessed through consultation with a panel of ten experts from diverse professional backgrounds, including AI experts, psychiatrists, psychologists, therapists, and digital health experts. We used the expert panel to assess the content validity of the initial set of 24 items. Each item was rated as “not essential,” “useful but not essential,” or “essential.” The Content Validity Ratio (CVR) was calculated using the following formula, where N represents the total number of experts and n denotes the number who rated an item as “essential.”

$\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{v}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{y} \mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}={\displaystyle \frac{\mathrm{n}-\frac{\mathrm{N}}{2}}{\frac{\mathrm{N}}{2}}}$

Following suggested guidelines [55], items with a CVR below 0.80 were excluded, resulting in the retention of 14 items.

To evaluate face validity, cognitive interviews [56] were conducted with 15 participants, all of whom demonstrated clear comprehension of the items. Further validation was performed using the item-level face validity index, based on ratings from 15 participants using a four-point Likert scale (1 = not clear to 4 = highly clear). In accordance with suggested guidelines [57], items with an item-level face validity index above 0.80 were retained. All 14 items met this criterion, with item-level face validity index ranging from 0.933 to 1.000.

After the scale’s initial development phase, it comprised 14 items designed to assess attitudes towards the use of AI chatbots for mental health support. Responses were recorded on a five-point Likert scale as follows: 1 (strongly disagree), 2 (disagree), 3 (neither disagree nor agree), 4 (agree), 5 (strongly agree). Supplementary Table S1 shows the 14 items that were produced after the assessment of the content and face validity of the AIMHS. To reduce acquiescence bias, we employed positive and negative items towards AI chatbots use. In particular, we used seven positive items and seven negative items. Examples of positive items were the following: (1) Artificial intelligence chatbots can demonstrate sufficient social and cultural sensitivity to support diverse populations, (2) Artificial intelligence chatbots can demonstrate better problem-solving skills compared to a human therapist, and (3) Artificial intelligence chatbots can expand access to mental health care by reducing geographic barriers. Examples of negative items were the following: (1) Artificial intelligence chatbots cannot understand people’s emotions, (2) Artificial intelligence chatbots cannot provide correct information on mental health issues, and (3) Artificial intelligence chatbots cannot achieve empathy levels comparable to those of a human therapist.

2.2. Participants and Procedure

The study sample consisted of adults aged 18 years or older. Inclusion criteria required participants to engage with AI chatbots, social media platforms, or websites for a minimum of 30 minutes per day, thereby ensuring a basic level of interaction with digital technologies. Data collection was conducted through an anonymous online survey, developed via Google Forms and disseminated across multiple digital platforms, including Facebook, Instagram, and LinkedIn. To enhance recruitment, a promotional video was additionally produced and shared on TikTok. Data collection took place in October 2025.

In total, 428 individuals completed the questionnaire. Of these, 77.6% (n = 332) were female and 22.4% (n = 96) were male. The mean age of participants was 41.1 years (standard deviation; 14.1), with a median of 45 years (range: 18–65). The reported mean daily use of AI chatbots, social media platforms, and websites was 3.5 hours (SD = 2.6), with a median of 3 hours (range: 30 minutes–14 hours).

2.3. Item Analysis

After the scale’s initial development phase, an item analysis was carried out on the 14 generated items. This analysis, based on the full sample (n=428), examined inter-item correlations, corrected item-total correlations, floor and ceiling effects, skewness, kurtosis, and Cronbach’s alpha (calculated when each individual item was removed) for the 14 items [58]. Inter-item correlations were compared against the recommended range of 0.15 to 0.75 [59], while corrected item-total correlations needed to exceed 0.30 to confirm sufficient discriminatory power [60].

Each item was rated on a five-point Likert scale with anchors: strongly disagree, disagree, neither disagree nor agree, agree, strongly agree. Floor and ceiling effects were identified if over 85% of respondents chose either the lowest (“strongly disagree”) or highest (“strongly agree”) option [61]. Distribution normality was evaluated using skewness (acceptable between -2 and +2) and kurtosis (acceptable between -3 and +3) [62].

An introductory note was provided to participants to describe and clarify the role of AI chatbots in mental health support. Supplementary Table S1 presents this introductory note. We asked participants to assess their attitudes towards AI chatbots for mental health support by answering the AIMHS.

2.4. Construct Validity

To evaluate the construct validity of the AIMHS, we applied both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). According to the literature, EFA requires a minimum of either 50 participants or five participants per item [63], while CFA typically requires at least 200 participants [58]. Our sample size met these criteria. For stronger psychometric validation, participants were randomly split into two groups: one group of 214 participants for the EFA and another 214 for the CFA. Using independent samples for each factor analysis (i.e., EFA and CFA) strengthened the reliability and validity of our results, with both subsamples surpassing the recommended thresholds.

We began with an EFA to identify the underlying factor structure of the AIMHS, followed by a CFA to test the factor model suggested by the EFA. This stage included the 14 items that remained after the scale’s initial development and item analysis. Before conducting the EFA, data adequacy was confirmed using the Kaiser-Meyer-Olkin (KMO) index and Bartlett’s test of sphericity. Standard acceptability thresholds were applied: KMO values above 0.70 and a statistically significant Bartlett’s test (p < 0.05) [62,64]. For the EFA, we used oblique rotation (direct oblimin method in SPSS) since this method allow the factors to correlate as we expected some correlation in our study [58]. Indeed, there was a statistically significant correlation (r = 0.5, p-value < 0.001) between the two factors that emerged through validity and reliability analysis. Since the data followed the normal distribution, the maximum likelihood estimator was used [58]. We applied the following acceptability criteria: (a) eigenvalues greater than 1, (b) factor loadings above 0.60, (c) communalities exceeding 0.40, and (d) total explained variance over 65% [62]. Reliability was further tested using Cronbach’s alpha, with coefficients above 0.70 considered satisfactory [65].

Subsequently, a CFA was conducted to confirm the AIMHS factor structure. Model fit was assessed through several indices: absolute fit measures (Root Mean Square Error of Approximation [RMSEA] and Goodness of Fit Index [GFI]), relative fit measures (Normed Fit Index [NFI] and Comparative Fit Index [CFI]), and one parsimonious fit measure (ratio of chi-square to degrees of freedom [x²/df]). Acceptable thresholds were RMSEA < 0.10, GFI > 0.90, NFI > 0.90, CFI > 0.90, and x²/df < 5 [66,67]. Additionally, standardized regression weights between items and factors were calculated.

Moreover, we assessed measurement invariance across demographic subgroups, specifically examining differences by gender, age, and daily use of AI chatbots, social media platforms, and websites. Participants were divided into two groups for age and usage time based on median values. Configural invariance was tested first, followed by metric invariance. For configural invariance, we evaluated RMSEA, GFI, NFI, and CFI, with acceptable thresholds defined as RMSEA < 0.10 and GFI, NFI, and CFI > 0.90. For metric invariance, a p-value greater than 0.05 indicated invariance [68].

2.5. Concurrent Validity

Following the construct validity assessment, a two-factor model with 5 items was established for the AIMHS. Each item was rated on a five-point Likert scale ranging from “strongly disagree” to “strongly agree”. Scores were assigned so that higher values reflected more positive attitudes towards AI chatbots. Thus, scores for positive items were the following: 1 for “strongly disagree”, 2 for “disagree”, 3 for “neither disagree nor agree”, 4 for “agree”, and 5 for “strongly agree”. On the other hand, scores for the negative items were the following: 5 for “strongly disagree”, 4 for “disagree”, 3 for “neither disagree nor agree”, 2 for “agree”, and 1 for “strongly agree”. To obtain the total AIMHS score, we sum the answers in all items and divide the aggregate by total number of answers. Thus, the total AIMHS score ranges from 1 to 5. Similarly, we calculate total score for each factor, and, thus, this score also ranges from 1 to 5. Higher scores indicate more positive attitudes towards AI mental health chatbots.

There are no other scales that measure attitudes towards the use of AI chatbots for mental health support. Thus, we used three other scales to measure concurrent validity of the AIMHS. In particular, we used the Artificial Intelligence Attitude Scale (AIAS-4) [24], the Attitudes Towards Artificial Intelligence Scale (ATAI) [21], and the Short Trust in Automation Scale (S-TIAS) [29].

We used the AIAS-4 [24] to measure general attitude toward AI. The AIAS-4 includes four items such as “I believe that artificial intelligence will improve my life” and “I think artificial intelligence technology is positive for humanity”. Answers are on a 10-point Likert scale from 1 (not at all) to 10 (completely agree). Total score is the average of all the item scores and ranges from 1 to 10. Higher scores indicate more positive attitudes towards AI. We used the valid Greek version of the AIAS-4 [69]. In our study, Cronbach’s alpha for the AIAS-4 was 0.917. We expected a positive correlation between the AIAS-4 and the AIMHS.

We used the ATAI [21] to measure acceptance and fear of AI. The ATAI includes five items; two items measure acceptance of AI, and three items measure fear of AI. The two items for the acceptance of AI are the following: “I trust artificial intelligence” and “Artificial intelligence will benefit humankind”. The three items for the fear of AI are the following: “I fear artificial intelligence”, “Artificial intelligence will destroy humankind” and “Artificial intelligence will cause many job losses”. Answers are on an 11-point Likert scale from 0 (strongly disagree) to 10 (strongly agree). Total score is the average of all the item scores and ranges from 0 to 10. Higher scores in the acceptance subscale indicate higher levels of acceptance of AI, and, thus, more positive attitudes towards AI. Higher scores in the fear subscale indicate higher levels of fear of AI, and, thus, more negative attitudes towards AI. We used the valid Greek version of the ATAI [70]. In our study, Cronbach’s alpha for the acceptance subscale was 0.738, and for the fear subscale was 0.721. We expected a positive correlation between the acceptance subscale and the AIMHS. Also, we expected a negative correlation between the fear subscale and the AIMHS.

We used the S-TIAS [29] to measure trust in AI. The S-TIAS includes three items; “I am confident in the artificial intelligence assistant”, “The artificial intelligence assistant is reliable”, and “I can trust the AI assistant”. Answers are on a 7-point Likert scale from 1 (not at all) to 7 (extremely). Total score is the average of all the item scores and ranges from 1 to 7. Higher scores indicate higher levels of trust in AI, and, thus, more positive attitudes towards AI. We used the valid Greek version of the S-TIAS [71]. In our study, Cronbach’s alpha for the S-TIAS was 0.912. We expected a positive correlation between the S-TIAS and the AIMHS.

2.6. Reliability

To assess the internal consistency of the AIMHS, we computed Cronbach’s alpha using the full sample (n = 428). Values above 0.70 for Cronbach’s alpha were regarded as acceptable [65]. We also examined corrected item-total correlations and Cronbach’s alpha if an item was deleted for the final five-item version of the AIMHS, with correlations above 0.30 considered satisfactory [60].

In addition, a test-retest reliability study was carried out with 50 participants who completed the AIMHS twice within a five-day interval. For the five items, Cohen’s kappa was calculated due to the five-point ordinal response format, while a two-way mixed intraclass correlation coefficient (absolute agreement) was used to evaluate the total AIMHS score.

2.7. Ethical Issues

The study utilized an anonymous and voluntary data collection process. Before taking part, participants were fully informed about the research aims and procedures and subsequently provided informed consent. Ethical approval was granted by the Institutional Review Board of the Faculty of Nursing, National and Kapodistrian University of Athens (Protocol Approval #01, September 14, 2025). All procedures adhered to the ethical standards of the Declaration of Helsinki [72].

2.8. Statistical Analysis

Categorical variables are described using absolute frequencies (n) and percentages, while continuous variables are summarized with measures of central tendency (mean, median) and dispersion (standard deviation, range), along with minimum and maximum values. Normality of scale distributions was evaluated using both statistical tests (Kolmogorov-Smirnov) and graphical methods (Q-Q plots). As all scales showed normal distribution, correlations between them were assessed with Pearson’s correlation coefficients. P-values less than 0.05 were considered as statistically significant. We performed CFA with AMOS version 21 (Amos Development Corporation, 2018). All other analyses were conducted with IBM SPSS 28.0 (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 28.0. Armonk, NY: IBM Corp.).

3. Results

3.1. Item Analysis

Item analysis included the 14 items developed during the initial phase of the AIMHS. All items demonstrated acceptable psychometric properties, including corrected item–total correlations, inter-item correlations, floor and ceiling effects, skewness, and kurtosis (Table 1). The internal consistency of the scale was high, with a Cronbach’s alpha of 0.888, which decreased upon removal of any single item. Supplementary Table S2 provides the inter-item correlation matrix for the 14 items.

3.2. Exploratory Factor Analysis

The Kaiser-Meyer-Olkin index was 0.861, and Bartlett’s test of sphericity was significant (p < 0.001), confirming sample adequacy for EFA. An oblique rotation (direct oblimin in SPSS) was applied to the 14 items (Table 1). Results of the initial EFA are presented in Table 2. Seven items (#1, #2, #5, #10, #11, #13, and #14) exhibited inadequate factor loadings (< 0.60) and/or communalities (< 0.40). This analysis yielded three factors explaining 59.758% of the variance. A second EFA was conducted after removing these items. The KMO index was 0.766, and Bartlett’s test remained significant (p < 0.001). Table 3 summarizes these results, where two items (#3 and #12) showed insufficient loadings or/and communalities. The second EFA identified two factors accounting for 66.199% of the variance. A third EFA was then performed after excluding items #3 and #12. The KMO index was 0.708, and Bartlett’s test was significant (p < 0.001). In this third model, all remaining items (#4, #6, #7, #8, and #9) had factor loadings above 0.649 and communalities above 0.415. The EFA identified two factors explaining 81.280% of the variance, making the third model (Table 4) the optimal structure derived from EFA.

After all, the EFA identified a two-factor five-item model for the AIMHS. These factors were named “technical advantages” and “personal advantages” after consideration of the items that included. In particular, the factor “technical advantages” included the items “Artificial intelligence chatbots cannot achieve empathy levels comparable to those of a human therapist” and “Artificial intelligence chatbots can demonstrate better problem-solving skills compared to a human therapist”. Also, the factor “personal advantages” included the items “Artificial intelligence chatbots can expand access to mental health care by reducing geographic barriers”, “Artificial intelligence chatbots can expand access to mental health care by providing continuous access (24/7 availability)”, and “Artificial intelligence chatbots can expand access to mental health care by reducing financial barriers”.

Cronbach’s alpha for the AIMHS (third model, Table 4) was 0.798, while for the factor “technical advantages” was 0.728, and for the factor “personal advantages” was 0.881.

3.3. Confirmatory Factor Analysis

Confirmatory factor analysis was conducted to validate the factor structure of the AIMHS identified through EFA (Figure 2). The goodness-of-fit indices indicated that the two-factor model comprising five items demonstrated an excellent fit to the data since x2/df was 1.485, RMSEA was 0.048 (90% confidence interval = 0.0001 to 0.122), GFI was 0.989, NFI was 0.987 and CFI was 0.996. Additionally, standardized regression weights between items and factors ranged from 0.63 to 0.92 (p < 0.001 in all cases). We found a statistically significant correlation between the two factors (0.5, p-value < 0.001).

In conclusion, EFA and CFA identified a two-factor five-item model for the AIMHS. As we described above in detail, the two factors were named “technical advantages” and “personal advantages”.

3.4. Measurement Invariance

Table 5 presents the results of the configural measurement invariance analysis for the two-factor five-item CFA model across gender, age, and daily use of AI chatbots, social media platforms, and websites. The RMSEA, GFI, NFI, and CFI indices for these demographic groups indicated that the two-factor CFA model demonstrated very good fit for all comparisons, including females versus males, younger versus older participants, and those with lower versus higher daily use of AI chatbots, social media platforms, and websites. Furthermore, metric invariance was supported, as the p-values for gender (0.597), age (0.204), and daily use of AI chatbots, social media platforms, and websites (0.261) exceeded the 0.05 threshold.

3.5. Concurrent Validity

We found a positive correlation between the AIAS-4 and the AIMHS (r = 0.405, p-value < 0.001), suggesting that participants with more positive attitudes towards AI have more positive attitudes towards AI mental health chatbots.

Moreover, we found a positive correlation between the acceptance subscale of the ATAI and the AIMHS (r = 0.401, p-value < 0.001), suggesting that participants with higher levels of acceptance of AI have more positive attitudes towards AI mental health chatbots. On the other hand, we found a negative correlation between the fear subscale of the ATAI and the AIMHS (r = -0.151, p-value = 0.002), suggesting that participants with higher levels of fear of AI have more negative attitudes towards AI mental health chatbots.

Additionally, we found a positive correlation between the S-TIAS and the AIMHS (r = 0.450, p-value < 0.001), suggesting that participants with higher levels of trust in AI have more positive attitudes towards AI mental health chatbots.

Therefore, the concurrent validity of the AIMHS was excellent. Table 6 shows the correlations between the AIMHS, and the AIAS-4, the ATAI and the S-TIAS.

3.6. Reliability

Cronbach’s alpha for the AIMHS was 0.798, for the factor “technical advantages” was 0.728, and for the factor “personal advantages” was 0.881.

Corrected item-total correlations for the five items ranged between 0.451 and 0.734, while removal of each single item did not increase Cronbach’s alpha for the AIMHS (Supplementary Table S4).

Cohen’s kappa for the five items ranged from 0.760 to 0.848 (p < 0.001 in all cases), (Supplementary Table S5).

Intraclass correlation coefficient for the AIMHS was 0.938 (95% confidence interval; 0.866 to 0.968, p < 0.001).

Afterall, the reliability of the AIMHS was excellent.

4. Discussion

According to the World Health Organization (WHO), the global prevalence of mental disorders continues to rise, posing a significant public health challenge. Recent estimates indicate that approximately one in five individuals worldwide—equivalent to nearly 1.7 billion people—are currently living with a mental disorder, with depressive and anxiety disorders being the most common [73]. Depression affects about 5% of adults globally and remains the leading cause of disability, while anxiety disorders impact roughly 4% of adults [74]. The burden of these conditions is particularly pronounced in low- and middle-income countries, where access to mental health care remains limited and comorbidity between depression and anxiety disorders is frequent [73]. These data underscore the urgent need for scalable, evidence-based interventions and systemic reforms to strengthen mental health services worldwide.

In this context, AI chatbots seem to be valuable as a first line of support and self-help tool [8,9,10,11,12,13]. Usage of mental health chatbots and virtual therapists has increased by 320% between 2020 and 2022. By 2023, around 84% of mental health professionals had either adopted or considered integrating AI tools into their work. Looking ahead to 2032, it’s expected that nearly all—99%—will be using AI technologies as part of their everyday practice [75]. This surge reflects a broader trend toward digital mental health solutions, driven by accessibility, affordability, and the demand for immediate support. Market projections indicate continued expansion, with the mental health chatbot sector expected to grow from $380 million in 2025 to $1.65 billion by 2033, representing a compound annual growth rate of approximately 20% [76].

Since AI-based chatbots are increasingly being used in mental health support as complementary tools to traditional therapy there is an urgent need to create valid tools to measure attitudes toward the use of AI chatbots for mental health support. To the best of our knowledge, there is no scale that measures individuals’ attitudes toward the use of AI chatbots for mental health support. Therefore, we developed and validated the Artificial Intelligence in Mental Health Scale to measure individuals’ attitudes toward the use of AI-based chatbots for mental health support. Our comprehensive assessment of validity and reliability indicates that the AIMHS demonstrates strong psychometric soundness as a measure of attitudes toward the use of AI-based chatbots for mental health support. Comprising five items and requiring only a few minutes to administer, the AIMHS offers a concise and user-friendly option while maintaining robust measurement properties.

We adhered to the recommended guidelines [42,43,44] for the development and validation of the AIMHS. Following an extensive literature review on instruments, scales and tools that assess attitudes towards AI, and mental health issues, we initially generated 24 items intended to measure attitudes towards AI mental health chatbots. The content validity of these items was evaluated using the content validity ratio, resulting in the removal of 10 items and leaving 14 items for further analysis. Face validity was assessed through cognitive interviews and the calculation of the item-level face validity index, after which all 14 items advanced to the next stage of evaluation. Subsequently, item analysis was conducted, leading to adequate indices for the 14 items. Consequently, 14 items were retained for construct validity testing. Exploratory factor analysis revealed that nine items exhibited unacceptable factor loadings and communalities, prompting their removal. The remaining five items demonstrated satisfactory values in the EFA, which identified two factors. As we expected, factor analysis revealed a two-factor model for AIMHS; technical advantages and personal advantages. Confirmatory factor analysis further supported the two-factor five-item structure identified by the EFA. To further assess validity, three validated scales were employed to examine concurrent. In particular, we used the AIAS-4, the ATAI, and the S-TIAS. Statistically significant correlations were observed between the AIMHS and these scales, indicating strong construct validity.

4.1. Limitations

Our study had several limitations. First, this study was conducted within a single country using a convenience sampling method. For instance, the proportion of male participants was notably lower than that of the general population. Consequently, the results may not be widely generalizable. Future research should aim to include more representative and diverse samples to further validate the AIMHS. Despite these limitations, the psychometric evaluation remains strong, as the sample size met all required standards. Moreover, our findings supported configural measurement invariance and metric invariance considering several demographic variables. However, it is important for future studies to explore the psychometric properties of the AIMHS across various populations and cultural contexts. Second, we employed a cross-sectional design to evaluate the validity of the AIMHS. However, since individuals’ attitudes may shift over time, longitudinal studies are needed to better understand individuals’ attitudes toward the use of AI chatbots for mental health support. Third, while we conducted a comprehensive psychometric analysis of the AIMHS, future researchers may consider additional methods—such as assessing divergent, criterion, and known-groups validity—to further examine its reliability and validity. Fourth, as this was the initial validation of the AIMHS, we did not attempt to establish a cut-off score. Future investigations could explore threshold values to help differentiate between participant groups. Fifth, self-report instruments are inherently susceptible to social desirability bias in studies like this one. However, since our study was conducted anonymously and participation was entirely voluntary, we believe that the likelihood of such bias influencing responses was minimal. Sixth, we used three scales to examine the concurrent validity of the AIMHS: (1) Artificial Intelligence Attitude Scale, (2) Attitudes Towards Artificial Intelligence Scale, and (3) Short Trust in Automation Scale. The concurrent validity of the AIMHS can be further explored by using several other scales such as the Attitudes Towards Artificial Intelligence Scale [20], the General Attitudes Towards Artificial Intelligence Scale (GAAIS) [22], and the Artificial Intelligence Attitudes Inventory (AIAI) [25]. Finally, the AIMHS as well as the other scales we used in our study were self-report instruments, and, thus, an information bias is probable due to some participant subjectivity.

5. Conclusions

Given the rapid integration of AI into mental health care, assessing attitudes toward AI-based chatbots is essential for understanding their acceptance and potential impact. User perceptions regarding trust, privacy, perceived usefulness, and emotional safety significantly influence adoption and sustained engagement. If individuals perceive chatbots as helpful, trustworthy, and easy to use, they are more likely to integrate them into their mental health routines.

However, currently, there is no widely validated scale specifically for measurement of attitudes toward AI mental health chatbots. To address this gap, we developed and validated the Artificial Intelligence in Mental Health Scale. Measuring attitudes toward these technologies can inform the design of user-centered interventions, guide policy development, and ensure that digital mental health solutions are implemented in a manner that is both effective and ethically responsible. Even the most advanced chatbot will fail to deliver benefits if users are skeptical or resistant. Measuring attitudes helps identify barriers such as privacy concerns, lack of empathy perception, or fear of misdiagnosis. Understanding public attitudes informs regulatory frameworks, ethical guidelines, and implementation strategies for digital mental health interventions. Developers can use attitude data to improve user experience, cultural sensitivity, and trust-building features in AI systems.

After thoroughly evaluating its reliability and validity, we determined that the AIMHS is a concise and accessible instrument with strong psychometric qualities. The results indicate that AIMHS functions as a two-factor scale comprising five items, designed to gauge individuals’ perceptions of AI-based mental health chatbots. Considering the study’s limitations, we suggest translating and validating the AIMHS across various languages and demographic groups to further examine its consistency and accuracy. Overall, the AIMHS shows potential as a valuable resource for assessing attitudes toward AI mental health chatbots and could assist policymakers, educators, healthcare providers, and other stakeholders in enhancing mental health services. For example, both researchers and practitioners can use the AIMHS to evaluate the effectiveness of mental health interventions by measuring attitudes before and after chatbot interaction to assess changes. Furthermore, researchers in all disciplines may now use a valid tool such as AIMHS to explore associations between attitudes and variables like usage intention, mental health outcomes, or demographics. For instance, the AIMHS may be used to predict intention to use AI chatbots for mental health support, examine factors influencing acceptance, and evaluate interventions aimed at improving acceptance.

Assessing individuals’ acceptance of AI-based mental health chatbots is critical for understanding their potential integration into mental healthcare services. Acceptance influences both initial adoption and sustained engagement, which are essential for achieving therapeutic benefits. Measuring acceptance through validated scales enables researchers to identify key determinants such as perceived usefulness, ease of use, trust, and privacy concerns. These insights can inform the design of user-centered interventions, guide ethical implementation, and support policy development aimed at promoting equitable access to digital mental health solutions.