Submitted:
20 July 2023
Posted:
21 July 2023
You are already at the latest version
Abstract
The launch of Open AI's chatbot, ChatGPT, has generated a lot of attention and discussion among professionals in several fields. Many concerns and challenges have been brought up by researchers from various fields, particularly in relation to the harm that using these tools for medical diagnosis and treatment recommendations can cause. Additionally, it has been debated if ChatGPT is dependable, efficient, and helpful for clinicians and medical professionals. Therefore, in this study, we assess ChatGPT's effectiveness in providing mental health support, particularly for issues related to anxiety and depression, based on the chatbot's responses and cross-questioning. The findings indicate that there are significant inconsistencies and that ChatGPT's reliability is low in this specific domain. As a result, care must be used when using ChatGPT as a complementary mental health resource.
Keywords:
Mental Health
; ChatGPT
; Anxiety
; Depression
; LLM
Introduction
ChatGPT is a sophisticated large language model designed to interpret text-based inquiries and generate natural language responses that mimic human speech. It has been utilized in a variety of fields, including education (Tlili et al., 2023), scientific research (Farhat et al., 2023; Hill-Yardin et al., 2023), and computer programming (Surameery & Shakor, 2023). Recently, ChatGPT has been applied in the medical field, where it has demonstrated impressive results in medicine-related tasks such as disease diagnosis (Balas & Ing, 2023) and evaluation of medical knowledge (Kung et al., 2023). For instance, it has scored above 60% in medical exams such as the United States Medical Licensing Examination (Mbakwe et al., 2023), and has achieved a success rate of more than 90% in clinical diagnoses (Luca Masala et al., 2023). Moreover, ChatGPT has the potential to enhance various medical fields, such as gastroenterology and neurology, by generating high-quality research questions (Lahat et al., 2023). However, experts agree that the accuracy, reliability, and source of information provided by ChatGPT must be validated by medical professionals before it can be implemented in educational or clinical settings.
Mental health refers to an individual’s overall psychological and emotional well-being. It can be influenced by a range of factors, including genetics, environmental factors, and life experiences. A significant number of individuals who require mental health interventions do not receive the necessary care, with around 70% lacking access to these services (Wainberg et al., 2017). Common mental health conditions, such as depression, anxiety, bipolar disorder, and schizophrenia, can have a profound impact on a person’s quality of life. Positive communication with a trustworthy and unbiased listener is beneficial for individuals with mental health concerns, especially for conditions such as depression and anxiety. Given that ChatGPT is a natural language processing (NLP) model, it has the potential to provide personalized responses that resemble human interaction and could be a helpful tool in mental health resources. Therefore, this study aims to assess the potential benefits and risks of using ChatGPT as a complementary mental health resource.
Evaluating ChatGPT for mental health support
ChatGPT was queried regarding responses related to depression and anxiety. In the first query, ChatGPT’s suggested response to a person experiencing anxiety was evaluated. The second query requested information about medication related to anxiety. An inquiry about alternative medication was made in the third query, followed by cross-questioning ChatGPT’s responses in queries 4. In the fifth query, medication for depression was requested.
Discussion
Depression and anxiety are two common mental health disorders that affect millions of people worldwide (Chisholm et al., 2007). While depression is characterized by feelings of sadness, hopelessness, and loss of interest in daily activities, anxiety is characterized by excessive worry, nervousness, and fear. However, depression and anxiety are separate conditions, they often coexist and share similar symptoms. For those struggling with mental health concerns, ChatGPT can be a helpful tool, offering not only encouraging discussion but also advice on how to live a healthy lifestyle. In the event of anxiety, ChatGPT advises professional advice as well as effective symptom control strategies including exercise and stress management approaches. It does not initially offer a list of drugs for treating anxiety, but when requested, it does so. However, it emphasises the value of counselling and a change in lifestyle over drugs. Our interactions with ChatGPT revealed that the advanced NLP model not only converses like a human but also exhibits generosity and prudence when making health-related recommendations, always advising professional consultation at the end of each conversation. Several advantages of using ChatGPT have been found by this study, especially for people who are struggling with mental health issues including depression and anxiety:
Personalized Care: ChatGPT can use natural language processing to offer personalized treatment to patients based on their unique symptoms, medical history, and mental health concerns.
Access to Care: ChatGPT can be accessed remotely, which makes it a useful tool for individuals who live in rural or remote areas or who are unable to access in-person mental health care.
Cost-Effective: ChatGPT may offer a cost-effective alternative to traditional mental health treatment options, such as in-person therapy sessions.
Despite its benefits, ChatGPT is not without its limitations. Altering prompts or repeating them can alter the responses generated by ChatGPT. In our study, we observed that despite ChatGPT providing helpful advice and repeatedly urging users to seek expert consultation, certain prompts resulted in the model generating a list of prescribed medications for the subject condition. This could potentially harm society or individuals, particularly those dealing with depression or other mental health issues. Medications should only be taken under the guidance of a medical professional. In such instances, ChatGPT could function more like a conventional search engine, returning similar results for the same query. Overall, ChatGPT has the potential to be a useful tool in the field of mental health treatment, but it should not be viewed as a substitute for in-person mental health care. Rather, it should be seen as a complementary tool that can supplement existing mental health treatment options. It is important to use ChatGPT with caution and under the guidance of someone knowledgeable about its use.
Conclusion
To summarize, although ChatGPT shows promise as a diagnostic aid for identifying diseases, supporting medical professionals, and assisting with medical exams, it is not suitable for providing drug prescriptions or medication recommendations. It is crucial for individuals to be cautious when seeking health guidance from ChatGPT and verify the information they receive by consulting other sources or experts.
Conflicts of Interest
There is no conflict of interest.
References
- Balas, M.; Ing, E.B. Conversational AI Models for ophthalmic diagnosis: Comparison of ChatGPT and the Isabel Pro Differential Diagnosis Generator. JFO Open Ophthalmology 2023, 1, 100005. [Google Scholar] [CrossRef]
- Chisholm, D.; Flisher, A.; Lund, C.; Patel, V.; Saxena, S.; Thornicroft, G.; Tomlinson, M. Scale up services for mental disorders: a call for action. The Lancet 2007, 370, 1241–1252. [Google Scholar] [CrossRef]
- Farhat, F.; Sohail, S.S.; Madsen, D.Ø. How trustworthy is ChatGPT? The case of bibliometric analyses. Cogent Engineering 2023, 10, 2222988. [Google Scholar] [CrossRef]
- Hill-Yardin, E.L.; Hutchinson, M.R.; Laycock, R.; Spencer, S.J. A Chat(GPT) about the future of scientific publishing. Brain, Behavior, and Immunity 2023, 110, 152–154. [Google Scholar] [CrossRef] [PubMed]
- Kung, T.H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepaño, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; Tsengid, V. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health 2023, 2, e0000198. [Google Scholar] [CrossRef] [PubMed]
- Lahat, A.; Shachar, E.; Avidan, B.; Shatz, Z.; Glicksberg, B.S.; Klang, E. Evaluating the use of large language model in identifying top research questions in gastroenterology. Scientific Reports 2023, 13, 1–6. [Google Scholar] [CrossRef] [PubMed]
- Luca Masala, G.; De Pietro, G.; Hirosawa, T.; Harada, Y.; Yokose, M.; Sakamoto, T.; Kawamura, R.; Shimizu, T. Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study. International Journal of Environmental Research and Public Health 2023, 20, 3378. [Google Scholar] [CrossRef]
- Mbakwe, A.B.; Lourentzou, I.; Celi, L.A.; Mechanic, O.J.; Dagan, A. ChatGPT passing USMLE shines a spotlight on the flaws of medical education. PLOS Digital Health 2023, 2, e0000205. [Google Scholar] [CrossRef] [PubMed]
- Surameery NM, S.; Shakor, M.Y. Use Chat GPT to Solve Programming Bugs. International Journal of Information Technology & Computer Engineering (IJITC) 2023, 3, 17–22. [Google Scholar] [CrossRef]
- Tlili, A.; Shehata, B.; Adarkwah, M.A.; Bozkurt, A.; Hickey, D.T.; Huang, R.; Agyemang, B. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments 2023, 10. [Google Scholar] [CrossRef]
- Wainberg, M.L.; Scorza, P.; Shultz, J.M.; Helpman, L.; Mootz, J.J.; Johnson, K.A.; Neria, Y.; Bradford JM, E.; Oquendo, M.A.; Arbuckle, M.R. Challenges and Opportunities in Global Mental Health: a Research-to-Practice Perspective. Current Psychiatry Reports 2017, 19, 1–10. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
