Machine Learning Analysis of Social Media Posts Written in Natural Language May Enhance Suicidal Ideation Detection in Romanian Adults with Major Depression

1. Introduction

Depression, the most commonly diagnosed mental disease [1], is a significant public health concern [2] as it affects over 300 million individuals on a daily basis, worldwide [3]. The complex and varied symptomatology, along with the challenges associated with treatment and the high rate of recurrence result in a life-long health burden for patients [4,5] and a clinical challenge for healthcare providers [6]. Patients grappling with major depressive disorder (MDD) often highlight social uncertainty and feelings of helplessness as prominent factors influencing their struggle with their mental health problems [7]. The presence of social stigma and marginalization [8], negative, hate comments or franc emotional bullying [9] further exacerbates their self-esteem problems, potentially triggering or intensifying suicidal thoughts [10]. Due to all negative social repercussions, MDD subjects tend to conceal suicidal ideation (SI) when they interact with others [11].

Suicide is one of the major leading causes of death, worldwide [12]. More than 1% of all world reported deaths are a consequence of a suicidal act [13]. The suicidal act is a complex socio-medical phenomenon where various biological, psychological, social and cultural factors may play different roles [14]. Suicide is described as a continuum that starts with suicide thoughts (active or passive) and ends with suicidal behaviors (from attempts to completed suicide). This concept is not unanimously accepted mainly because far less than one third of subjects with suicide ideation (SI) will actually attempt suicide [15] and only 3-4% of them will complete suicide [16]. The epidemiology of the suicidal act is multifaceted [17], the highest rate of completed suicide being observed in male subjects over 55 years old (the age and gender paradox – [18] and with a majority of nonfatal self- injuries seen in 15-20 years old females [19]. Early detection of suicidal behavior can lead to suicide prevention [20] but screening for SI is a serious challenge. Suicide is a subject that is strongly influenced by widespread socio-cultural and religious ideas [21] and subjects planning suicide will often adopt compensatory behaviors [22]. There is no suicide risk assessment tool or procedure that is able to correctly identify suicide behavior [23]. In 1983 Pokorny demonstrated in a large prospective study, that predicting a suicidal act is rather a matter of chance than a scientific deduction [24]. Hawgood and coworkers [25] considered that a change in the SI detection paradigm (universal screening) may lead to a better prevention of suicide but results are unreliable as both false positive and negative cases were high [26].

Psychiatric patients have an increased lifetime risk for committing suicide [27]. SI was often described in subjects with various forms of mental disorders [28] and a strong association between SI and MDD was constantly reported [29]. More than two-thirds of subjects that completed suicides were diagnosed with a form of mood disorder [30] Detecting suicidal thoughts in psychiatric patients is a common but difficult task for the treating physician [31] and often physicians will not carefully test for SI as the process is time and resource consuming [32].

The abundance of health data available online provides great access to information pertinent to the management of many health conditions [33]. Online support groups and discussion forums provide a modern avenue for chronic patients to share their personal experience, to exchange treatment information and seek peer support [34]. For individuals with MDD, online support forums create a space with reduced social pressure, allowing them to connect with others who understand their struggles [35]. Although large written texts, such as forum posts, may not be as popular as they were a decade ago, they remain a significant means for MDD individuals to connect with their peers [36]. The anonymity provided in these forums [37] often encourages contributors to express their genuine thoughts and opinions, providing valuable insights into their ideation [38]. Conducting punctual analyses of these texts using computational linguistics techniques, such as text mining, can offer a deeper understanding of the emotional state of the individuals involved [39]. Anonymity also plays a crucial role in driving individuals with SI to seek peer interaction and support on social media platforms [40,41,42]. Multiple published reviews have explored the state of suicide prediction in social media, particularly in the context of various mental illnesses [43,44]. The consensus from these studies is that while predicting SI is possible, it is heavily influenced by factors such as social media access, language, culture, and education. Age often determines the choice of social media platforms [45], with newer platforms like Twitter, Reddit, and TikTok proving effective for predicting suicidal ideation in young, non-MDD individuals [46,47]. However, older individuals with chronic conditions often prefer specialized social channels like discussion forums that allow extensive text contributions and are not driven by commercial interests (controlled access). For non-native English speakers, posting short texts in English is non-natural [48]. Therefore, detecting SI in MDD individuals from diverse non-English speaking cultures [49] based on their social media contributions has become an increasingly important area of research [50,51].

The development of natural language processing algorithms had revolutionized computerized text analysis [52] providing valuable support for medical research. Text mining can adopt various tactics (terms tracking, automatic summarization, information retrieval or text categorization) in response of different needs. Algorithms used in text categorization (often named “classifiers”) can categorize large texts written in unstructured, natural languages based on distinct linguistic patterns which are associated with various sentiments or emotions. A classifier must be first trained using clear, predefined rules [53] which are considered the “ground truth”. After training, algorithms are capable to detect similar patterns in unstructured texts, and predict association with a predefined class of sentiments. The whole process of text categorization is performed under “supervised learning” technique. In contrast, unsupervised learning methods allow algorithms to autonomously determine classification rules (using clustering algorithms) often leading to interesting results. However, the outcomes of unsupervised learning may vary when applied to texts written in non-English languages.

Supervised training text categorization were validated and have various applications (from email spam filtering to hate speech recognition). It allows a good SI prediction in social media posts [54]. There are several formal preliminary steps that have to be followed before text analysis (text preprocessing, feature extraction and feature selection). All these steps are grouped into the “training phase” of the algorithm [55]. Preprocessing consists in removing of words that are not important for information retrieval (like articles and prepositions – known as stop-words). Removing stop-words will reduce the volume of text from which the real information has to be extracted. Feature extraction and selection are important stages of training, and are associated with “the bag of words” method. After preprocessing, key words from text are organized as vectors (vector space model – Salton 1975) and finally the text will be classified. Several machine learning platforms (Knime, Orange, RapidMiner, Weka) are offering similar text mining algorithms. Decision to use one platform or another is more a personal preference. Weka (Waikato Environment for Knowledge Analysis [56]) is a collection of machine learning algorithms that has been designed to transform and preprocess raw text data and, based on attributes selection, to classify or cluster unstructured text. It also allows a substantial statistical evaluation of the results. It has a user-friendly graphic interface and may work well even with minimal programming.

In Romania, a South Eastern European country [57], 86% of population is practicing the Christian Orthodox religion under the lead of the authoritarian, autocephalous, Romanian Orthodox Christian Church. The Church is firmly condemning suicide considering it the most unpardonable capital sin. This idea is widely embraced by the older, religious population of the country. The strong socio-cultural opposition to suicide is often associated with a generalized stigmata concerning any forms of mental disorder. The overall percentage of completed suicide acts is lower than the European average and remained constant over the last decade [58]. Little is known about SI in Romanian MDD patients and the subject is difficult to investigate, as most of the subjects decide to avoid it in open medical discussions.

Our research interest is to improve SI detection in Romanian psychiatric patients. As classical medical screening is difficult, we evaluated how natural language posts sent to a Romanian social media forum can be used to early detect SI in MDD patients.

2. Materials and Methods

We used WEKA [59] text mining algorithms to detect SI in Romanian forum texts written by subjects having a clear MDD diagnostic. The “training dataset” included 15 posts that were extracted from a dedicated suicide prevention forum (https://www.romedic.ro/forum/ despre/sinuciderea). Inclusion criteria were suicidal history and ideation (present ideation, previous suicidal tentative, emergency hospitalization after suicidal attempt, family history of suicide) and a mood disorder diagnostic (most of the times, a MDD suggested by diagnostic and or treatment). Authors were carefully balanced (age and sex) with 15 subjects that posted notes which did not include clear suicide ideation, sampled from a depression support forum (inclusion criteria – MDD in medical history and/or prescribed medication). The initial word corpus of the training set contained 4651 words (30 instances with 2 attributes – High Probability=HP and Low Probability=LP of SI). No socio-demographic information was used for training purposes.

In the “testing dataset” we included 125 unstructured, naturally written texts which were selected from a total of 657 posts sent over a 5 years period (2014-2018) to a dedicated Romanian depression support forum (https://www.romedic.ro/forum/despre/depresia). Inclusion criterium was a clear MDD that was established based on mentioned medical history and/or prescribed medication. Main text exclusion criteria were texts which suggest therapies or drugs positioning or the absence of minimal socio-demographic information (sex and age). All posts were done by subjects with masked identity (pen name) but socio-demographic characteristics (age and sex) were available in all of the selected cases. All subjects included in this dataset were initially considered with a low probability of SI. The word corpus of the test dataset comprised 26872 words.

The text classification tactic was “supervised learning”. The training dataset was considered ground truth. The bag of words was trimmed using a Romanian stop-word dictionary (312 words – available @ https://github.com/stopwords-iso/stopwords-ro). Lemmatization and stemming were performed manually as existing dictionaries generated linguistic errors. After preprocessing, the training bag of words was reduced to 1073 words. Finally, we used a decision tree classifier (J48, the WEKA algorithm derived from C4.5) for the text classification. A “decision tree” is a non-parametric machine learning method able to predict the value of a target variable based on simple decision rules learned from the data features. It is easy to understand from a medical standpoint and can be graphically visualized [60]. Only cases with 100% prediction confidence were selected for the final analysis. Selected cases were analyzed from socio-demographic, peers’ interest (number of comments received), professional advice and support and texts statistics (number of words per text excerpt) perspectives.

Statistics. Results were analyzed using the PSPP software (GNU project, 2015). Only basic descriptive statistic (mean and standard deviation) is presented here. Scores were analyzed with Student’s T-test. P value was set at p=0.05.

Ethics. Both depression and suicide support forums are open to public. Both are totally anonymized and the de-identification process is performed by data curators. Our study used only non-identifiable data following the Canadian ethics research provisions for secondary data use studies (TCPs 2(2018)). The data investigation research was performed in accordance with declaration of Helsinki principles.

3. Results

The training dataset (30 subjects – 12 males and 18 females) had a mean age of 34.56±7.8 years. The mean age of the testing dataset (125 subjects – 51 males and 74 females) was 30.8±10.6 years. Detailed socio-demographic data are presented in Table 1. After preprocessing, feature selection and extraction from the training dataset, the algorithm generated a decision tree with 11 nodes and 6 leaves (Figure 1).

32 subjects (25%) from the testing dataset were classified as having a high probability of SI. 91 subjects remained with a low probability of SI. 2 cases were rejected from LP cohort analysis (both females) as the text and socio-demographic info were identical but texts were posted under different pen names. Details concerning both cohorts after text classification are presented in Table 2.

Table 3 shows comparative data extracted from the two predicted cohorts (SI+ and SI-), after text classification. Beside socio-demographics (sex and age), medical (disease duration) and linguistics (number of words from texts excerpt) statistics are showed. The triggered general responses and the professional responses are compared as well.

4. Discussion

Depression is the most commonly diagnosed mental disease, worldwide. Depressed patients may face serious social stigmata and marginalization, circumstances that may originate suicidal thoughts. In cultures where there is a franc opposition against suicide and apprehension towards mental illness, MDD patients with SI will conceal their mental health issues, making SI screening a notoriously difficult task. Depressed subjects having SI will rather accept to discuss their mental health problems in texts send to social media platforms, where their true identity can be masked. In Romania, the older population of Christian Orthodox faith may show apprehension against both suicide and mental health issues. Mid-aged subjects with major depression and SI will often avoid any open discussion about their problems and will prefer the use of social media platforms for discussions and advice under the protection of anonymity.

Our research interest is to improve SI screening in Romanian MDD patients. As classical clinical screening often fails, we postulated that social media posts that are written in natural language can help the early detection of SI in MDD patients. Forums posts excerpts are usually long but for a machine learning algorithm perspective, text classification is fast once the algorithm is trained. 125 posts from a public depression support forum were included in a “testing dataset”. The text mining algorithm (decision tree C4.5) was trained using a “training dataset” (30 posts). In 15 of them, MDD patients shared their suicidal thoughts. These cases were carefully balanced (sex and age) with 15 posts provided by MDD subjects without any suicidal ideation. After the full text preprocessing, the trained decision tree algorithm classified the posts from the “testing dataset” based on six specific words (suicide wish, sleep, disease, thoughts, death, self-annihilation – translated from Romanian). These words were similar with those mentioned by Tadesse and coworkers when they used different deep learning text classification methods to detect SI in English speaking population (2020). After classification, 32 posts (25%) were classified as having a high probability of suicidal ideation content (SI+) (21 m/11f having a mean age of 34.1±10.2 years and a disease duration of 3.75±1.79 years). 91 posts were considered with a low probability of suicidal ideation content (SI-) (30m/61f, mean age 30.2±10.2 years, disease duration 3.05±1.24 years). Between the two cohorts, only age was significantly different (p=0.05).

We evaluated the clinical precision of the algorithm classification based on socio-demographic (sex, age and disease duration) and linguistic criteria (excerpts length). Finally, we assessed how posts from both cohorts triggered general (peers) and professionals’ responses.

Both training and testing datasets initially included 40% males. After text classification, the SI+ cohort contained 65% males (mean age of 36.7±10.3 years). This group was significantly older than the equivalent group included in the predicted SI- cohort (28% males, mean age 27.5±8.23 years). Our confidence in the text classification precision increased as this observation is in line with “the gender and age paradox” observed in the real-life studies (data sampled from depressed subjects that actually committed suicide). The predicted SI+ males had also a longer major depression history (mean 3.8±3.84 years), significantly different than what was disclosed by males without SI (mean 2.6±2.6 years). As expected, the group of depressed females without SI showed the longest history of disease (average 4.5±6.42 years).

From a computational linguistic perspective, the average excerpt length observed in the predicted SI+ cohort (221.5±157.25 words) was significantly shorter than what was seen in the SI- cohort (329.9± 179.8 words). A similar finding was reported by Lehrman and coworkers based on English texts analysis (2012). The use of the word “disease” between the six classification items may suggest an association with other various clinical diagnostics but there was no qualitative evaluation of the written texts so this observation has to be deepened in a further study.

We evaluated the number of triggered answers coming from peers and from medical professionals. The number of general answers was significantly higher in the SI+ cohort (3.75±1.79 comments). The number of comments coming from medical professionals was significantly lower (1±0.8 advices) in the SI+ cohort. This observation is important as several researchers already stated that clinicians are reluctant to treat individuals with SI because of a suboptimal response to therapy [61]. Loss of one patient due to suicide is seen as one of the most stressful events in the life of a clinician/therapist [62,63]. Our observation concerning the low professional involvement in possible SI cases reflected by social media has to be elucidated in the future.

Limitations

A serious limitation of any text mining study in psychiatry or psychology is coming from the known reality that no known actuarial algorithm can capture the full complexity of a person’s mental state. The low number of texts used in the classification process is another important limitation of our study but it comes from the low number of Romanian depression support forums in use. The absence of any prospective long-term information and the strict quantitative analysis of the excerpts are also limitations of this study.

5. Conclusions

Large text posts written in natural language can be used for suicidal ideation detection in Romanian subjects with major depression. Early detection of suicidal thoughts can prevent a possible suicidal behavior. Trained algorithms can classify social media forums posts (using a text mining approach) and express a probability of suicidal ideation content. Actuarial methods are reliable and fast screening tools, allowing healthcare professionals to early identify individuals at risk and to provide appropriate support and intervention. Future studies are needed to better understand how suicidal thoughts will evolve in time in identified depressed subjects that used web support forums and how social pressure that sometimes exists in the on-line environment can further influence these patients.