CultureManip: A Benchmark for Mental Manipulation Detection Across Multilingual Settings

1. Introduction

Mental manipulation is the intentional use of language to influence or control someone’s thoughts, emotions, or decisions [1]. These manipulative techniques can be subtle and difficult to detect, making them a significant challenge for both humans and artificial intelligence (AI) systems [2,3]. As large language models (LLMs) become increasingly integrated into digital communication, ensuring that these models can recognize and mitigate manipulative language is crucial for preventing misinformation, exploitation, and unethical persuasion As discussed by [4].

Recent efforts, such as MentalManip [5], have primarily focused on detecting manipulative intent within English-language conversations [6,7]. Studies evaluating models like GPT-4 Turbo, LLaMA-2-13B, and RoBERTa-base have revealed that state-of-the-art LLMs struggle to reliably identify manipulative content due to the inherently subjective nature of annotation and the complexity of linguistic manipulation [2].

Figure 1. Example of conversation analysis by ChatGPT 3.5-Turbo.

Figure 1. Example of conversation analysis by ChatGPT 3.5-Turbo.

Our research seeks to address this gap by introducing CultureManip, a multilingual benchmark designed to explore how mental manipulation manifests across different languages. Unlike previous studies that focus solely on English, our work investigates the performance of LLMs in detecting manipulation across linguistic structures and cultural contexts.

To achieve this, we conduct experiments using ChatGPT-3.5 Turbo [8] , evaluating its performance under zero-shot prompting. By examining the cross-linguistic aspects of mental manipulation, our research contributes to the development of more robust AI moderation systems, ultimately fostering safer and more ethical AI-assisted communication across a multilingual spectrum.

2. Related Work

Recent efforts to detect mental manipulation in language have focused heavily on evaluating and improving large language models (LLMs) using the MentalManip dataset Wang et al. [5]. This dataset was used to conducted baseline evaluations using models like GPT-4 Turbo and LLaMA-2-13B. Their findings revealed that current state-of-the-art models struggle with reliably identifying manipulative content. The annotation process, being inherently subjective, introduces potential inconsistencies that may not align with broader societal perceptions of verbal manipulation. Furthermore, [9] demonstrated the difficulty for LLMs to classify fine-grained sarcasm, highlighting the shortcoming of LLMs in detecting human conversational nuance.

Extended work on this topic examined the performance of LLMs using prompting strategies, particularly Chain-of-Thought (CoT) prompting [7]. Their results demonstrated that CoT without example-based learning (i.e., zero-shot CoT) underperformed relative to simpler prompting strategies as model complexity increased.

More recently, a new novel prompting approach was proposed: Intent-Aware Prompting (IAP) [6]. This prompting technique is aimed at improving the detection of mental manipulation by emphasizing the intent behind statements. IAP provided better performance over existing prompting techniques, notably through a substantial reduction in false negatives. The authors recommend the development of a more diverse and representative dataset to improve the generalization of future models.

3. Constructing CultureManip

3.1. Taxonomy

To construct our dataset, CultureManip, we based off of MentalManip [5]. In which our data is separated into 5 columns representing:

• ID: the identification number for each conversation block
• Conversation: Each conversation block
• Presence of Manipulation: Decide whether manipulative or not. (1 for manipulative, 0 for non-manipulative) Represents the Presence of Manipulation line in Figure 2.
• Manipulation Dialogue: whichever line(s) that specify manipulation
• Manipulation Technique: Decide what type of manipulation(s) are present based on the CultureManip taxonomy sheet (Figure 2) [5].

Our taxonomy sheet is a replica of the taxonomy sheet shown in MentalManip however, we did not implement the Targeted Vulnerability aspect. To ensure clarity, we used the definitions given through the MentalManip paper for each manipulation technique [5].

3.2. Data Source and Preprocessing

To support our multilingual analysis, we utilized dialogue data from the OPUS corpus [10], an open collection of parallel corpora available at https://opus.nlpl.eu. Specifically, we extracted data in four languages: English, Spanish, Chinese, and Tagalog.

We used ChatGPT-3.5 Turbo to organize raw dialogue data into conversation blocks by grouping contiguous lines from the same exchange. Each block, representing a short conversation with two or more participants, was then formatted into rows in a spreadsheet.

3.3. Human Annotation

To identify instances of mental manipulation within our multilingual dataset, we enlisted two native-speaking annotators for each target language: English, Spanish, Chinese, and Tagalog. Each annotator was provided with a structured spreadsheet with 100 conversations.

The annotation protocol was standardized across languages, with annotators following shared instructions. For each conversation in Column B, annotators identified manipulative lines, transcribing them into Column C. Column D indicated manipulation presence (1 for manipulative, 0 for non-manipulative), and Column E contained selected manipulation techniques from a predefined taxonomy, including labels like Intimidation, Brandishing, and Anger. An example annotation is shown below:

• Conversation Block: “You’re impossible, you would even delude a saint! I will divorce you!”
• Manipulative Dialogue: “You’re impossible, you would even delude a saint! I will divorce you!”
• Presence of Manipulation: 1
• Manipulation Techniques: Intimidation, Brandishing, Anger

4. Experiments

4.1. Experimental Setup

We conducted experiments in English, Spanish, Chinese, and Tagalog to measure the ability of LLMs to detect manipulation on a multilingual level. Like the human annotators, the LLM was tasked to give a binary response for each conversation, indicating whether there was presence of manipulation. All baseline experiments were carried out using zero-shot prompting, asking the model to read the conversation and detect whether manipulation is present or not, with temperature and top P set at 0.3, as we found that these two values produced the most consistent answers.

We then used a raw inter-annotator agreement score (percentage agreement) to check for agreement between human-human and human-LLM. This was calculated by counting how many times both annotators assigned the same label to a conversation, then dividing by the total number of conversations. The result is a percentage, with a score closer to 1 indicating high agreement and closer to 0 indicating little agreement. In essence, the inter-annotator agreement score represents the raw agreement between annotators or between a human and the model.

4.2. Results

Overall, the results indicate that English remains the language with the highest agreement between the LLM’s predictions and human judgments, which aligns with expectations given the predominance of English data in the model’s training data. The model’s performance decreases progressively with Spanish, Chinese, and especially Tagalog, indicating that performance decreases from high resource to low resource languages.

Table 1. Inter-annotation agreement Scores for different Languages. Group 1 is between two human annotators. Group 2 is between ChatGPT-3.5 Turbo and two human annotators.

Language	Group 1	Group 2
English	62%	48%
Spanish	76%	41%
Tagalog	37%	20%
Chinese	50%	28%

Table 1. Inter-annotation agreement Scores for different Languages. Group 1 is between two human annotators. Group 2 is between ChatGPT-3.5 Turbo and two human annotators.

Language	Group 1	Group 2
English	62%	48%
Spanish	76%	41%
Tagalog	37%	20%
Chinese	50%	28%

Our language selection—three major languages (English, Spanish, and Chinese) paired with a lower-resource language (Tagalog)—was intentional to highlight this disparity. These findings are consistent with recent research such as the MEGAVERSE benchmark [11] and StringrayBench [12], both of which shows that large language models tend to perform significantly better on classification tasks like sentiment analysis and named entity recognition in high-resource languages, while struggling on lower-resource languages like Tagalog.

Between English and Spanish, the agreement was relatively higher (62% and 76%, respectively), indicating a moderately strong consensus between annotators. This suggests that the annotators generally shared similar interpretations and criteria when evaluating dialogues in these languages. The higher agreement in Spanish, even exceeding that in English, could be due to clearer linguistic markers or less ambiguity in the sample dialogues, which is confirmed by both annotators.

In contrast, the agreement dropped substantially for Chinese (50%) and was lowest for Tagalog (37%). The lower agreement scores suggest increased difficulty and ambiguity in annotating manipulation in these languages, which was confirmed by testimonies of the annotators (see Appendix).

5. Multilingual Analysis

Spanish Spanish-speaking cultures rely on high-context communication, using implicit cues and shared background, while English communication is more direct [13]. This cultural difference makes subtle manipulative signals in Spanish harder for GPT-3.5 Turbo, which is primarily trained on English data, to detect [8]. As a result, LLMs like GPT-3.5 Turbo, dependent on surface-level features, struggle to identify subtle manipulation [14]. Moreover, the collectivist values and indirect communication in Spanish cultures, as noted by Hofstede [15], and the use of complex rhetorical features like politeness and figurative language [16], further complicate manipulation detection, leading to lower agreement scores in Spanish than in English.

Tagalog Detecting manipulation in Tagalog is challenging due to the indirect communication style of Filipino culture, where Filipinos often avoid direct confrontation to preserve hiya (shame) and prevent loss of face, using subtle hints and ambiguous language instead [17,18]. This indirectness obscures manipulative intent, making it hard to identify manipulation through text alone. Additionally, the cultural value of Pakikisama, which emphasizes social harmony and conformity, leads individuals to avoid open disagreement [19]. This can bias annotators, causing them to overlook manipulative behaviors. Our annotation experience reflected these challenges, with low inter-annotator agreement and high subjectivity, particularly for Tagalog conversations. Language models like GPT-3.5 Turbo, trained on more explicit languages, also struggle with these subtleties.

Chinese In Chinese culture, maintaining face (miànzi) is a key social value, leading to indirect communication that conceals manipulative intent through nuanced language, body language, and implied meanings rather than direct statements [20,21,22]. This cultural indirectness makes detecting manipulation from text alone challenging, as nonverbal cues like tone and facial expressions are absent. Models like GPT-3.5 Turbo, which rely on surface-level textual patterns and lack cultural grounding, struggle to interpret these subtleties [14,20,21,22]. Additionally, [23] highlights that LLMs tend to overestimate performance, especially in non-Latin languages like Chinese, due to biases and lack of cultural understanding, making reliable manipulation detection difficult for both humans and LLMs.

6. Future Work and Conclusions

This study introduces CultureManip, a new multilingual benchmark for detecting verbal manipulation across languages. Using annotated conversational data, we examined how well models like GPT-3.5 Turbo identify manipulation. Results revealed significant differences in detection accuracy between languages, reflecting cultural variation and the subjective nature of manipulation. These findings echo prior research and highlight the need for more culturally adaptive models. Current LLMs, often trained primarily on English, show reduced effectiveness in other languages—underscoring the importance of multilingual, culturally sensitive approaches to manipulation detection.

Future work should refine datasets and develop standardized annotation guidelines to improve consistency across languages. Studying cross-lingual generalization can reveal shared or language-specific manipulation patterns [24]. Incorporating multimodal data—such as tone or gestures—may boost accuracy by adding context [25]. To address subjectivity and bias in monolingual data, localized guidelines are essential. Exploring sources like podcast transcripts or real conversations may also yield more natural dialogue than subtitle-based data.

Although detecting manipulation is a challenging task, both for humans and LLMs, the development of CultureManip serves as a foundational step in advancing multilingual manipulation detection. As LLMs continue to evolve, it will be crucial to assess the capabilities of smaller, more specialized models for this task. By making CultureManip publicly available, we hope to inspire further research and innovation in the field, encouraging the development of more accurate, context-aware, and culturally sensitive manipulation detection systems. This work lays the groundwork for future efforts aimed at improving the reliability of LLMs in detecting manipulation across diverse languages and cultural contexts.