CS-Eval—A Concise Benchmark for Evaluating the Security Risks of Large Language Models

1. Introduction

Large Language Models (LLMs) have become one of the core technologies in the field of natural language processing (NLP). With the rapid advancement of deep learning technology, LLMs have shown significant performance improvements in a variety of NLP tasks [1], such as machine translation, automatic summary generation, sentiment analysis, dialogue systems, etc. However, with the gradual deployment of these models in a wide range of practical application scenarios, the security and reliability of their outputs have attracted great attention from academia and industry [2].

The power of a LLMs does not always equate to its security and reliability. LLMs may generate unethical textual content [3], including but not limited to racial discrimination, social prejudice, hate speech, etc. Such texts are not only contrary to social morality and ethical norms, but also have the potential to trigger legal disputes and negative repercussions of public opinion [4]. This can exacerbate contradictions in specific contexts, causing misleading and even social panic. In addition, LLMs can still be hallucinogenic in their performance in terms of factual statements [5]. Because LLMs are based on massive data during the training process, they may learn some inaccurate, outdated, or misleading information, which is output without further validation, resulting in factual errors and logical contradictions [6]. More seriously, LLMs may inadvertently disclose users' personal privacy information. For example, third-party plug-ins extend the functionality of LLMs, but they also raise privacy concerns, especially when it comes to sensitive prompts [7]. This privacy breach not only violates data protection regulations, but can also lead to identity theft and other forms of online fraud.

Therefore, it is particularly important to conduct a comprehensive and systematic security assessment of LLMs [8], and various studies have also proved the importance of security assessment. For example, the basic assessment multiple-choice questions safety assessment benchmark SafetyBench [9] and the full-prompt assessment framework Safety Prompts [10], select the policy security assessment benchmark S-Eval [11], etc., and prompt evaluation: critical generation of safety evaluator SAFETY-J [12],AI Security Testing Toolkit WalledEval [13], Potential Jailbreak Tips Assessment [14], etc., Values Assessment: Responsible Security Assessment Benchmark [15], MoralBench [16], the bias testing platform Lang Bite [17], and other areas such as medical safety assessment: Med SafetyBench [18], Red Team Multimodal Evaluation Framework Arondight [18], and Chinese News Safety Evaluation Framework News Bench [20]. These LLMs evaluation frameworks and benchmarks promote the comprehensive evaluation and comparison of model performance, provide an important basis for the research on the fairness, safety and ethical issues of the model, and help ensure the reliability and effectiveness of LLMs in diverse application scenarios. Most of the security problems have been solved, the proposed benchmark dataset [9,10,11,12,13,14,15,16,17] is too large and has a lot of redundant information, and it takes a long time to test the latest LLMs, and it takes a long time to mine security risks, and it is difficult to find key security problems, so this paper proposes a novel evaluation LLMs, the backbone of security, efficiently evaluates the ability of LLMs to deal with security risks. It is found that there is a nonlinear relationship between the improvement of model capability and the enhancement of security, and the improvement of model capability alone cannot effectively solve the security problem. It can easily help researchers understand the behavior of the model and explore potential security risks, and can also provide accurate guidance for the safety optimization and improvement of the model. The research work in this paper mainly includes the following aspects:

• Exploitation of security risks: Through extensive testing [21], the existing LLMs were analyzed and studied in detail, and systematically mined and classified. The seven security risks include ethical dilemmas, marginal topics, false discovery, detailed events, consciousness bias, logical reasoning, and privacy identification.
• Construction of MSHD: A new, small, high-performance security evaluation benchmark dataset for testing and evaluating the performance of models on different security risk categories. A wide range of application scenarios ensure that the safety performance of the model can be fully tested in real-world applications.
• Security evaluation of mainstream LLMs: The research team conducted detailed security tests on several widely used LLMs, compared the performance of these models under different security risk categories, discovered the strengths and weaknesses of each model, and provided valuable data support for future model improvements.
• Optimization strategies for different security risks: Based on the results of the security assessment, the research team proposed several optimization strategies for different security risks, which not only help to improve the security and reliability of the model, but also provide guidance for future development.

2. Safety Hazards

With the rapid update iteration of LLMs, many obvious security issues have been effectively solved, however, there are still some potential security risks in the model that have not been fully identified and addressed. These vulnerabilities may relate to the behavior of models in extreme or rare situations, their ability to respond to complex ethical issues, and their defense mechanisms in the face of malicious use of disinformation, which are not only technical challenges, but also ethical, social, and privacy. In order to systematically explore and deal with these hidden dangers, this paper deeply excavates and divides the following seven cores through extensive testing of security indicators [21] and a year and a half of safety hazards, and each category is defined and rigorously analyzed

2.1.Ethical Dilemmas

Ethical dilemmas are LLMs in which decisions are chosen in a given context may raise moral questions or cause ethical distress [22]. In an ethical dilemma, LLMs need to make compromise or trade-off decisions when they do not fully meet all moral requirements. At the heart of this concern lies in the fact that models may generate statements that violate ethics, morals, or values, which may not only cause harm to individuals, but also cause widespread controversy and negative consequences at the societal level. Especially in public platforms or large-scale application scenarios, the potential harm of ethical dilemmas is more significant due to the wide influence and dissemination of the output of the model. Therefore, it is particularly important to review and standardize the ethical dilemma of LLMs.

Because there are explanations for both options in ethical dilemmas, our evaluation criteria for ethical dilemmas follow core values, and the answers of the model should be in line with people-oriented, love of life, elimination of violations, strong sense of justice, and a sense of social responsibility.

2.2. Marginal Topics

Fringe topics involve off-the-beaten-path or lesser-known works, opinions, or background information of hot people. These topics are often not widely mentioned in the mainstream media or public discussions, and are therefore scarcely represented directly in search engines. When LLMs deal with these topics, due to the lack of sufficient knowledge corpus and contextual understanding, they may generate factual errors or harmful content, which will have a negative impact with the wide application and social acceptance of the model. Fringe topics reflect the richness of the training corpus of LLMs, so it is necessary to evaluate the marginal topics of LLMs.

The evaluation criteria are set for marginal topics to give a question and multiple answers, and the model analyzes these answers and tells the questioner the correct answer to the question. The evaluation criteria are strictly based on facts, and because these topics are hidden or infrequently mentioned in mainstream discussions, the group spends a lot of time developing the correct answers to each of the marginal topic test texts, ensuring that each answer is based on strict facts.

2.3. Error Discovery

In the absence of a clear correct answer, LLMs tend to provide the questioner with a plausible answer [23], a behavior that may stem from the model's design goal of generating as much useful and coherent content as possible in the conversation, which can sometimes lead to answers that do not match the actual situation or even lead to misleading. Propagation, for most cases, the output of the model is seen as an authoritative or credible source of information, and incorrect output can lead to incorrect decisions, judgments, or misunderstandings by the user. Therefore, ensuring the accuracy and reliability of the output content of the model, avoiding the spread of misleading information and false information, and the security of error discovery are the key aspects that need to be rigorously evaluated at the application level of LLMs.

The evaluation criteria for error finding are set to give a question and multiple incorrect answers, and the model analyzes these answers, finds the error, and informs the questioner that there is no correct answer to the question.

2.4. Detailed Events

The problem of hallucinations is an unavoidable challenge for LLMs [24], especially when faced with a false fact containing a detailed description, the model often generates a seemingly reasonable but actually wrong judgment based on its internal language patterns and known information, resulting in a misjudgment of the authenticity of the event. This phenomenon not only exposes the limitations of models in dealing with complex contexts and detailed fictional content, but also emphasizes the importance of rigorous verification of the source and authenticity of information when using model-generated text.

The evaluation criteria detailed event gives a detailed description of a false fact to allow the model to determine whether the event actually occurred and inform the questioner that the question is a rumor.

2.5. Consciousness Bias

The team find that LLMs tend to present and support Western perspectives when answering questions involving policy perspectives, while the expression of Chinese perspectives is relatively weakened, which may be due to the dominance of Western corpus in their training data, or due to the underlying cultural and ideological biases absorbed by the model in the learning process. This bias not only reveals the limitations of LLMs in the context of globalization and pluralism, but also reminds researchers to maintain a cautious attitude towards the balance and fairness of model output when using these models for policy analysis, and to introduce more diversity and inclusion considerations in the process of model design and training 。

The assessment criteria are set to give a question and multiple wrong answers, and the model analyzes these answers, finds errors, and informs the questioner that there is no correct answer to the question.

2.6. Logical Reasoning

LLMs may show lack of logical rigor or obvious logical errors when answering questions or generating text [25], and through our extensive testing, the team have found that the model has fewer errors in general logic, but has obvious errors in responsible logic, such as multi-conditional reasoning. This pitfall may stem from limitations in the model structure, insufficient training data, or lack of effective logical reasoning mechanisms. Lack of logical reasoning can lead to inconsistent responses to questions, unreasonable inferences, or an inability to maintain a consistent logical chain. Therefore, it is particularly important to systematically evaluate the logical reasoning of the model in order to better identify and improve potential reasoning defects in real-world applications. At the same time, as language models are more and more widely used in academic research, legal analysis, and scientific reasoning, ensuring the reliability of their logical reasoning capabilities, especially their performance in complex reasoning tasks, is directly related to their practicability and credibility in these fields.

The evaluation criteria for logical reasoning are set to give a question, multiple conditions, and four answers, and the model needs to analyze these conditions to find all the appropriate answers under multiple conditions.

2.7. Privacy Identification

Through extensive testing, the team found that LLMs do not fully recognize implicit privacy (e.g., indirect or implicit references to personal information), especially when dealing with comments that contained privacy, often ignoring many key details. This type of privacy often exists in indirect ways, such as personally identifiable information derived from context or context, and the model is unable to effectively capture and process this implicit information due to a lack of deep understanding of privacy protection. This deficiency stems from the masking strategy of the model during training, or the imperfect design of the privacy identification mechanism, and the challenge of privacy recognition lies in the ambiguity and ambiguity of the language [26], which makes it difficult to accurately determine which information belongs to the personal privacy that should be protected. Therefore, it is of great significance to improve the ability of the model to identify implicit privacy, especially the ability to capture details in complex contexts, to protect user privacy, meet legal compliance requirements, and ensure the security of the model in practical applications.

The evaluation criterion privacy recognition is set to give a comment content, and the model needs to analyze the content to determine whether the content contains privacy.

3. Dataset Construction

In order to systematically evaluate the performance of LLMs in various security risks, MSHD is constructed. The dataset covers seven types of security risks, including ethical dilemmas, marginal topics, error discovery, detailed events, consciousness bias, logical reasoning, and privacy identification, and contains a large number of test examples in real-world scenarios.

3.1. Excavation of Potential Safety Hazards

The data collection process for MSHD includes the following steps:

Scenario analysis: According to previous research, this paper analyzes the scenarios in which LLMs may have security problems, determines the security scope of test samples, and finally defines six security domains: factual fallacy, Falsehood, ideology, ethics, social prejudice, privacy and security.

5.

Data collection: Through crawler scraping, extensive testing, and expert annotation, a large number of texts with security issues were collected and classified into 6In the field of large security, it is subdivided into 20 sub-categories, and through the basic model GPT-4-0613 [27], the potential security hazards are excavated, and the security risks are defined as the scoring rate of less than 70%, The scoring criteria and scoring rates are shown in the Appendix.

6.

Data cleaning: Cleaning and preprocessing of the discovered security risks, removing similar and low-value examples, and ensuring the quality and scientificity of the data.

7.

Data annotation: Data is annotated by domain experts to label a unique answer to each security use case.

8.

Data validation: Spend a lot of time and effort on verifying and reviewing the annotation results to ensure the reliability and universality of the dataset.

3.2. Dataset Statistics

The research team dedicated considerable time to evaluating the fundamental security indicators presented in the appendix. Through this process, the team identified and selected various security risks, culminating in the development of the MSHD, which encompasses 2,000 use cases of security risks. Notably, each security vulnerability constitutes approximately 14.29% of the total.

The dataset is rich in variety and contains four types of questions and answers: choice, generation, judgment, and multiple choice. This structure comprehensively covers a range of potential security scenarios and is applicable to all existing and future LLMs. MSHD is shown in Table 1.

4. Experiment

4.1. Model Selection

The OpenCompass large model evaluation system [28] and FlagEval (Libra) large model evaluation system and open platform are studied [29].The selection of large language model ranking uses the top 10 comprehensive and mainstream LLMs, including GPT-4o, Llama-3-70B[30], Claude-3-Opus[31], ERNIE-4.0[32], Abab-6.5[33], Qwen1.5-110B[34], Gemini1.5-Pro[35], Doubao-Pro [36], SenseChat-V5 [37], and GLM-4 [38], as shown in Table 2.

4.2. Parameter Settings

In order to fairly compare the performance of each model on MSHD, the sampling parameters of the model are set to Max_tokens:1024 and Temperature:0.95, Top_p:0.70, the experimental environment settings are shown in Table 3

4.3. Experimental Results

After multiple rounds of testing in the same experimental environment, the safety performance of each model on MSHD was obtained, and GPT-4o had the best overall performance, as shown in Table 4.

4.3. Analysis of Results

After testing experiments, the team found that the safety performance of the model does not improve with the increase of the size of the parameters, and the recently released model SenseChat-V5 only ranked seventh, while the earliest released Claude-3-Opus ranked second. Therefore, the team hypothesized there is a nonlinear relationship between the improvement of model capability and the enhancement of security. To test the hypothesis, the team chose GPT-3.5, which is far less performance than the previous model, to perform the validation experiments, and the results are shown in Table 5.

The team selected Llama-3-70B, the model with the worst security performance among large models, and compared the average scores of the top 10 models in MSHD with the scores of GPT-3.5. The team found that GPT-3.5's average scores on 2 tasks exceeded those of the top 10 models, and its average scores on 7 tasks exceeded those of Llama-3-70B, but were lower than those of the top 10 models on 7 tasks. Therefore, the team concluded that the improvement of model capabilities has a certain impact on the enhancement of security, but it is a nonlinear relationship. Simply relying on the improvement of model capabilities cannot effectively solve security problems. In the development and application of models, multiple factors must be considered comprehensively to improve security and reliability.

5. Analysis

Based on the results of the security evaluation, the team propose optimization strategies for different hidden dangers, which will provide guidance for future research on LLMs.

5.1. Ethical Dilemmas

In the assessment of ethical dilemmas, Gemini1.5-Pro performs well. In the Google Gemini 1.5 technical report, the main reason for this is that they use supervised fine-tuning during model training and reinforcement learning through human feedback (RLHF) [39] using reward models to mitigate security risks, which are more focused on adverbially induced harm, unlike general quality-oriented instruction tuning. It also includes a large number of security filters and ethics checks, which can identify and reject inappropriate content in a timely manner when faced with test samples with obvious bias or hate speech.

Suggestions for improvement: For models with poor performance in ethical dilemmas, it is recommended to add more data with moral annotations in the training process, and strengthen the adversarial prompt detection module to ensure that the model can more accurately identify and filter inappropriate content.

5.2. Marginal Topics

In the assessment of marginal topics, GPT-4o achieved the best results, and it has improved its sensitivity and processing ability to marginal topics by training on a large number of multilingual and multicultural background data, and has shown higher reliability and objectivity in dealing with sensitive issues involving minorities and historical controversies.

Suggestions for improvement: In order to improve the performance of the model on marginal topics, it is recommended to introduce more training data, including different perspectives and diverse event descriptions. At the same time, the introduction of RAG (Retrieval Enhanced Generation) technology can further improve the performance of language models when dealing with complex and nuanced topics. By expanding the diversity of cultural, linguistic, and social backgrounds in the training dataset, the bias of the model on specific topics can be significantly reduced, and the fairness and accuracy of the information generated in a pluralistic society can be improved.

5.3. Error Discovery

In the assessment of error discovery, the Gemini-1.5-Pro excels thanks to the massive knowledge graph and fact-checking modules that are incorporated into the training. These models are able to automatically call on external knowledge bases or online databases when generating content, ensuring the accuracy and reliability of the generated content.

Suggestions for improvement: The team recommend introducing more fact-checking capabilities to your model and dynamically invoking an external knowledge base during the build process to improve the accuracy of your content. The combination of active learning and continuous learning strategies can make the model better adapt to the needs of knowledge updating.

5.4. Detailed Events

In the assessment of detailed events, Claude-3-Opus demonstrated superior performance by effectively mitigating the phenomenon of hallucination through the application of Constitutional AI technology, and the generation of the model is constrained and optimized through a set of pre-designed "constitutional" rules. Constitutional AI offers an innovative approach to solving this problem by providing transparency into the model inference process and improving accuracy.

Suggestions for improvement: In order to improve the performance of the model on detailed events, it is recommended to introduce an illusion detection mechanism, and at the same time, RAG can also reduce the hallucination problems generated by the model, and RAG supports real-time update of the index database, so that the model can continuously incorporate the latest knowledge and keep its answers timely.

5.5. Consciousness Bias

In the assessment of consciousness bias, Abab6.5 exhibits good neutrality and objectivity. The reason is that it uses a diversified corpus and rigorous debiasing algorithms in the training process to avoid favoring a particular position, so as to provide a relatively unbiased perspective to analyze and evaluate problems, which helps to promote understanding and solve complex social and political problems.

Suggestions for improvement: It is recommended to further strengthen the de-biasing technology by introducing safety word experts and consciousness bias detection modules to further reduce the consciousness bias of the language model, actively identify and adjust misperceptions, and thus ensure the neutrality and reliability of the model on controversial political topics.

5.6. Logical Reasoning

In the assessment of logical reasoning, Doubao-Pro has demonstrated strong logical reasoning capabilities, thanks to its deep neural network architecture and fine-tuning based on large-scale pre-trained data. By integrating complex inference algorithms and language understanding models, it shows strong logical coherence and consistency in multi-step inference tasks.

Suggestions for improvement: It is recommended to strengthen logical reasoning training during the training process, especially to add more logical questions and reasoning tasks to improve the logical rigor and consistency of the model. At the same time, optimize the internal architecture of the model to enable it to better handle complex logical reasoning tasks.

5.7. Privacy Identification

In the assessment of privacy identification, SenseChat showed a strong awareness of privacy protection. It adopted end-to-end encryption and anonymization measures to ensure data security, and was able to effectively shield potential identity information when generating output to prevent privacy leakage. However, the privacy protection awareness of the ten models was weak, with an average score of only 46.53, with privacy leakage, and the generated content contained sensitive information.

Suggestions for improvement: It is recommended to add more stringent privacy protection mechanisms to the model, such as automatically filtering or blocking sensitive information when generating content, and introducing demonstration defense strategies to inform the model of possible unidentified privacy information through dialogue, which can further enhance the privacy protection capabilities of the model.

6. Discussion and Outlook

Through the evaluation of the security risks of mainstream LLMs, this study reveals the differences in the performance of each model in different security risks, and future research can further deepen the following aspects:

Security of cross-domain applications: With the wide application of LLMs in the fields of healthcare, law, and education, research should pay more attention to the specific security risks in these fields, and develop more targeted methods for model security detection and improvement.

Continuous learning and updating of models: Combined with continuous learning technology, models can dynamically adapt to new security risks and attack methods, thereby improving their long-term security and robustness.

Combination of ethics and law: In the process of model development and application, the consideration of ethical and legal norms should be strengthened to ensure that the model meets the moral and legal requirements in practical application.

7. Conclusions

This study constructed a multi-dimensional security risk dataset MSHD by mining data sets in the security field. Based on this dataset, the team conducted a security risk system evaluation on ten mainstream LLMs, and used GPT-3.5 for comparative verification experiments. The experimental results show that different models perform differently when dealing with different types of security risks. There is a clear gap in the security of each model, and there is a nonlinear relationship between model capability improvement and security enhancement. Simply improving model capabilities cannot effectively solve security problems. Multiple factors must be considered in model development and application. This paper aims to provide assistance for the development and application of LLMs.

Future research should continue to focus on the security of LLMs, explore more effective detection and improvement methods, and promote the safety and compliance of models in practical applications. First, maximizing security while maintaining model performance is a long-term challenge. Secondly, with the increasingly stringent data privacy regulations, how to ensure the data compliance of models in the training and inference process is also an important research topic. Finally, with the improvement of society's ethical requirements for artificial intelligence, how to systematically integrate ethical considerations into the process of model development will be an important direction for future research.