Intelligent Gas Risk Assessment and Report Generation for Coal Mines: An Innovative Framework Based on GLM Fine-tuning

1. Introduction

Coal mine risk assessment plays a crucial role in modern coal mine safety management [1], serving as a vital foundation for helping the mining management team identify problems and potential risks throughout the entire lifecycle of coal mine operations [2,3,4]. This ensures compliance with high safety standards and facilitates the timely completion of assessments. By monitoring on-site equipment and generating scientific assessment reports, the mine management team can gain an accurate understanding of the ongoing gas monitoring process and the corresponding mine safety status. In this process, various types of mine sensors, especially those detecting gases such as methane and carbon monoxide, provide essential environmental data, forming the technical basis for the development of mining safety Internet of Things (IoT) and smart mine systems [5]. Intelligent report generation systems, built upon the data collected by these sensors, can swiftly identify quality and safety hazards, thereby reducing the likelihood of accidents. Moreover, these reports can pinpoint potential issues in gas monitoring, helping to mitigate risks related to project delays and cost overruns. This enables smart perception, comprehensive monitoring, autonomous analysis, early warnings, and effective control, thereby enhancing the overall safety management capabilities.

The traditional manual assessment process typically requires trained experts who generate evaluation reports based on data collected from on-site sensors. However, due to the complexity of the mine environment and the vast amount of monitoring data, the report generation process demands substantial human and material resources [6,7]. Moreover, this process heavily relies on the subjective judgment of the personnel, which inevitably leads to the risk of overlooking, insufficiently addressing, or improperly handling certain issues [8]. As a result, with the increasing pursuit of optimal control in coal mines, challenges continue to intensify, highlighting the urgent need for automated analysis and decision-making. The emergence of large language models (LLMs), particularly their applications in natural language processing and intelligent decision-making, has provided effective solutions for complex data analysis and intelligent report generation.

Based on this, the Intelligent Gas Risk Assessment and Report Generation Framework for Coal Mines (IGRARG) designed in this study integrates multi-source sensor data fine-tuning in coal mine environments and a multi-agent collaboration mechanism of LLMs to automatically generate gas risk assessment reports. It provides real-time alarm judgments, emergency measure suggestions, and trend analysis. The goal is to address the inefficiencies and unreliability associated with traditional manual assessment reports.

Specifically, the main contributions of this paper are as follows:

• This paper meticulously constructs a high-quality risk assessment (RA) dataset based on the gas alarm data collected by sensors at the coal mine production site, including historical alarm data from various measurement points such as CO, laser methane, smoke, etc., for fine-tuning a general-purpose large language model (LLM).
• Based on real-time data collection from wireless sensor networks and coal mine gas judgment standards, a specialized knowledge base is constructed. Through the integration of RAG (Retrieval-Augmented Generation) technology, efficient indexing, semantic matching, and intelligent inference analysis of the knowledge base are achieved, forming an efficient workflow covering alarm judgment to emergency measure suggestions, and ultimately enabling the automatic generation of safety reports for mine production.
• The framework adopts a hierarchical graph structure from LangGraph to optimize the collaborative interaction between LLM multi-agent systems. Through fine-tuning the parameters of agent models and task scheduling configuration, it ensures the efficient execution of report generation tasks within the predefined workflow. A Human-in-the-loop feedback mechanism is introduced to strengthen the model's decision-making through real-time user feedback.

The structure of this paper is as follows: Section 2 reviews the literature in the relevant field. Section 3 provides a detailed explanation of the proposed research framework. Section 4 describes the scene data collection and processing techniques, the technical details of the LLM, fine-tuning and debugging, as well as the specific processes for gas assessment and report generation within the framework. Section 5 presents the evaluation and validation of the framework to ensure its effectiveness in practical applications. Section 6 summarizes the research findings and outlines future research directions. The long-term goal of this paper is to contribute to the advancement of automated report generation for risk assessment.

2. Related Work

In this section, we review the existing literature relevant to our study. Current methods and techniques used for coal mining research are first discussed, emphasizing their limitations and the need for more advanced methods. We then explore recent advances in large language modeling and its application in various domains, providing the basis for our proposed framework. This comprehensive review provides a foundation for understanding the context and significance of our work.

2.1. Current Research on Intelligent Assessment Systems

Gas monitoring in coal mines is a critical phase in ensuring safe production. In recent years, technological advancements have led to significant changes in methods and tools used in this area. Traditional manual detection and analysis methods are often limited by their reliance on human resources and low process efficiency [9,10]. To overcome these limitations, the application of emerging technologies has become a key research direction in this field. Recent studies have focused on using automation technologies and intelligent systems to improve safety monitoring and decision-making processes. For example, some studies have explored the application of drones and IoT technologies in tunnel and mine monitoring [11,12,13], demonstrating how these tools can enhance the efficiency of assessments as part of intelligent monitoring systems. Similarly, other studies have examined the use of big data analytics in intelligent risk assessment [14,15,16]. These studies show that big data analysis and sensor integration can effectively cover gas monitoring areas within mines, reducing the risks to personnel during the risk evaluation process.

With the development of automation and data-driven technologies, the application of methods such as Artificial Intelligence (AI) and Machine Learning (ML) has introduced new perspectives for optimizing intelligent monitoring systems [17,18,19]. These methods leverage advanced algorithms and predictive model techniques to automate gas detection and alarm processes, enhancing the accuracy and efficiency of safety monitoring. Retrieval-augmented Generation (RAG) and rule-based reasoning are also crucial components in the field of intelligent assessment systems. The integration of semantic search and knowledge graph technologies has enabled coal mine safety monitoring systems based on diverse textual data [20], providing novel insights into the relationship between monitoring results and the risks of gas overexposure. At the same time, intelligent monitoring frameworks based on real-time data analysis and machine learning models offer an effective method for identifying and assessing gas risks [21]. AI technologies and data-driven models not only improve the efficiency of risk assessment but also ensure worker safety and project compliance [22,23,24]. The application of these technologies enhances both the efficiency of assessments and the safety of coal mine workers.

Furthermore, the technological development trends in intelligent monitoring systems indicate that enhanced sensor networks and AI-driven analytical capabilities will play a significant role in future gas monitoring and safety management. The use of IoT-based sensing technology in tunnel construction provides real-time on-site inspection capabilities [25], while intelligent predictive analytics highlights its potential to improve the efficiency of safety monitoring [26]. Overall, these studies reveal a shift in traditional coal mine gas monitoring methods toward technology-driven automation and intelligence, bringing both new opportunities and challenges to the coal industry.

2.2. Current Research on LLMs and Multi-Agent Structures

In recent years, the development of LLMs has continually propelled technological advancements in the field of natural language processing. Recent research has demonstrated the potential of these models across various application domains. For instance, GPT-4 [27] has shown greater accuracy in text generation and contextual understanding, further expanding the application scope of pre-trained models. Additionally, InstructGPT [28] has significantly improved the understanding and execution of user instructions by introducing an instruction-following mechanism within generative models. Lang Chain [29] enhances the reasoning capabilities and contextual understanding of models by integrating LLMs with knowledge graphs. Innovations in Transformer architectures, such as T5 [30] and RoBERTa [31], have also greatly improved model performance and application range. Furthermore, LLMs have been applied in specific domains, such as legal text analysis [32], medical diagnosis [33], and financial risk assessment [34]. These applications illustrate the deep integration and broad potential of LLMs in specialized fields.

Current research indicates that certain supervisory signals established through natural language processing can continue to train well-transferable visual models. This foundation paves the way for the emergence of multi-agent structures within LLMs [35,36]. The multi-agent structure of LLMs integrates task allocation and collaborative processing capabilities, representing a significant advancement in the field of NLP [37]. These structures can not only handle complex language data but also effectively process multiple tasks or functions through well-coordinated collaboration among agents, thereby enhancing the efficiency and accuracy of task execution. For instance, researchers from Tsinghua University, led by Sha et al., integrated LLMs into autonomous driving systems, leveraging their common-sense reasoning capabilities to enhance decision-making in complex scenarios, thereby improving safety, efficiency, and multi-vehicle coordination [38]. Similarly, Yubin Kim and colleagues demonstrated that by automatically allocating collaborative structures for LLMs, they could simulate real-world medical decision-making processes, achieving outstanding results across multiple medical benchmarks. Their research indicates that Multi-Agent Dynamics (MDAgents) can effectively adjust the number of agents to optimize efficiency and accuracy [39].

Despite significant advancements in LLMs and multi-agent structures across various fields, research and practical applications in specific areas such as coal mine safety monitoring and report generation remain largely insufficient. Currently, gas monitoring and assessment in coal mines primarily rely on manual execution and analysis, which is not only time-consuming and labor-intensive but also limited in processing real-time data in high-risk or complex environments. Furthermore, although there are studies on utilizing IoT technologies to assist with safety monitoring, these technologies often do not incorporate advanced methods using LLMs, multi-agent systems, and RAG techniques to understand and generate detailed assessment reports.

To bridge this gap, our research aims to explore the possibility of automatically generating coal mine gas monitoring and risk assessment reports by integrating an LLM-based multi-agent structure with real-time sensor data. This approach is expected to enhance the efficiency and accuracy of assessments, especially when dealing with complex tasks and data analysis. Existing research demonstrates that LLMs and RAG technologies possess strong capabilities in understanding and generating textual content. However, applying these technologies for automatic interpretation of coal mine monitoring scenarios and generating comprehensive reports remains an underdeveloped area. Therefore, the vision of this paper is to contribute to the advancement of automated coal mine risk assessment processes by establishing an intelligent framework that drives the automation of coal mine gas monitoring and risk assessment.

3. Overall Framework of IGRARG

The intelligent coal mine gas risk assessment and report generation system designed in this paper aims to collect real-time gas alarm data from various sensors on-site (such as methane, carbon monoxide, etc.) and generate standardized risk assessment reports based on this data. The generated reports include three main types: alarm judgment reports, action recommendation reports, and weekly reports. The goal is to ensure that the entire process, from data collection to report generation, is efficient, accurate, and highly operational.

The research framework presented in Figure 1 integrates cutting-edge technologies and is divided into two main modules: the data collection and processing module, and the intelligent risk assessment and report generation module. It constructs an optimized workflow to enhance the efficiency of coal mine safety management and the quality of report generation. The framework consists of two parallel processes that work together to support the automatic generation of intelligent risk assessment reports. In the data collection and processing phase, the system monitors methane, carbon monoxide, and other gas concentrations in real-time through sensors deployed at key locations in the mine, ensuring that the data in the database is up-to-date. Meanwhile, the training of the LLM proceeds concurrently, starting with the preparation of a pre-trained model as a foundation and creating a dataset for coal mine gas risk assessment. By cleaning and de-sensitizing the collected sensor data, a high-quality risk assessment dataset is built, which is then used to fine-tune the model, specifically optimizing its performance in coal mine risk assessment scenarios. Additionally, the system uses a vector database for efficient data storage and retrieval, constructing a local knowledge base. By integrating Retrieval-Augmented Generation (RAG) technology, the system dynamically retrieves relevant historical data and safety standards during report generation, ensuring an effective combination of knowledge base information and the generation tasks.

Through the real-time data update mechanism, the system ensures that the local knowledge base continuously reflects the latest status of coal mine gas monitoring, providing support for accurate alarm judgments and report generation. This process guarantees high accuracy and relevance of on-site data, offering a reliable foundation for subsequent analysis. The system retrieves and analyzes data based on gas alarm standards to achieve precise alarm judgments.

In the intelligent risk assessment report generation process, the measures and suggestions layer retrieves relevant treatment methods to propose specific solutions. The weekly report module automatically generates a comprehensive report, including trend analysis, risk assessment, and improvement measures, based on the weekly gas monitoring data, alarm records, and processing suggestions. Overall, the report generation process includes the automated generation of alarm judgments, measures and suggestions, and weekly reports. The system integrates alarm retrieval results and relevant charts to automatically output standardized coal mine risk assessment reports. The generated reports will be confirmed for accuracy and reliability after rigorous quality checks and manual review. The entire process is coordinated and managed by the Supervisor module, ensuring efficient collaboration across stages for rapid and accurate report generation.

4. Key Modules

4.1. Data Collection and Processing Module in Coal Mine Scenarios

This section introduces the design and implementation of the coal mine data collection and knowledge base construction module in the IGRARG framework. This module collects real-time data from key areas of the mine through multiple sensors, ensuring comprehensive environmental monitoring. It also integrates coal mine safety regulations and alarm standards to construct a dynamically updated local knowledge base. To enhance model performance, the collected data is cleaned and processed to build a high-quality coal mine risk assessment dataset, which is then used to fine-tune a pre-trained LLM, optimizing its performance in coal mine gas risk evaluation. This design ensures the real-time nature of data collection and the accuracy of decision support, providing a solid foundation for the system's intelligent analysis and emergency response.

4.1.1. Data Collection

In the IGRARG framework, the data collection module is a critical component for monitoring coal mine environmental safety. By deploying various sensors, the system can obtain real-time data on gas concentrations, environmental conditions, ventilation status, and geological parameters from key locations within the mine. The efficient layout and coordinated operation of these sensors ensure comprehensive monitoring of the entire mine area, providing the system with a wealth of fundamental data. The process of equipment data collection and transmission is shown in Figure 2.

The deployment of sensors in various areas and the types of data collected are shown in Table 1. These sensor data are transmitted in real time to the central processing system via a wireless network, ensuring that the data in the database is always up to date. The data undergoes preliminary processing, including data cleaning, noise reduction, and outlier removal. After standardization, this data is used for further analysis and decision-making, providing robust data support for the efficient operation of the system.

Through the collaborative work of multiple sensors, the IGRARG framework is capable of covering key areas of the mine, ensuring comprehensive data collection and real-time monitoring. This high-quality sensor data not only provides a reliable foundation for subsequent analyses but also lays a solid groundwork for monitoring judgments and automated report generation.

4.1.2. Data Processing and Knowledge Base Construction

In the IGRARG framework, the construction of the knowledge base is primarily achieved through the systematic processing and organization of the data collected by the sensors. This process ensures the accuracy and consistency of the data, providing strong support for the subsequent generation of intelligent evaluation reports.

First, the real-time data collected by sensors from key areas of the mine undergoes preprocessing, including denoising and outlier removal, to ensure data quality. After basic processing, the data is categorized and stored in a structured manner within the local knowledge base. The system employs a combination of structured and unstructured data, integrating vector databases and RAG (Retrieve and Generate) technology to mitigate the hallucination problem of large models, thus enabling efficient data storage, retrieval, and dynamic updates.

The system's data sources are diverse, including real-time sensor data monitoring, alarm determination standards, and historical alarm event records. The primary data consists of alarm data collected in real-time from various areas of the coal mine, such as alarm types and alarm durations. This data is in a fixed format, making it easy to process and store. Such data provides real-time information on the coal mine environment, serving as an important reference for the system’s immediate evaluations.

In contrast, unstructured data includes content like coal mine safety regulations and industry standards. This data is typically in text form, with diverse and complex content. Traditional methods struggle to efficiently handle this diverse information when dealing with gas over-limit alarms and formulating safety countermeasures. Therefore, we use a database to efficiently manage and process this data. To improve the system's data retrieval efficiency and reasoning accuracy, we introduce database management into the knowledge base. Through vectorization, the system converts both structured and unstructured data into high-dimensional vector representations. By utilizing vector similarity search technology, the system can quickly retrieve regulations and historical records relevant to current monitoring standards or needs. The database indexing mechanism ensures that the system maintains a high response speed even when handling massive amounts of data.

During the coal mine intelligent monitoring process, the system continuously relies on the database for real-time data retrieval and matching. Using RAG technology, the system can efficiently integrate data from the knowledge base and dynamically update relevant information. Figure 3 illustrates the specific flow of knowledge retrieval.

The specific operations are as follows:

• Retrieval Phase;

When the monitoring system detects changes in gas concentration, it invokes the vector database to quickly match relevant safety standards, regulations, or historical event records through the RAG retrieval module. This retrieved information provides a basis for subsequent analysis and judgments.

• Generation Phase;

The retrieved data is sent to the generation model, which, combined with existing real-time data and historical records, automatically generates alarm judgments and recommended response measures. RAG technology ensures that the generated results not only comply with standards and regulations but also make adaptive adjustments based on actual conditions.

In practical applications, the system integrates real-time sensor data, regulatory standards, and historical event records to assess the state of gas in the coal mine using RAG technology, generating corresponding response measures. Because RAG technology enhances the accuracy of model reasoning based on the retrieved relevant data, the reports generated by the system not only comply with current standards but can also be adjusted according to the actual conditions on-site. Finally, the system produces concise and clear reports in natural language, ensuring that coal mine operators can quickly understand and implement the necessary actions.

The design, which combines vector databases, RAG technology, and a knowledge base with real-time information updates, significantly enhances the system's response speed and reasoning accuracy. This ensures that the coal mine gas monitoring system can provide timely and accurate alerts and recommendations for handling issues in complex environments.

4.2. Fine-Tuning of the Large Language Model Phase

In the final stage of the coal mine gas intelligent monitoring report generation module, the gas alarm data and related information collected on-site are uploaded to the local database. Utilizing the multi-agent structure of LLMs and a hierarchical management mechanism, detailed judgment reports are generated rapidly. This process involves not only basic analysis of gas data but also in-depth interpretation of coal mine safety conditions and identification of potential risks. The generated reports are subsequently submitted to an expert team for rigorous review to ensure their accuracy, depth, and reliability.

4.2.1. Selection of the Base Large Language Model

The base model LLM chosen is Generative Language Model 4 Vision (GLM-4v) from the Generative Language Model (GLM) series, which has strong capabilities in Chinese text understanding and generation. The GLM model itself utilizes the Transformer architecture and is pre-trained on a large-scale multilingual corpus. To adapt the model for coal mine risk assessment and alarm generation tasks, fine-tuning on domain-specific data is necessary to enhance its ability to process coal mine gas monitoring data. The choice of the GLM series is based on its excellent performance in generation and reasoning tasks, as well as its good scalability, support for multi-task learning, and capability to handle high-dimensional text data.

4.2.2. Dataset Construction

To effectively fine-tune the GLM, it is first necessary to prepare a high-quality coal mine gas risk assessment dataset. This dataset includes gas concentration data collected from multiple sensors, such as methane and carbon monoxide, in conjunction with relevant coal mine safety regulations and alarm determination standards. After data collection, the data undergoes cleaning and labeling to eliminate invalid or erroneous entries and to anonymize sensitive information. Subsequently, based on coal mine safety standards and alarm determination rules, each data entry is labeled with the corresponding alarm level and recommended emergency response measures. These labeled data will serve as key inputs for model training. In terms of the final dataset format, Json is adopted to ensure structured storage and efficient retrieval of data. Each data record consists of instructions (task description), inputs (sensor data), and outputs (alarm level and emergency response recommendations). An example of the dataset is shown in Table 2. The key objective is to ensure the dataset's high quality and relevance, enabling it to effectively support the fine-tuning process of the GLM, thereby enhancing the model's performance in coal mine gas risk assessment.

4.2.3. Fine-Tuning

LoRA (Low-Rank Adaptation) [41] introduces additional training parameters to enhance the task adaptability of models without significantly increasing computational overhead. In LoRA, the weight matrix of the original LLM is decomposed into two smaller matrices, and the parameters of these low-rank matrices are adjusted through training. Specifically, let the weight matrix of the base model be $W\in {ℝ}^{d\times h}$, where $d$ represents the input feature dimension and $h$ denotes the hidden layer dimension. LoRA decomposes it as follows:

$$W=W_0+\Delta W=W_0+AB,$$

(1)

Let $W_0$ represent the original weight matrix of the base model, $A\in {ℝ}^{d\times r}$ and $B\in {ℝ}^{r\times h}$ denote the adaptation matrices introduced by LoRA, and $r$ is the rank. By fine-tuning these low-rank matrices, LoRA can effectively improve the model's performance on specific tasks without the need to retrain the entire LLM. Therefore, the introduction of LoRA significantly enhances the model's accuracy in gas risk assessment, particularly in the integrated analysis of multi-dimensional data such as carbon monoxide and carbon dioxide concentrations. QLoRA [42] further extends LoRA by employing quantization techniques to reduce memory and computational overhead. In QLoRA, the low-rank matrices $A$ and $B$ are quantized, reducing the required storage space and computational resources. The quantization process compresses the parameter values into lower-precision representations, thereby enabling more efficient model training and inference. The mathematical representation of QLoRA is:

$$A_q=Q\left(A\right),B_q=Q\left(B\right),$$

(2)

Here, $Q\left(\cdot \right)$ denotes the quantization operation, which typically uses 8-bit or lower bit-width integer representations. Through this approach, QLoRA not only reduces the model's storage requirements but also accelerates the inference process, enabling the model to run efficiently on resource-constrained edge computing devices.

4.2.4. Testing and Optimization

The fine-tuned GLM model will be evaluated using a validation set to assess its performance. To ensure its accuracy in coal mine gas monitoring, the model will be evaluated across multiple metrics, particularly accuracy, recall, and F1-score. During the testing process, special attention will be given to the model's accuracy in determining different alarm levels, ensuring its ability to correctly distinguish between normal and alarm states.

Furthermore, the model will be optimized through data augmentation techniques. By generating synthetic data (e.g., simulating sensor outputs under varying gas concentrations), the training dataset size will be increased, thereby further enhancing the model's robustness and adaptability.

4.3. Intelligent Judgment Report Generation Module

In the final stage of the coal mine gas intelligent monitoring report generation module, the gas alarm data and related information collected on-site are uploaded to the local database. Utilizing the multi-agent structure of LLMs and a hierarchical management mechanism, detailed judgment reports are generated rapidly. This process involves not only basic analysis of gas data but also in-depth interpretation of coal mine safety conditions and identification of potential risks. The generated reports are subsequently submitted to an expert team for rigorous review to ensure their accuracy, depth, and reliability.

4.3.1. Report Automatic Generation Process

The intelligent judgment report generation achieves an efficient and automated report generation process through the multi-agent structure of LLMs. This module consists of three main levels: the alarm judgment layer, the measure suggestion layer, and the weekly report generation layer. Each layer is composed of several key agents that work collaboratively to complete different tasks. Figure 4 illustrates the main roadmap of the report generation module.

• Alarm Judgment Layer;

In the coal mine gas monitoring system, the main responsibility of the alarm judgment layer is to analyze the real-time monitored gas concentration data and determine whether it exceeds safety standards, thereby triggering the alarm mechanism. This layer is designed to include two key agents:

• Gas Standard Retrieval Agent.

The task of this agent is to retrieve and obtain the current gas concentration standards set by safety regulations for coal mine operations from the vector database and extract the safety thresholds, providing a reference for the alarm judgment process.

• Alarm Judgment Agent.

This agent is responsible for receiving the safety standards provided by the gas standard retrieval agent and combining them with real-time monitoring data to assess whether there is an exceedance. If the detected gas concentration exceeds the preset safety threshold, the alarm judgment agent will immediately trigger an alarm. Meanwhile, the system will log the relevant alarm information for subsequent processing and review.

Through the collaborative work of two agents, the alarm judgment layer can identify the current status of the sensors and detect any abnormal data based on the real-time sensor data collected. This enables quick assessment of potential safety hazards in the mine environment and triggers the corresponding emergency response mechanisms.

2.

Measures Suggestion Layer;

The core objective of the recommendation layer is to provide effective emergency response solutions to on-site staff in the event of abnormal sensor status detection, ensuring that the correct actions are taken promptly to reduce the likelihood of accidents and potential losses. This layer operates through the collaboration of two key agents:

• Alarm Response Method Retrieval Agent.

Upon receiving an alarm signal, this agent quickly retrieves solutions related to the current alarm situation by accessing the knowledge base, which includes gas handling protocols and safety operation procedures from the vector database.

• Solution Measures Agent.

This agent generates personalized solution recommendations based on the information provided by the alarm response method retrieval agent, in conjunction with real-time data from the site. These recommendations may include specific actions such as adjusting the ventilation system, evacuating personnel, or initiating emergency rescue operations.

Through the collaborative efforts of these two agents, the measures suggestion layer can provide precise and timely operational guidance to coal mine personnel. This design significantly enhances the system's emergency response capability, providing technical support for the stable operation of coal mine safety and helping to reduce the risk of accidents caused by gas exceedance.

3.

Weekly Report Generation Layer.

The primary function of the weekly report generation layer is to convert coal mine safety monitoring data into structured weekly report information, assisting management in timely understanding the monitoring status of gas concentrations and making informed decisions. This layer consists of four key agents:

• Historical Alarm Data Retrieval Agent.

This agent focuses on retrieving historical alarm data related to gas exceedances from the relational database. By executing SQL queries, it extracts records associated with specific gas exceedance events from structured data tables. This approach efficiently processes large volumes of structured data, ensuring the accuracy and consistency of query results.

• Exceedance and Major Safety Risk Trend Data Retrieval Agent.

This agent is responsible for retrieving trend data on exceedances and significant safety risks from the vector database over the past year. Utilizing the vector database's efficient similarity search capabilities, it extracts changes related to gas exceedances and safety risks from a large set of multidimensional data. This agent can identify and analyze long-term trends and anomalies, providing deeper insights and forecasts.

• Chart Generation Agent.

This agent utilizes data visualization tools to process the data extracted by the data retrieval agents and generate corresponding statistical charts. In particular, the chart generation agent presents the concentration trends of methane and carbon monoxide as line graphs, helping management visually understand the dynamic changes in gas exceedances.

• Weekly Report Generation Agent.

This agent is the core module of this layer. It integrates data obtained from the retrieval agents and charts produced by the chart generation agent, along with other relevant information, to automatically generate a weekly report formatted to include the date, statistical data, alarm records, and trend charts. This agent achieves a high level of automation and standardization in the report generation process, significantly enhancing the efficiency of coal mine management and the standardization of reports.

The weekly report generation layer achieves the fully automated transformation of monitoring data into report information through the close collaboration of four agents: historical alarm data retrieval, exceedance and major safety risk trend data retrieval, chart generation, and weekly report generation. This design ensures effective analysis and timely presentation of coal mine safety production data, enabling management to make informed decisions based on accurate and detailed monitoring data.

Through the hierarchical design of the alarm judgment layer, measures suggestion layer, and weekly report generation layer, the system realizes end-to-end automated report generation for coal mine safety monitoring. This multi-layer structure ensures a comprehensive processing flow from data collection, alarm judgment, and emergency measures suggestions to weekly report generation, significantly enhancing the response speed, quality of response, and decision-making efficiency of coal mine managers regarding gas exceedance events. This process allows management to regularly receive detailed and visualized safety monitoring reports, providing reliable data support and a decision-making basis for safe coal mine operations.

4.3.2. Report Evaluation and Review

In this section, we will evaluate and review the intelligent judgment reports generated within the IGRARG framework.

The generated reports will be submitted to a panel of experts for scoring and review to ensure accuracy, depth, and reliability. This panel, consisting of experienced coal mine safety experts and big data analysts, will assess the reports based on a comprehensive set of evaluation criteria. These criteria include, but are not limited to: data accuracy, depth of analysis, readability and clarity of the report, and overall completeness. Only reports with a total score exceeding 80% will be considered qualified and recommended to the corresponding coal mine management team.

To enhance the fairness and transparency of the evaluation process, the system has introduced a cross-review mechanism. Each report is scored not only by one review panel but also randomly assigned to other panels for blind review. The scores from different panels will be weighted based on their expertise to calculate the final total score, ensuring the objectivity of the evaluation results and minimizing potential biases from a single panel. Additionally, all access to the reports will be logged for future tracking and auditing purposes.

At the same time, to assess the quality of the text generated by the model, we introduce checks for grammatical accuracy and text diversity. The former focuses on evaluating sentence structure, word usage, punctuation, and tense, which are fundamental to ensuring the readability and accuracy of the text. The latter reflects the richness of the generated vocabulary and is typically measured by calculating the proportion of unique words or the distribution of vocabulary within the text. This evaluation helps determine whether the generation model avoids repetition or simplistic word choices, producing diverse and expressive language.

Grammatical accuracy is assessed by counting the number of errors and calculating the error rate. A common formula used is:

$$Error Rate={\displaystyle \frac{Number of Errors}{Total Number of Words or Sentences}},$$

(3)

Text diversity is measured by the proportion of different words in the text, which can be expressed using the Type-Token Ratio (TTR) in Equation (4).

$$TTR={\displaystyle \frac{Number of Unique Words\left(Types\right)}{Total Number of Words\left(Tokens\right)}},$$

(4)

Shannon Entropy is a measure based on information theory, used to assess the distribution of words in a text. A higher entropy value indicates a more even usage of words and greater lexical richness. The formula is:

$$H\left(X\right)=-{{\displaystyle \sum }}_{i=1}^{n}p\left(x_i\right)logp\left(x_i\right),$$

(5)

Here, $H\left(X\right)$ represents the entropy value of the text, $p\left(x_i\right)$ represents the probability of the word $x_i$ appearing in the text, which is the number of occurrences of the word divided by the total number of words, and $\mathrm{n}$ denotes the total number of distinct words in the text.

To achieve a more comprehensive evaluation, we reference the text automatic evaluation framework BERTScore proposed by Tianyi Zhang et al [43]. Unlike traditional metrics like BLEU and ROUGE, BERTScore evaluates by calculating the similarity score between each token in the candidate sentence and each token in the reference sentence. It leverages contextual embeddings to compute token similarity rather than relying on exact matches. Additionally, BERTScore introduces the $F_1$ score as a core metric. The$F_1$ score is the harmonic mean of Context Recall and Context Precision, integrating accuracy and recall into the system's performance. The formula for calculating the F1 score is as follows:

$$F_1=2\times {\displaystyle \frac{Context Recall\times Context Precision}{Context Recall+Context Precision}},$$

(6)

Through this series of stringent processes and safety measures, we ensure the quality and security of coal mine safety monitoring reports, providing reliable and easily accessible information resources for coal mine safety management. This not only enhances safety and efficiency at the coal mining site but also offers valuable experience and data to support future projects. This evaluation and feedback mechanism guarantees the effectiveness and reliability of our IGRARG framework in practice, bringing innovation and progress to the coal mine safety management industry.

5. Experiment Results and Analysis

5.1. Experimental Platform and Model Configuration

In this study, the proposed framework was validated using Python on the VS Code platform. The experiments were conducted with Python 3.11.5 as the primary language, relying heavily on open-source libraries for data processing and experiment workflow management. To ensure the compatibility and version consistency of the dependencies, the Python environment was managed using Anaconda. It is important to note that all testing and validation in the experiments were carried out within a Chinese semantic environment.

Data processing and storage in the experiments were primarily conducted in a local environment, with data efficiently retrieved via a vector database. The vector database implementation utilized the Chroma library. A stable network connection was required in the experimental setup to facilitate the local invocation of the fine-tuned GLM model's API and generate inference results.

During the experimental process, we selected the fine-tuned GLM for experimentation and analysis. To manage API calls and vector database retrievals, the LangChain library was utilized. Text blocks were embedded using OpenAI's textembedding-ada-002 model. The experiments demonstrated that splitting the report into blocks of 500 characters (with an overlap of 20 characters between blocks) achieved optimal retrieval performance. During the retrieval process, we typically extracted the top 20 relevant blocks from the RE module. If the prompt length becomes too long (e.g., exceeding 4000 tokens) after inserting the retrieved blocks, we gradually remove the least relevant blocks until the prompt length fits within the context window.

In the LangGraph framework, LLMs are applied by using multiple agents to handle different tasks, with each agent responsible for specific functions such as gas standard retrieval, alarm judgment, and emergency response suggestions. This approach enhances both the efficiency and accuracy of report generation. Additionally, by integrating LLMs with the database through the LangGraph framework, real-time processing of coal mine gas monitoring data and corresponding report generation is ensured. Finally, the system's performance is monitored in real time, with feedback collected to optimize the system settings and report generation process, ensuring that the generated reports meet practical requirements.

5.2. Experimental Results Analysis

To conduct an in-depth analysis of the reports, we selected typical coal mine alarm judgments, focusing on three aspects: the different sensor installation positions, the maximum concentration values, and the processing results. We then compared the expert-generated judgment reports for these alarm events with those generated by the fine-tuned GLM and the non-fine-tuned GLM. As shown in Figure 5, Figure 6, Figure 7 and Figure 8, when generating automated judgment reports using the fine-tuned GLM and the non-fine-tuned GLM, we highlighted the key information in the detection data using bold font.

To evaluate the quality of the automatically generated judgment reports, we annotated the key information within the reports. Correct information was highlighted in green, while incorrect information was marked in red. Manually generated reports typically require several hours to complete, while the report generation speed of the LLM was significantly improved. Table 3 and Table 4 compare the alarm judgment and suggested actions generated by the two models under different evaluation metrics. The analysis shows that the reports generated by the LLM are more comprehensive than those manually written, although they exhibit some shortcomings in terms of details. The fine-tuned GLM slightly outperformed the non-fine-tuned GLM in terms of text generation accuracy and overall performance, though it had a slower processing speed. The non-fine-tuned GLM processed faster and was more efficient, making it suitable for scenarios that require rapid results, but it scored slightly lower on text similarity metrics. The comparison of reports generated by both GLMs indicates that, even with fine-tuning, the GLM did not show a significant advantage over the non-fine-tuned version in terms of output quality, but it demonstrated better logical organization and answer structuring. This analysis suggests that, in more complex scenarios, general LLMs exhibit similar performance regardless of fine-tuning.

Next, we initiated the report review process by examining the reports generated by the fine-tuned GLM. Three evaluation teams, named A, B, and C, were identified. Their experience weight indices were 45%, 35%, and 20%, respectively. Each review team consisted of five reviewers who scored each report based on five different metrics. Figure 9, Figure 10 and Figure 11 display the score distributions for each team across the three reports. By weighting the scores of the three review teams according to their experience indices, the overall weighted scores for the three reports were calculated, as shown in Figure 12. The alarm range judgment score was 80.24, the action suggestion generation score was 80.76, and the weekly report generation score was 81.83. Therefore, all three judgment reports scored above 80 points.

Certainly, when compared to reports written by professional coal mine safety engineers, these reports still have some shortcomings. These deficiencies may be attributed to the model not being fine-tuned with an industry-specific dataset. Nevertheless, these results support the notion that automatically generated judgment reports not only have significant potential but can meet, and in some areas possibly exceed, predefined safety report writing standards. This highlights the potential of automated solutions to enhance the efficiency and accuracy of coal mine safety management processes.

Therefore, it can be reasonably inferred that the integration of data processing methods such as RAG technology with advanced LLM multi-agent systems like GLM represents a significant advancement in the field of mine safety management. Our experimental study emphasizes the feasibility of using such technologies to optimize the efficiency and effectiveness of the judgment process, marking a major improvement over traditional manual methods.

5.3. Discussion

In comparison with traditional methods, IGRARG has demonstrated significant advancements in several key areas. Firstly, by automating the process, IGRARG significantly reduces the labor and time required for traditional manual judgments, thereby greatly improving work efficiency. Secondly, the reduction of human error is another important advantage. Compared to manual judgment, which is prone to subjective biases, the automation in IGRARG substantially lowers the error rate, ensuring consistency and accuracy in the reports. Additionally, IGRARG also shows significant improvements in scalability and applicability, making it adaptable to diverse scenarios and conditions.

However, these improvements also bring new challenges. Firstly, the reliance on high-quality training data means that any flaws in the data can potentially affect the model's performance. During the fine-tuning process, the model's performance is highly dependent on the quality and diversity of the dataset. Therefore, inconsistencies or incomplete parts of the data may lead to biased training outcomes. Secondly, the use of a single model may overlook certain key data points. This issue can be mitigated by adopting a multi-model parallel strategy and enhancing coordination between models. The fine-tuned GLM model, by integrating a specific coal mine risk judgment dataset, improves accuracy and robustness in this domain. However, in complex scenarios, the collaborative work of multiple models can further enhance overall performance, ensuring that the system can better handle various real-world situations.

6. Conclusions

The IGRARG framework proposed in this study aims to achieve the automatic generation of coal mine gas monitoring judgment reports, marking a significant advancement in coal mine production technology, especially in improving production safety quality and management efficiency. This framework integrates the coal mine scene data collection and processing module with the intelligent report generation module, relying on high-quality data collected in real-time by sensors to ensure that the system can make accurate alarm judgments and emergency response suggestions based on the latest on-site conditions. Compared to traditional manual judgment methods, the IGRARG framework significantly improves the efficiency and accuracy of report generation, solving the inefficiencies and inaccuracies in traditional methods and providing a more reliable and automated solution.

In the data processing stage, the fine-tuned LLM based on specific datasets in the coal mine gas monitoring field effectively improves the system's performance in tasks such as alarm judgment and action suggestion, better adapting it to the practical needs of the coal mine industry and providing efficient and precise decision support. Through fine-tuning, the performance gaps in specific scenarios are reduced, further enhancing the system's reliability and robustness in practical applications.

In future research, we plan to conduct domain-specific fine-tuning of the framework's dataset and further test it in real-world applications to evaluate its performance in different coal mine scenarios. Specifically, we will explore the adaptability and performance of LLMs in various operating environments to ensure the practical value of the IGRARG framework in coal mine safety management. By continuously optimizing the framework, we hope to fully leverage the potential of LLMs to provide more robust support for coal mine safety monitoring and management.