Submitted:
09 October 2026
Posted:
10 October 2026
You are already at the latest version
Abstract
Task ontologies are core knowledge carriers for intelligent decision-making and multi-agent collaboration, yet traditional construction approaches suffer from low automation, inconsistent hierarchical semantics, insufficient relational modeling, and poor dynamic adaptability. To address these limitations, this study proposes a large language model (LLM)-driven automatic construction method for hierarchical task ontologies. Built upon domain corpus modeling, the framework integrates hierarchical feature extraction, dynamic weight clustering, and event gap identification to support end-to-end ontology generation and incremental updating. Comprehensive experiments on emergency rescue scenarios were conducted using three mainstream LLMs: Deepseek, Doubao, and ERNIE Bot. Quantitative results demonstrate that the proposed method yields superior F1-scores in feature extraction and overall ontology quality compared with classic BiLSTM-CRF and baseline ablation models. Consistent performance across multiple LLMs verifies the framework’s robustness, while the optimal results achieved by Doubao further validate the effectiveness and strong generalization capability of the proposed LLM-driven ontology construction paradigm.
Keywords:
large language models
; automatic task ontology construction
; hierarchical feature extraction
; dynamic weight hierarchical clustering
; ontology dynamic update mechanism
1. Introduction
As a conceptual model abstracted from the real world, ontology is a core technical foundation for knowledge representation and reasoning, playing a key role in intelligent retrieval, the semantic web, and intelligent collaboration. As a special form of ontology, task ontology takes tasks as cognitive units, focusing on the basic attributes, execution activities, participating roles, process dependencies, and hierarchical relationships of tasks. It is different from static conceptual ontologies that emphasize static concept-attribute correlations and event ontologies that describe the dynamic behavior of individual events. The core feature of task ontology lies in the coordination and cooperation of multiple agents through the basic task process to achieve a common task goal. As a structured knowledge carrier supporting task scheduling and intelligent decision-making, task ontology has irreplaceable value in scenarios such as software project management, intelligent manufacturing, and government services. A structured task ontology can provide structured task knowledge support for upper-level task scheduling and intelligent decision-making in these key domains.
With the diversification and dynamization of task types in complex scenarios, the demand for large-scale, highly adaptable task ontologies has increased sharply. However, existing task ontology construction methods face three core bottlenecks: first, heavy reliance on manual work, domain experts are required to manually sort out task processes and element relationships during the construction process, which not only consumes a lot of labor and time costs but also easily leads to inconsistencies in the ontology logic system due to differences in experts’ subjective cognition; second, significant limitations of semi-automatic construction technologies, although existing methods introduce rule templates or shallow statistical learning to reduce manual intervention, key links still require manual involvement, and their cross-domain generalization ability is weak; third, insufficient dynamic adaptability, lacking of effective automatic update mechanisms, making it difficult to adapt to new task types or process changes in the domain, resulting in the loss of ontology timeliness.
The rapid development of large language models (LLMs) represents a paradigm shift in ontology construction. It has fundamentally changed how knowledge is extracted and represented from unstructured text. LLMs serve as three core engines that drive the entire ontology construction process:
- 1.
- Semantic Understanding Engine: LLMs enable deep semantic understanding of unstructured text, allowing us to extract complex task elements and relationships that cannot be captured by traditional statistical or rule-based methods.
- 2.
- Semantic Reasoning Engine: LLMs enable reasoning about implicit task dependencies, collaboration patterns, and hierarchical structures. This is critical for task ontology construction, which requires understanding not just what is explicitly stated in the text, but also what is implied.
- 3.
- Knowledge Integration Engine: LLMs enable incremental integration of new task knowledge into the existing ontology. This allows our framework to adapt to evolving domain requirements without full reconstruction.
Through technologies such as prompt engineering and chain-of-thought prompting, we can further optimize the performance of LLMs in these three roles. Relying on the massive task knowledge accumulated during the pre-training phase, LLMs can automatically extract multi-dimensional features such as task subjects, goals, and environments from unstructured texts, reason about complex process relationships, and quickly integrate new task knowledge through incremental learning. This provides a solid foundation for the full-process automatic construction of task ontologies.
Based on this, this study designs an LLM-driven automatic task ontology construction method. Through multi-module collaboration and core algorithm innovation, this method realizes end-to-end automatic generation from domain texts to task ontologies. The core innovations are reflected in the following four aspects:
- 1.
- To address the lack of a systematic solution for task ontology construction, a full-process automatic construction framework assisted by LLMs is designed, integrating prompt engineering, chain-of-thought prompting, and few-shot learning technologies. Through the design of auxiliary algorithms, the construction and dynamic maintenance of domain task ontologies are finally realized without manual annotation or expert knowledge.
- 2.
- To solve the problems of incomplete task element extraction and disconnection between abstract and concrete features, a bottom-up hierarchical element extraction mechanism is proposed. First, basic attributes such as task subjects and time are extracted, and then abstract features such as task activities and information requirements are refined through semantic clustering. A cross-level mapping relationship is established to complete the accurate extraction of task elements.
- 3.
- To tackle the disconnection between the task hierarchical structure and domain task execution logic, a dynamic weight hierarchical clustering algorithm based on the semantic similarity of task goals is designed. Low-level clustering focuses on the similarity of task goals and activities, while high-level clustering focuses on task types and domain commonalities, ensuring that the hierarchical division at different levels conforms to the actual execution logic of tasks, thereby forming a referenceable task ontology hierarchical structure.
- 4.
- To solve the insufficient dynamism of task ontologies, an event gap identification and incremental update mechanism is constructed. The update process is triggered through semantic similarity judgment, and new task knowledge is automatically integrated to ensure the timeliness of the ontology.
Experiments are conducted in the field of emergency disaster rescue to verify the advantages of the method in construction efficiency, accuracy, and dynamism, providing a reusable solution for task ontology construction in dynamic knowledge scenarios.
The remaining parts of the paper are structured as follows: Section 2 reviews related work on task ontology construction. Section 3 elaborates on the methodological framework and core modules of the proposed method. Section 4 presents implementation details and core algorithms, along with a running example. Section 5 reports the experimental setup, results and analysis, including cross-domain generalization validation. Section 6 concludes the study and outlines future work.
2. Related Work
To accurately describe the knowledge structure and dynamic evolution laws of the real world, ontologies are divided into three core types: conceptual ontologies, event ontologies, and task ontologies. As core components of knowledge engineering, these three types of ontologies present a hierarchical progressive relationship of basic support-dynamic extension-coordinated complementarity, jointly constructing a complete knowledge system from static concept definition to dynamic behavior modeling [1]. Their internal logic can be summarized as follows: conceptual ontologies provide a bottom-level semantic framework for the entire knowledge system; event ontologies and task ontologies are dynamic extensions based on this framework, focusing on objective process description and goal-driven behavior respectively, and forming a cross-complementary synergy in the modeling dimension.
2.1. Foundations of Task Ontology in Knowledge Engineering
The concept of task ontology traces its roots back to the KADS methodology for knowledge engineering [2], which first introduced the idea of modeling expertise at multiple levels of abstraction. This foundational work inspired the development of early enterprise ontologies such as the TOVE (Toronto Virtual Enterprise) ontology [3], which provided a formal representation of activities, resources, and organizational structures.
Subsequent developments in semantic web technologies led to the creation of OWL-S [4], an ontology for describing web services that introduced the process model as a core component for representing task execution flows. More recently, the Pattern Ontology (PATO) [5] has provided a framework for designing reusable ontology patterns, which has influenced the modular design of modern task ontologies.
With the rise of large language models, new approaches to ontology construction have emerged. SPIRES (Structured Prompt Interrogation and Recursive Extraction of Semantics) [6] and OntoGPT [7] have demonstrated how LLMs can be used to automatically populate ontologies from unstructured text, addressing the long-standing bottleneck of manual knowledge engineering.
As a standardized carrier of domain knowledge, conceptual ontologies systematically define core concepts, concept attributes, and semantic relationships between concepts, providing a unified semantic benchmark for event ontologies and task ontologies. Whether it is the dynamic evolution process described by event ontologies or the goal-oriented behavior focused on by task ontologies, their components need to take predefined concepts in conceptual ontologies as semantic anchors to ensure semantic consistency and interoperability between different ontology modules [8].
Event ontologies take event instances as core modeling units, focusing on the objective recording of dynamically occurring processes and laws in the domain. The core of their modeling lies in clarifying the trigger conditions, participating roles, time sequences, and causal relationships of events [9]. The semantic definition of these descriptive elements relies on the predefinition of noun instance concepts in conceptual ontologies. The event type system constructed by event ontologies can not only provide a trigger context for task ontologies but also provide an important basis for the class hierarchy division of task ontologies.
Task ontologies take specific task processes as core modeling units, focusing on human or system behaviors with clear goal orientation. Their core lies in defining the expected goals, execution subjects, time dimensions, environmental constraints, execution processes, and resource requirements of tasks [10]. From the perspective of semantic foundation, the core components of tasks need to undergo semantic calibration based on conceptual ontologies. From the perspective of process design, the execution logic of tasks often draws on the modeling ideas of temporal relationships in event ontologies. The objective processes described by event ontologies can supplement and support the execution processes of task ontologies. The goal orientation is the core feature that distinguishes task ontologies from the objective process nature of event ontologies: task ontologies emphasize the coordination of multi-agent behaviors and the planning of orderly steps to achieve preset goals, while event ontologies focus on the objective recording of dynamic changes in things states and their inherent laws [11].
In summary, there is a close relationship between conceptual ontologies, event ontologies, and task ontologies: conceptual ontologies provide a unified semantic foundation, event ontologies provide behavioral process references, and task ontologies add goal and constraint dimensions based on the two. The three work together to realize the comprehensive modeling of domain knowledge from static structures to dynamic behaviors [12].
As an important branch of the knowledge engineering field, task ontology construction technology has always evolved around two core goals: improving automation efficiency and optimizing the depth of process relationship modeling. It has formed a clear development path of manual construction → semi-automatic construction → intelligent construction exploration[13]. The technical paths and method systems in different stages show significant differences, which are essentially processes of gradually improving automation, continuously refining modeling granularity, and constantly enhancing the depth of semantic understanding [14]. The following is a systematic review of the technical characteristics, typical methods, and inherent limitations of each stage.
2.2. Manual Construction Stage of Task Ontologies
In the early stage of task ontology development, due to the lack of effective automated tools and algorithm support, manual construction became the main technical path. The core feature of this stage is high reliance on the professional knowledge and practical experience of domain experts. Construction methods are mainly manual processes such as the skeleton method, prototype method, and seven-step method. Experts need to lead the core work of task class definition, element division, process sorting, and relationship organization throughout the process to ensure that the ontology is highly consistent with the domain knowledge system [15].
Regarding typical cases of manual task ontology construction, Yusuke et al. [16] manually constructed a task ontology using the eight-step method for the field of task-based service navigation systems, adjusted the ontology structure based on roles, and finally formed a task system through manual construction. The ontology construction methodology proposed by Noy et al. [17] also includes a large number of manual intervention links, and the ontology is completed through manual intervention, and experts need to annotate and verify task relationships one by one to ensure the rationality of the task.
Although the above manual construction methods can maximize the logical rigor and domain adaptability of the ontology, relevant studies have clearly pointed out their inherent limitations: first, long construction cycles, the construction of a single domain task ontology often requires several experts to spend months or even years, making it difficult to meet the needs of large-scale and rapid ontology construction; second, insufficient scalability, when new task types emerge or existing processes are adjusted in the domain, experts need to re-sort out the ontology structure and modify it manually, making it impossible to quickly adapt to dynamically changing business scenarios; third, high labor costs, the scarcity of domain experts leads to high construction costs, and differences in experts’ knowledge backgrounds may cause ontology consistency issues [18]; fourth, poor reusability, manually constructed ontologies are often custom-developed for specific scenarios, requiring a lot of reconstruction work during cross-domain migration.
2.3. Semi-Automatic Construction Stage of Task Ontologies
With the continuous expansion of the demand for task ontologies in various domains, the efficiency bottleneck and cost issues of manual construction have become increasingly prominent, promoting research to transition to the semi-automatic construction stage. The core goal of this stage is to reduce the intensity of manual intervention and improve construction efficiency while ensuring ontology quality, forming two typical technical paths.
2.3.1. Rule Template-Based Construction Methods
The core idea of this method is to manually design task trigger word dictionaries and process rule bases. In terms of element extraction, some studies have realized the association of high-frequency feature words of specific task types with task classes, and completed task matching by identifying trigger words in texts [19]; in terms of relationship modeling, relevant studies have annotated explicit relationships such as task decomposition and division of labor through pre-defined regular expressions [20] to complete the formulation of rule templates. Sungmin et al. [21] designed a template library containing 128 rules for the field of intelligent building manufacturing, successfully extracting the execution subjects, steps, and constraint conditions of equipment assembly tasks, with an automatic element annotation accuracy of 78%, realizing the semi-automatic construction of ontologies guided by rule templates. Some scholars designed a standardized processing process for relational databases, extracted the ontology structure from them, and finally formed an ontology construction paradigm based on templates [22]. Pan et al.[23] designed rule templates based on first-order predicate logic to complete the element extraction of outpatient diagnosis and treatment tasks in the medical field, effectively reducing the workload of manual annotation.
However, with the increasing application of ontologies, the inherent defects of such methods have also been revealed: first, strong domain dependence, rule templates need to be jointly designed by domain experts and technicians, and the dictionary and rule base need to be reconstructed during cross-domain migration, resulting in weak generalization ability. Second, insufficient robustness, it is difficult to handle colloquial expressions, ambiguous texts, or implicit logics, and the matching accuracy decreases significantly when task descriptions deviate from the preset format of rules [24]. Third, limited modeling scope, rule template methods can only capture explicit process relationships between tasks, and cannot cover complex relationships such as implicit collaboration and dynamic information interaction. Fourth, high maintenance costs, as domain tasks increase and processes change, the rule base needs to be continuously iteratively updated, making it difficult to adapt to dynamically changing business scenarios.
2.3.2. Shallow Statistical Learning-Based Construction Methods
Shallow statistical learning-based construction methods take conditional random fields (CRF) and basic neural network models as the core, training task extraction models through annotated corpora to realize semi-automatic extraction of elements and relationships. Their core advantage is the ability to automatically learn text features, reducing reliance on manual rules [25]. In terms of technical application, the CRF model proposed by Hao et al. [26] is widely used in task trigger word recognition and element extraction. Tests on the ACE 2005 task dataset show that the F1-score of this model in trigger word recognition and subject extraction is significantly higher than that of other methods and significantly better than rule-based methods; Wang et al. [27] used the BiLSTM-CRF model to realize the detection of subtasks such as sample collection and data analysis in the geological field task ontology, with an F1-score of 79.3%. Scholars constructed a task classifier based on the SVM model and successfully generated abstract summaries of texts [28]. Scholars employed Bidirectional Long Short-Term Memory (BiLSTM) neural networks combined with CRF to detect and extract random event streams from Twitter, yielding favorable results [29].
Although shallow statistical learning methods have improved automation efficiency, they still have some technical shortcomings [30]: first, high reliance on large-scale annotated corpora, the annotation process consumes a lot of labor and material resources, and re-annotation of adaptive data is required during domain migration, resulting in high costs. Second, limited relationship modeling ability, only explicit relationships between tasks can be captured, and implicit collaboration logics or information dependence relationships cannot be identified. Third, insufficient dynamic adaptability, when new task types emerge in the domain, the model needs to be retrained and parameters optimized, making it impossible to achieve rapid updates. Fourth, insufficient depth of feature learning, the model mainly relies on manually designed surface features, making it difficult to capture deep semantic information hidden in texts [31].
2.4. General Ontology Learning from Unstructured Text
Before focusing on task-specific ontology construction, it is important to review the broader field of general ontology learning from unstructured text, which provides the foundational technologies for our LLM-driven approach.
Traditional ontology learning methods rely on statistical techniques and linguistic patterns to extract concepts and relationships from text. These methods typically require large amounts of annotated data and manual feature engineering, which limits their scalability and cross-domain applicability.
With the development of deep learning, neural network-based methods have emerged as the state of the art for ontology learning. Models such as BERT [41] and its variants have demonstrated excellent performance in entity recognition and relationship extraction tasks. However, these methods still require task-specific fine-tuning and do not fully leverage the commonsense knowledge and reasoning capabilities of large language models.
The recent emergence of large language models has revolutionized ontology learning. Two particularly influential approaches are:
- 1.
- SPIRES (Structured Prompt Interrogation and Recursive Extraction of Semantics) [6]: A method that uses structured prompts to guide LLMs in extracting structured information from scientific literature. SPIRES has been successfully used to construct domain-specific ontologies in biology and medicine.
- 2.
- OntoGPT [7]: A framework that integrates LLMs with existing ontologies to enable automatic knowledge extraction and ontology population. OntoGPT supports multiple LLM backends and provides a flexible API for ontology engineering tasks.
These LLM-based methods have demonstrated the ability to extract high-quality ontological knowledge from unstructured text with minimal supervision. However, they are not specifically optimized for task ontology construction, which requires modeling complex process relationships, hierarchical structures, and dynamic execution semantics. Our work builds on these general methods and extends them to address the unique challenges of task ontology construction.
2.5. Exploration of Intelligent Construction of Task Ontologies
With the breakthrough development of deep learning technology in the field of natural language processing, pre-trained language models have provided a new paradigm for task ontology construction, promoting the field to enter the stage of intelligent construction exploration [32]. The core feature of this stage is relying on the semantic understanding and automatic feature learning capabilities of deep learning models to break through the technical bottlenecks of semi-automatic construction and achieve dual improvements in construction efficiency and modeling depth [33].
In terms of technical application, the BERT model applied by Ho et al. [34] provides strong support for task ontology element extraction. Improved models based on BERT have performed well in multiple domains: Gan et al. [35] applied the BERT-CRF model to ship flag state supervision tasks by extracting entities and data from unstructured data; some scholars designed a task relationship modeling model based on the transformer architecture, converting task decomposition relationship prediction into a pre-training task to realize the automatic identification of hierarchical relationships between tasks, with an identification accuracy of 83.7% for electronic medical record recognition tasks, providing an effective method for intelligent task ontology construction [36]. The multi-branch BiGRU-CRF model proposed by Xie et al. [37] has a higher F1-score in entity and attribute extraction than traditional statistical learning methods in the construction of geological disaster field task ontologies, improving the quality and efficiency of task ontology construction.
However, existing studies still have three significant shortcomings: first, lack of systematic design, current methods mostly focus on a single link and have not formed a full-process automatic framework from corpus preprocessing, feature learning, element extraction, relationship modeling to ontology generation, and a lot of manual intervention is still required in subsequent integration links. Second, insufficient semantic depth, element extraction mostly relies on surface field matching, without analyzing the internal correlations between elements from the deep semantic level, without distinguishing between basic attributes and abstract features, resulting in limited semantic expression ability of the ontology. Third, weak dynamic update mechanism, when new task types or process changes emerge in the domain, the model needs to be retrained and the full-process construction needs to be executed, making incremental updates difficult and resulting in the loss of ontology timeliness.
Large language models provide a new path to solve the above problems with their excellent semantic understanding and few-shot learning capabilities. Specifically, in terms of element extraction, Pedram [38] used few-shot prompts to guide GPT-3.5 to complete end-to-end extraction of the three elements of subject-goal-environment; in terms of relationship modeling, scholars proposed a chain-of-thought prompting method, making full use of the reasoning ability of LLMs to guide the model to reason about task collaboration and information interaction relationships, thereby improving the integrity of relationship modeling in task ontologies [39]; at the same time, scholars designed a task ontology construction framework based on the LLaMA model and BERT model, realizing the automatic integration of some processes [40].
Although the above automatic task ontology construction methods have improved construction efficiency, existing LLM-related studies still have obvious limitations: first, lack of a hierarchical semantic design for element extraction, no effective distinction and cross-level association between basic attributes and abstract features, resulting in the disconnection between abstract features and basic attributes. Second, task hierarchical clustering lacks domain task execution logic guidance, mostly relying on text semantic similarity for clustering, and the hierarchical division is inconsistent with the actual execution process. Third, the dynamic update mechanism is not yet mature, lack of incremental learning solutions for new tasks, making it difficult to adapt to high-frequency changing domain scenarios. Fourth, insufficient controllability of model output, LLMs may generate elements or relationships that do not conform to domain specifications, requiring a lot of manual verification.
While these existing task ontologies have made significant contributions to the field, they are not directly suitable for our purpose for several reasons:
- 1.
- 2.
- Lack of automation support: Traditional ontologies are designed to be manually constructed and maintained, and do not provide mechanisms for automatic population from unstructured text.
- 3.
- Limited dynamic adaptability: Existing ontologies are typically static once created, and do not support incremental updates as new task knowledge emerges.
- 4.
- Inconsistent element frameworks: Different ontologies use different element frameworks for representing tasks, making it difficult to integrate knowledge from multiple sources.
These limitations motivated us to develop a new element framework specifically designed for automatic, domain-agnostic, and dynamically adaptable task ontology construction.
To fill the above research gaps, this study designs a full-process automatic task ontology construction scheme driven by LLMs. By constructing a hierarchical feature extraction mechanism, a dynamic weight clustering algorithm, and an incremental update framework, the shortcomings of existing methods in systematicness, semantic depth, and dynamic adaptability are systematically solved, and the automatic generation of high-quality task ontologies is realized.
3. Methodological Framework of ATOC-LLM
3.1. Fully Automatic Task Ontology Construction Framework
The LLM-driven automatic task ontology construction method proposed in this study includes six core modules: automatic domain text collection and corpus preprocessing, hierarchical task ontology element extraction, task core process relationship modeling, task hierarchical structure construction, OWL ontology generation, and ontology dynamic update. These modules collaborate with each other to realize the full-process automation of task ontologies, and the architecture is shown in Figure 1. The core logic of each module is as follows:
- 1.
-
Module 1: Automatic Domain Text Collection and Corpus Preprocessing.The purpose of this module is to construct a domain corpus database required for task ontology modeling, including structured, semi-structured, and unstructured data. Domain corpora are retrieved through AI Agents, and LLMs are used to evaluate the correlation between the corpora and the target domain, screen them, and store them to ensure the data quality of the constructed task ontology domain database.
- 2.
-
Module 2: Hierarchical Task Ontology Element Extraction.Based on the preprocessed corpora, this module designs a bottom-up hierarchical mechanism to extract task elements from the domain database, considering the inherent logic and different semantic levels between task elements. Specifically, for bottom-level features, basic attributes such as task subjects, task time, task environments, and task goals are extracted. For top-level features, abstract features such as task activities and information requirements are refined from the extracted bottom-level features through semantic clustering, and a cross-level mapping relationship is established to form a bottom-up feature set of the task ontology.
- 3.
-
Module 3: Task Core Process Relationship Modeling.The task of this module is to further conduct semantic mining on the bottom-level and top-level features extracted in Module 2 and analyze four types of key structured relationships in the task ontology: task decomposition relationship (decomposing the entire task into different process stages), task division relationship (corresponding subjects to each task activity based on task decomposition, describing the responsibilities of each subject in the task completion process), task collaboration relationship (the cooperation between various subjects in the task completion process), and task information relationship (the information acquisition and transmission process existing in the task collaboration process). This enables a more accurate description of the collaborative completion logic between various subjects in the dynamic task execution process.
- 4.
-
Module 4: Task Hierarchical Structure Construction.In this module, for the task subunits with different task goals obtained through extraction, semantic vectors of the task subunits are first generated, and a dynamic weight hierarchical clustering algorithm is used to construct a parent task-subtask hierarchical structure based on the semantic similarity of task goals. Different class hierarchies are distinguished, and the weights of different features are dynamically adjusted to ensure that the finally generated task ontology hierarchy conforms to the actual domain task execution logic.
- 5.
-
Module 5: OWL Ontology Generation.The hierarchical structure and core relationships generated in Module 4 are mapped to OWL classes, individuals, and attributes, generating an ontology file that conforms to RDF/OWL specifications, and outputting text reports and visualization results.
- 6.
-
Module 6: Ontology Dynamic Update.In this module, AI Agents complete the task of monitoring new task corpora. When the overall semantic similarity is lower than a preset threshold, the update process is triggered, corpora are automatically crawled, and the automatic task ontology construction process is repeated to realize the incremental update of the task ontology, thereby improving the timeliness and availability of the constructed ontology.
To illustrate how our framework works in practice, we will use a running example from the emergency rescue domain. A detailed step-by-step walkthrough of this example is provided in Section 4.2.
3.2. Steps of the Automatic Task Ontology Construction Method
3.2.1. Module of Automatic Domain Text Collection and Corpus Preprocessing
The purpose of this module is to construct a task ontology domain database, providing task data support for subsequent task ontology element extraction and core relationship extraction. To automatically collect rich task corpus data from different sources, we design the use of AI Agents to complete the automatic collection of domain documents. When constructing the domain database, the target domain of the task ontology to be constructed is first determined, AI Agents are used to complete the preliminary crawling and collection of event data, and at the same time, to improve the quality and availability of the database, the collected data is further screened through task text summary analysis. The text understanding and analysis capabilities of LLMs are used to perform overall summary generation and topic classification of task corpus data, and finally form a task corpus database.
AI Agent Design for Domain Corpus Collection Our multi-agent system consists of three specialized agents that work together in a pipeline:
- 1.
- Retrieval Agent: This agent is responsible for retrieving documents from multiple sources, including academic databases, official websites, and news media. It uses domain-specific keywords and query expansion techniques to ensure comprehensive coverage of the target domain.
- 2.
- Filtering Agent: This agent evaluates the retrieved documents based on three criteria: domain relevance, information completeness, and source authority. It uses an LLM to generate a relevance score for each document and filters out documents that do not meet the threshold.
- 3.
- Preprocessing Agent: This agent processes the filtered documents to extract text segments containing task descriptions. It removes irrelevant content such as advertisements, headers, and footers, and standardizes the format of dates, locations, and organization names.
The agents communicate with each other through a message queue, and the entire pipeline is fully automated, requiring no manual intervention.
In this study, the emergency rescue field is selected as the target domain for the task ontology to be generated when collecting the domain dataset. The collection of domain data needs to comprehensively consider the professionalism, authority, and coverage of data sources to meet the needs of domain task ontology construction. Here, three data sources, academic papers, news media, and official websites, are selected as basic data sources, and intelligent agents can be flexibly guided to select differentiated data sources for different domains. Table 1 shows the construction results of the corpus database constructed by this module, presenting the specific content of the data collected by different LLMs for specific domains. The gold standard was constructed by three domain experts who independently annotated 10 randomly selected documents. Disagreements were resolved through group discussion to form the final reference annotations. The formal description of the workflow of this module is as follows:
1. Multi-Source Data Collection
Let the set of data sources be , where is the academic database, is the official platform, and is the news media. AI Agents collect raw data from each source to form a raw data set: , where represents the raw data subset collected from source s, and each element is a single document .
2. Data Screening
For each raw document , the LLM screens the collected data based on three aspects: domain relevance (Rel), information completeness (Com), and authority (Aut). The data relevance scoring function is defined as:
where T represents the target domain of the task ontology to be generated, represent the weights of different dimensions, represents the domain relevance score, which measures the matching degree between the document data topic and T, represents the information completeness score, which measures whether the document includes all core elements of the task ontology, and measures the authority of the task data source. We elaborate on the weights with the adopted values of , and . These parameters are determined according to the practical characteristics of the task ontology. We assign the highest weight to domain relevance because ensuring that documents are actually about the target domain is the most critical factor for corpus quality. Information completeness and source authority are equally important secondary factors. Notably, varying weight configurations will alter the corpus generation rules and further shape the stylistic features of the final ontology. Accordingly, we adopt this set of weights to achieve a relatively balanced performance.
where is positively correlated with the number of target domain keywords appearing in the document, is positively correlated with the relevance between the document topic and the target domain, M is the total number of task elements obtained through extraction, and is the existence indicator of different elements , which takes a value of 1 if it exists and 0 if it does not exist. To ensure the credibility of the task document sources, has different values for documents from different data sources, and the details are shown in Appendix A. The core elements of the task ontology used in the completeness measure are: subject, time, environment, goal, activity, and information requirement. These elements were identified based on a comprehensive review of existing task ontology research (including TOVE and OWL-S) and interviews with three domain experts in emergency management.
Let the scoring threshold be , and a high-quality corpus database is obtained after screening:
3. Corpus Preprocessing
Irrelevant paragraphs, format symbols, and duplicate content in the documents are removed, and text segments containing task descriptions are extracted through LLMs to reduce the difficulty of subsequent analysis. Table 1 shows the results of the emergency rescue domain corpus database constructed by this module. Taking the earthquake rescue scenario in Section 4.2 as an example, this module filters and retains official disaster reports and rescue operation documents as valid corpus.
3.2.2. Module of Hierarchical Task Ontology Element Extraction
Based on LLMs and clustering analysis, this module adopts a hierarchical extraction mechanism to distinguish and extract bottom-level basic features and top-level abstract features respectively, considering the differences in the feature distribution of task ontology elements, and constructs a structured element set containing multi-dimensional correlation relationships. For the bottom-level features and top-level features in the task ontology, the bottom-level features focus on describing the basic attributes of the task ontology and are easy to extract from texts; the top-level features focus on more abstract elements in the task ontology, such as task activity stages and information requirements, which are integrated and refined from the bottom-level features.
The bottom-level features and top-level features are in a progressive correlation relationship in task ontology element extraction. The bottom-level features are the foundation and basis for the formation of top-level features, and the top-level features are the integration and subsumption of the bottom-level features. By focusing on the basic attributes of the task ontology, the bottom-level features extract specific and directly accessible information from texts, providing raw data support for the top-level features. At the same time, the top-level features integrate and refine the bottom-level features, converting scattered basic attributes into abstract elements with generality. This integration not only endows the bottom-level features with more macro significance but also can reversely guide the extraction direction of the bottom-level features, ensuring that the extraction of bottom-level features is more in line with the core needs of the task ontology, and finally jointly supports the structured element set with multi-dimensional correlations.
Bottom-Level Feature Extraction
Our bottom-level feature framework is grounded in the TOVE ontology [3], which defines the core attributes of organizational tasks. Our top-level feature framework extends the process model from OWL-S [4], which describes the execution flow of web services.
Feature Definition: The bottom-level features are the basic attributes of the task, including a 4-tuple of task subject, task time, task environment, and task goal , where respectively represent the task subject, task time, task environment, and task goal of the i-th task ontology.
Extraction Strategy: Prompt engineering, few-shot learning, and chain-of-thought prompting are used to guide the LLM to extract bottom-level features from the corpora and output them in a structured form. The design of the prompt engineering input is shown in Table 2.
In addition to the base prompt shown in Table 2, we use two additional strategies to improve extraction accuracy:
- 1.
- Few-shot learning: We provide 3–5 examples of correctly extracted task features in the prompt. These examples are selected from a gold standard dataset and cover a variety of task types and domains.
- 2.
- Chain-of-thought prompting: We ask the LLM to explain its reasoning process before generating the final output. For example, the prompt includes: “First, identify all the subjects involved in the task. Then, identify the time and location where the task takes place. Finally, identify the goal of the task." This helps the LLM avoid common errors such as missing features or misclassifying relationships.
Semantic Clustering and Feature Association
After completing the extraction of bottom-level features, it is necessary to convert the extracted bottom-level features into semantic vectors through LLM semantic representation technology to capture the semantic correlations between various features for further analysis of the top-level features of the task ontology. Then, through task clustering, a clustering algorithm is designed with the task goal as the core element to group the bottom-level features, so that features serving the same task goal are clustered into one category to form various task clusters. Finally, feature fusion of the clusters is completed: the bottom-level features of similar task clusters are merged, duplicate attributes are removed, and a set of shared bottom-level feature attributes at the cluster level is generated to provide a basis for the next step of top-level feature extraction. The specific design of the bottom-level feature clustering algorithm is shown in Section 4.1.1.
Top-Level Feature Extraction
Feature Definition: The top-level features are abstract elements of the task, including task activities (Task Activity) and information requirements (Information Need), denoted as a 2-tuple . Together with the aforementioned bottom-level features , they form a set of six elements of the task ontology . After that, scenario-based prompts are designed based on the clustering results, combined with chain-of-thought prompting and prompt engineering to guide the LLM to refine the top-level features from the bottom-level features. The design of the prompt engineering input is shown in Table 3.
Feature Integration
After completing the extraction of top-level features based on the bottom-level feature task clusters, a mapping relationship between bottom-level and top-level features is established, missing elements are supplemented, and an element correlation matrix is generated. The output of the module is a structured task element set , and each task entry includes a unique identifier, a set of bottom-level features, and a set of top-level features under the task.
Taking the earthquake rescue scenario in Section 4.2 as an example, this module extracts core task elements such as rescue subject, time and environment from disaster-related texts.
3.2.3. Module of Task Core Process Relationship Modeling
This module converts task elements into four types of structured relationships to describe the dynamic execution logic of tasks. First, the logic in the extracted six task elements is further decomposed to form four types of structured relationships: task division , task decomposition , task collaboration , and task information relationship . This objectively and accurately reflects the dynamic execution logic of tasks, the cooperation process between task subjects, and the information flow process. The schematic diagram of task core process relationship modeling is shown in Figure 2.
Relationship Definition: First, the core connotations and internal logics of the four types of relationships are clarified. These relationships are grounded in existing task ontology research: Task decomposition and division relationships are derived from the process decomposition model in OWL-S [4], which describes how complex processes are broken down into simpler sub-processes. Task collaboration and information relationships extend the interaction model in the TOVE ontology [3], which models organizational interactions and information flows in enterprise environments.
Specifically: - Task division refers to the different task activities that task subjects are responsible for to achieve the goal. - Task decomposition refers to the relationship of decomposing a task into different sub-activities to achieve the goal. - Task collaboration refers to the cooperation mode between subjects to promote task execution. - Task information relationship refers to the information that subjects need to obtain or transmit during the division of labor and collaboration.
The four types of relationships comprehensively reflect the coordination and cooperation process between various subjects in the task process to achieve a common task goal.
Relationship Generation Strategy: Based on the structured element set of each task subunit, the LLM is used to generate the four types of relationships through prompt engineering, few-shot learning, and chain-of-thought prompting. The design of the prompt engineering input is shown in Table 4.
Prompt Engineering for Relationship Extraction
For relationship extraction, we use a more sophisticated prompt engineering strategy that includes three components:
- 1.
- Clear relationship definitions: We provide explicit definitions of each of the four relationship types in the prompt, along with examples of valid and invalid relationships.
- 2.
- Few-shot examples: We include 2–3 examples of correctly extracted relationships for each type, demonstrating how to identify both explicit and implicit relationships.
- 3.
- Chain-of-thought reasoning: We ask the LLM to first analyze the task execution logic, then identify the relationships between different task elements. This significantly improves the accuracy of extracting complex implicit relationships.
The relationship generation process follows a certain internal logic: first, the overall task process is decomposed to obtain the task decomposition relationship under this task process. Then, based on the role of the task subject in different task processes, the task division relationship of the task subject in the task is generated. Further, based on the task division relationship, the task collaboration relationship generated by the coordination and cooperation of various subjects to complete the task during the division of labor is explored. Finally, the task information requirement relationship generated during the collaboration process is analyzed based on the task collaboration relationship.
Taking the earthquake rescue scenario in Section 4.2 as an example, this module identifies task decomposition, collaboration and information interaction relationships among rescue activities.
3.2.4. Module of Task Hierarchical Structure Construction
Based on the semantic similarity of task goals, this module generates a parent task-subtask hierarchical structure through hierarchical clustering and supports visual display. The specific process is as follows:
Semantic Vector Generation: Features such as task goals and core activities of each task subunit are extracted from the structured task element set, and semantic vectors of the task subunits are generated through LLM semantic representation technology to describe their semantic distribution;
Dynamic Weight Hierarchical Clustering: A hierarchical clustering algorithm based on the semantic similarity of task goals is designed, with the task goal as the core to adjust weights. First, the cosine similarity of the semantic vectors of each task subunit is calculated, and task subunits with more similar semantics have higher clustering priority; second, weights are dynamically adjusted during the clustering process, in which low-level clustering focuses on the similarity of task goals and activities, and high-level clustering focuses on task types and domain commonalities to ensure that the hierarchical structure conforms to the task execution logic.
Taking the earthquake rescue scenario in Section 4.2 as an example, this module builds a layered task hierarchy conforming to the actual execution logic of emergency rescue work.
The algorithm flow of dynamic weight hierarchical clustering is specifically designed in Section 4.1.2.
3.2.5. Module of OWL Ontology Generation
This module integrates the task core relationships and hierarchical structure into an ontology that conforms to OWL specifications, outputs readable reports and usable structured files. The specific process is as follows:
Ontology Mapping: Based on Semantic Web theory, the parent tasks and subtasks in the task hierarchical structure are mapped to OWL classes, task subunits are mapped to individuals of the classes, the division relationships, decomposition relationships, collaboration relationships, and information interaction relationships in the task core relationships are mapped to object properties, and attributes such as time and environment in the task elements are mapped to data properties.
Our mapping process follows the OWL 2 DL specification and uses the following rules:
- 1.
- Task hierarchy mapping: Each task in the hierarchical structure is mapped to an OWL class. The parent-child relationship between tasks is mapped to an rdfs:subClassOf relationship between classes.
- 2.
- Task subunit mapping: Each task subunit is mapped to an OWL individual of the corresponding task class.
- 3.
-
Core relationship mapping:
- Task decomposition: mapped to an hasSubTask object property
- Task division: mapped to a performedBy object property
- Task collaboration: mapped to a collaboratesWith object property
- Information flow: mapped to a requiresInformation object property.
- 4.
- Task attribute mapping: Each task attribute (time, location, etc.) is mapped to an OWL data property.
The resulting ontology is serialized in OWL-Turtle format, which is both human-readable and machine-processable.
Ontology Serialization: The mapped ontology is serialized in OWL-turtle format, following the naming and design specifications of the semantic web to ensure that the ontology file can be parsed and reused by mainstream semantic web tools.
Text Report Generation: A task ontology generation text report is output, including structured information such as task lists, element sets, core relationships, hierarchical structures, and visualization result descriptions, which is convenient for users to view and audit.
User Interaction: Users are prompted to view the ontology file, visualization results, and text report, and the system is supported to automatically open related files. At the same time, a manual adjustment interface is provided to allow users to fine-tune the ontology.
Taking the earthquake rescue scenario in Section 4.2 as an example, this module converts the extracted rescue task knowledge into standard OWL ontology files.
3.2.6. Module of Ontology Dynamic Update
The automatic task ontology update module aims to monitor new task elements in the corpus through a preset cycle or incremental trigger mechanism, and realize the continuous dynamic iteration of the task ontology through element matching, relationship reasoning, and conflict verification, ensuring the timeliness and completeness of the task ontology.
AI Agents are used to monitor new task texts in the domain or existing task change information in real time. When the semantic similarity between the new task corpus data and the existing task ontology is lower than a preset threshold, the update process is triggered. AI Agents automatically crawl the new task corpus, repeat the processes of corpus preprocessing, element extraction, relationship modeling, and hierarchical construction, integrate new task classes or changed relationships into the existing task ontology, and execute incremental update operations to ensure the timeliness of the ontology.
Let the task ontology set be , and the input of the new event is to trigger the ontology update to calculate the maximum similarity between each new task ontology and the existing ontology classes:
where is the center vector of the existing cluster C. When the above maximum similarity is less than the given threshold , the automatic update process is triggered.
Targeted Data Supplement and Incremental Update
When entering the automatic update process, the ontology is automatically updated by incrementally updating the domain database. AI Agents are activated to crawl corpora containing new task elements, and new task classes are added to the cluster set. The local cluster centers and similarities are recalculated:
The task element extraction task is performed, and the aforementioned clustering iteration process is repeated. Finally, a new cluster set is obtained, and the classes, entities, and attributes are remapped. Finally, the updated ontology is generated.
Consistency Verification
The logical consistency of the updated ontology is verified using preset ontology axioms, and if passed, the current ontology dynamic update process is completed.
Taking the earthquake rescue scenario in Section 4.2 as an example, this module supplements new rescue task information and optimizes the existing emergency rescue ontology in real time.
4. Implementation Details and Core Algorithms
4.1. Core Implementation Algorithm Technologies
4.1.1. Bottom-Level Feature Clustering Algorithm
1. Core Goal of the Algorithm For the set of bottom-level features and independent task goals, with the semantic context correlation as the core, the comprehensive correlation score is calculated by integrating the embedding vector similarity. Through initial cluster construction, intra-cluster optimization, and redundant cluster merging, the accurate clustering of bottom-level features to the corresponding task goals is realized. Finally, the bottom-level feature clustering results that conform to the domain task execution logic and include core goals, feature subsets, and similarity indicators are output, providing a basis for further top-level feature extraction.
2. Predefined Definitions and Symbol Descriptions represents the set of task ontology bottom-level features, n is the total number of bottom-level features, and each is a 4-tuple , where respectively represent the participant set, time description, environment description, and task goal description of the i-th task ontology.
represents the set of independent task goals extracted from the bottom-level features, , indicating that multiple features may correspond to the same goal.
represents the semantic embedding vector of the bottom-level feature generated by the LLM, and d is the embedding dimension.
represents the semantic embedding vector of the task goal generated by the LLM.
represents the semantic context correlation between the bottom-level feature and the task goal .
represents the cosine similarity between the embedding vectors and .
represents the comprehensive semantic similarity between the bottom-level feature and the task goal , which integrates the context correlation and cosine similarity.
K represents the number of clustering clusters. represents the set of final clustering results, and each is a subset of bottom-level features belonging to the same task goal. represents the similarity threshold.
3. Formal Steps of the Algorithm
Step 1: Generation of Semantic Embeddings for Bottom-Level Features and Task Goals Based on the semantic representation capability of the LLM, all bottom-level features F and independent task goals G are embedded and encoded: The text description of the bottom-level feature is input, and is generated through the text embedding interface of the LLM; the text description of the task goal is input, and is generated.
Step 2: Calculation of Comprehensive Semantic Similarity For each bottom-level feature and each task goal , the comprehensive semantic similarity is calculated, which integrates the semantic context correlation (the core judgment basis emphasized by the user) and the embedding vector similarity:
where is a weight coefficient, which prioritizes ensuring the core role of context correlation, and its optimal value is determined through cross-validation.
represents the context correlation judgment task performed by the LLM. To ensure full reproducibility of this step, we use a fixed standardized prompt template (presented in Appendix C).
where is the cosine similarity, and the purpose is to standardize the cosine similarity.
Step 3: Initial Cluster Initialization For each independent task goal , an empty cluster is initialized. If a bottom-level feature only satisfies with one task goal , then is assigned to .
If a bottom-level feature satisfies with multiple task goals , then the with the maximum is selected, and is assigned to the corresponding .
Step 4: Cluster Optimization and Deduplication For each initial cluster , the average comprehensive similarity of all features in the cluster is calculated:
If , the cluster is split: features in the cluster with are screened out, and Steps 2–3 are re-executed to determine whether to assign them to other clusters or form new clusters;
Merge redundant clusters: if the task goals and corresponding to two clusters and satisfy (similar semantic similarity), then and are merged, and the goal of the new cluster is the semantic fusion description of and .
Step 5: Output Clustering Results Finally, the cluster set is output, and each cluster includes: Cluster ID , core task goal , subset of bottom-level features in the cluster , and average comprehensive similarity in the cluster .
4.1.2. Task Subunit Hierarchical Clustering Algorithm Based on Dynamic Weights
1. Core Goal of the Algorithm
For the set of task subunits, based on the hierarchical characteristics of domain task execution logic, which refers to the actual processes, rules and dependencies that tasks follow when executed in a specific domain, the weights of the four types of features (task goal, task activity, task type, and domain commonality) are dynamically adjusted: low-level clustering focuses on the similarity of specific features, and high-level clustering focuses on the similarity of common features. Finally, a task hierarchical structure that conforms to the domain task execution logic is generated.
2. Predefined Definitions and Symbol Descriptions
represents the set of task subunits, p is the total number of task subunits, and each is a 4-tuple , where: represents the task goal of the q-th subunit, represents the core activity set of the q-th subunit, represents the task type of the q-th subunit, and represents the domain to which the q-th subunit belongs.
represents the four feature dimensions of the task subunit, corresponding to goal, activity, type, and domain respectively.
represents the semantic embedding vector of the task subunit in the feature dimension , .
l represents the clustering level variable, represents the lowest level, represents the highest level, and L is the maximum level.
represents the weight of the feature dimension during clustering at level l, satisfying .
represents the weighted semantic similarity between task subunits and during clustering at level l.
represents the cluster merging operation, merging the two clusters and with the highest similarity into a new cluster .
represents the final task hierarchical structure, and represents the set of task clusters at level l.
3. Formal Steps of the Algorithm
Step 1: Generation of Task Subunit Feature Embeddings For the four feature dimensions of each task subunit , different semantic embeddings are generated through the LLM:
- Goal embedding : generated by inputting the text description of ; - Activity embedding : generated based on the input text of the activity sequence of ; - Type embedding : generated by inputting the text of ; - Domain embedding : generated by inputting the text of .
Step 2: Definition of Dynamic Weight Function The weights of the four types of features are adjusted according to the clustering level variable l. The core logic is reflected in: the lower the level, the more the weight tilts to specific features (such as task goals, task activities); the higher the level, the more the weight tilts to common features (task types, task domains). This dynamic weighting scheme is grounded in cognitive hierarchy theory: when humans process tasks at a fine-grained level, they prioritize concrete goals and operational actions; at a coarse-grained level, they categorize tasks by domain type and functional commonality. The weight function is defined as follows:
Two boundary cases of levels are shown as follows: When (at the lowest level), , , , . When (at the highest level), , , , .
is the level proportional coefficient, which ensures that the weights are adjusted linearly and smoothly with the level.
Step 3: Initial Level Initialization (l = 1) Each task subunit is regarded as an initial cluster, that is, , where .
The feature embedding of the initial cluster is calculated: for each , its embedding in the feature dimension is .
Step 4: Iterative Hierarchical Clustering (l = 1 to l = L) For each level l, the following operations are performed:
1. Calculation of Weighted Semantic Similarity Between Clusters
For any two clusters and at level , the cosine similarity in each feature dimension is first calculated:
Then, the comprehensive similarity between clusters is obtained through dynamic weight weighting:
If a cluster contains multiple subunits, the feature embedding of the cluster is the average of the embeddings of all subunits in the cluster, i.e.,
2. Selection of Optimal Merging Pair
Find the pair of clusters and with the highest similarity at level , i.e.,
3. Cluster Merging and Calculation of New Cluster Embedding
Execute to generate a new cluster .
Subunits contained in the new cluster: ;
Feature embedding of the new cluster:
Name generation of the new cluster: Through the LLM, a generalized task name is generated based on the goals and activities of the subunits in the cluster to obtain the name of the new cluster.
4. Update of Level Cluster Set
Remove and from the cluster set at level , add , and obtain the cluster set at level l.
5. Stopping Condition Judgment
The clustering is stopped and the maximum level is determined when any of the following conditions is met: - . - The cluster set contains only one cluster (root task cluster). - The similarity between all clusters is lower than the preset threshold .
The maximum level L is preset based on the typical complexity of tasks in the target domain. For example, in the emergency rescue domain, we typically set , corresponding to the hierarchy: root task → domain task → sub-task → atomic task. This ensures that the generated hierarchy is neither too shallow (lacking detail) nor too deep (overly complex).
Step 5: Output Task Hierarchical Structure Finally, the hierarchical structure is output and visualized in the form of a tree diagram:
- Root node: The highest-level cluster . - Intermediate nodes: Intermediate-level clusters. - Leaf nodes: Original task subunits of the lowest-level clusters. - Hierarchical relationship: The parent node contains all child nodes, and the semantics of the child nodes belong to the business scope of the parent node.
4.2. Running Example: Emergency Rescue Task Ontology Construction
To illustrate how our method works in practice, we use a concrete example from the emergency rescue domain. Consider the following input text:
“On March 28, 2025, a 7.7-magnitude earthquake struck central Myanmar. The Myanmar National Disaster Management Council (NDMC) activated the highest level of emergency response. The Chinese International Search and Rescue Team (CISAR) was dispatched to the affected area to conduct search and rescue operations. The team was divided into three groups: a search group, a rescue group, and a medical group. The search group used drones to locate survivors, the rescue group used heavy equipment to extract survivors from collapsed buildings, and the medical group provided emergency medical treatment to the injured."
Our method processes this text through the following steps:
- 1.
- Corpus collection and preprocessing: The text is identified as relevant to emergency rescue and added to the domain corpus.
- 2.
- Bottom-level feature extraction: The LLM extracts the following bottom-level features: Subjects: Myanmar NDMC, CISAR, search group, rescue group, medical group Time: March 28, 2025 Environment: central Myanmar, earthquake-affected area Goals: conduct search and rescue operations, locate survivors, extract survivors, provide medical treatment.
- 3.
- Semantic clustering: Features are clustered based on their shared goal of “earthquake search and rescue".
- 4.
- Top-level feature extraction: The LLM refines the following top-level features: Task activities: activate emergency response, dispatch rescue team, divide into groups, locate survivors, extract survivors, provide medical treatment Information requirements: earthquake magnitude, affected area, survivor locations, building collapse status, medical supply status.
- 5.
-
Relationship modeling: Four types of relationships are generated:
- Task decomposition: earthquake search and rescue → activate response, dispatch team, search, rescue, medical treatment.
- Task division: CISAR → search group (locate survivors), rescue group (extract survivors), medical group (provide treatment).
- Task collaboration: search group provides survivor locations to rescue group.
- Information flow: drones provide location information to search group.
This example demonstrates how our method automatically transforms unstructured text into a structured task ontology.
5. Performance Analysis
5.1. Experimental Design
5.1.1. Experimental Domain and Dataset
To verify the effectiveness of the task ontology construction method proposed in this study, experiments were conducted focusing on the field of emergency disaster rescue. This domain is characterized by standardized task processes, diverse participating entities, complex collaboration logic, and high dynamism, which aligns with the modeling requirements of task ontology construction. Meanwhile, the abundance of relevant corpus resources facilitates further quantitative evaluation. To ensure the comprehensiveness and objectivity of the experiments, a multi-source dataset was constructed from diverse channels in accordance with the considerations outlined in Section 3.2.1, followed by data preprocessing. The preprocessing stage included text cleaning and format standardization: specifically, text cleaning involved removing formatting symbols, duplicate paragraphs, and irrelevant descriptions while retaining task-related text segments; format standardization entailed converting unstructured text into structured text and unifying the expression formats of time, locations, and institutional names.
5.1.2. Evaluation Metrics
Considering the full-process nature of task ontology construction, metrics were designed to cover three core phases: element extraction, ontology quality, and dynamic update. All metrics were normalized to a 0 to 100 scale (or 0 to 1 for F1-score) for consistency.
(1) Element Extraction Performance Metrics
To evaluate the layered hierarchical extraction mechanism, precision (P), recall (R), and F1-score were adopted to measure accuracy and completeness.
Precision (P): Proportion of correctly extracted elements to total extracted elements ().
Recall (R): Proportion of correctly extracted elements to total actual elements ().
F1-Score: Harmonic mean of P and R (), balancing accuracy and completeness.
Where = true positives (correctly extracted elements), = false positives (incorrectly extracted elements), = false negatives (unextracted correct elements). Separate evaluations were conducted for bottom-level features (subject, time, environment, goal) and top-level features (task activity, information requirement).
(2) Ontology Quality Evaluation Metrics
To assess the overall quality of constructed ontologies, five key metrics were designed, aligned with domain requirements for task knowledge modeling:
- 1.
- Feature Extraction Accuracy: precision and completeness of bottom-level/top-level feature extraction.
- 2.
- Hierarchical Structure Rationality: consistency of parent-subtask hierarchy with real-world domain task execution logic.
- 3.
- Core Relationship Richness: completeness of four task relationships.
- 4.
- Domain Application Value: practical utility in scenario tasks.
- 5.
- Ontology Extensibility: adaptability to new task types.
The weighted overall quality score was calculated as:
where – are the raw scores of each metric, and weights – were determined by averaging the weights assigned by three independent LLMs (DeepSeek1, Doubao2 and ERNIE Bot3). Each LLM was asked to assign weights to the five quality metrics based on their importance for task ontologies, using a 0–1 scale. The average weights reflect the consensus among the LLMs on the relative importance of each metric, as shown in Table 5.
Each of the five quality metrics is scored by three independent LLMs. The inter-rater reliability (Fleiss’ kappa) for the overall quality score is 0.78, indicating substantial agreement among LLMs.
5.1.3. Comparison with Existing Benchmarks and Metrics
To the best of our knowledge, there are no publicly available benchmark datasets specifically designed for evaluating automatic task ontology construction methods. Most existing ontology learning benchmarks focus on general concept extraction or relationship discovery, not on the specific requirements of task ontology construction (such as hierarchical structure and process relationships).
Similarly, while standard ontology metrics such as logical consistency are important, they do not capture critical aspects of task ontology quality such as hierarchical structure rationality and domain application value. Our metrics complement standard metrics by including these domain-specific aspects, providing a more comprehensive evaluation of task ontology quality.
5.1.4. Evaluation Framework
(1) Comparative Methods
Six representative methods were selected for comparison, covering different stages of task ontology construction:
- 1.
- BiLSTM-CRF [27]: A traditional sequence labeling model relying on manually designed features.
- 2.
- BERT-CRF [41]: A state-of-the-art pre-LLM deep learning model for information extraction.
- 3.
- GPT-3.5 [42]: A general-purpose LLM used for zero-shot information extraction without our specialized framework.
- 4.
- BASE-SFE (Ablation 1): Our proposed method without the layered hierarchical extraction module.
- 5.
- BASE-DWHC (Ablation 2): Our proposed method without the dynamic weight hierarchical clustering module.
- 6.
- Proposed Method: The complete ATOC-LLM framework, validated using three LLMs: Deepseek, Doubao, and ERNIE Bot.
(2) Experimental Procedures
- 1.
- Data Preparation: We use the database construction method in 3.2.1 to collect corpus data and perform necessary preprocessing. In order to further validate, we select some documents from the database and annotate them with domain experts to obtain the task ontology elements and relationships.
- 2.
- Parameter Optimization: Hyperparameters of all methods were tuned before experiment to ensure fair comparison.
- 3.
- Element Extraction Experiment: All methods performed feature extraction on the test set, with F1-score calculated separately for bottom-level and top-level features.
- 4.
- Ontology Quality Experiment: Ontologies were built using extraction results from each method, and the five quality metrics were evaluated by automated semantic consistency checks.
- 5.
- Generalization Test: The proposed method was re-run with Deepseek, Doubao, and ERNIE Bot to verify model adaptability.
5.2. Experimental Results and Analysis
5.2.1. Element Extraction Performance
Bottom-Level Feature Extraction Results
Table 6 presents the performance of different methods in extracting bottom-level features (subject, time, environment, goal). Notably, BASE-DWHC was excluded from this experiment because dynamic weight clustering affects ontology hierarchy but not feature extraction itself. BASE1, BASE2, and BASE3 represent experiments conducted by Deepseek, Doubao, and ERNIE Bot using the method proposed in this paper, respectively.
Figure 3 visualizes these results. The proposed method (all three LLMs) outperforms BiLSTM-CRF by 12.4 to 17.1% in F1-score, demonstrating LLMs’ superiority in deep semantic understanding. Compared with the state-of-the-art pre-LLM method BERT-CRF, our method achieves an improvement of 8.0 to 13.8% in F1-score, and outperforms the general-purpose LLM baseline GPT-3.5 by 4.0 to 8.3%. This indicates that our specialized hierarchical extraction framework effectively enhances the performance of LLMs in task feature extraction. The F1-score of BASE-SFE (0.83) is 5.9 to 8.4% lower than the proposed method, confirming that hierarchical extraction reduces feature redundancy. Doubao achieves the best performance (F1=0.91), attributed to its stronger domain adaptation for emergency rescue text.
Top-Level Feature Extraction Results
Table 7 shows the performance in extracting top-level features, which is a more challenging task requiring abstract semantic aggregation.
Figure 4 illustrates the results, with three critical insights: The F1-score of the proposed method is 23.0% to 42.6% higher than BiLSTM-CRF, as LLMs can infer abstract task activities and information dependencies. Our method outperforms BERT-CRF by 16.7% to 20.8% and GPT-3.5 by 6.3% to 10.1%, demonstrating that our framework provides significant value even compared to powerful general-purpose LLMs, especially for abstract feature extraction. The performance gap between bottom-level (0.83) and top-level (0.69) features for BASE-SFE is 16.9%, revealing that without hierarchical mapping, abstract features become disconnected from basic attributes.
5.2.2. Ontology Quality Evaluation
To obtain the quality scores in Table 8, we employed three LLMs as mentioned earlier in the article. For each ontology construction method and each of the five evaluation dimensions, each LLM assigned a score from 0 to 100 based on predefined rubrics. The reported score for each dimension is the arithmetic mean of the three LLMs’ ratings. This multi-LLM evaluation protocol mitigates individual model bias and ensures more robust and objective quality assessment. Table 8 presents the quality scores of ontologies constructed by different methods and Figure 5 visualizes the multi-dimensional performance.
Analysis of the various indicators of the ontology quality assessment results: 1. Feature Extraction Accuracy: The proposed method outperforms BiLSTM-CRF, BERT-CRF, and GPT-3.5, as hierarchical extraction ensures semantic consistency between subject and task activity. 2. Hierarchical Rationality: BASE-DWHC scores 10.0 to 15.0% lower than the proposed method, verifying that dynamic weight clustering aligns hierarchy with domain task execution logic. Notably, GPT-3.5 scores 78 in this metric, lower than our method, indicating that our domain-aware hierarchical clustering produces more logically consistent task structures than general-purpose LLMs. 3. Core Relationship Richness: The proposed method captures more implicit relationships compared to BERT-CRF, GPT-3.5, and BiLSTM-CRF. This is because our relationship extraction module is specifically designed to model the four core types of task relationships. 4. Domain Application Value: The proposed method supports practical tasks like cross-border resource allocation better than ablation models and baselines, as its ontology retains task execution logic. 5. Ontology Extensibility: The proposed method (86 to 89) adapts to new tasks 35.9 to 41.3% better than BiLSTM-CRF (63) and 14.7 to 18.7% better than GPT-3.5 (75). This may be due to its modular design and complete feature extraction results.
From the weighted score perspective, the proposed method (83.9 to 86.4) outperforms BiLSTM-CRF by 21.2 to 24.8%, BERT-CRF by 11.7 to 15.0%, GPT-3.5 by 4.6 to 7.7%, and ablation models by 9.0 to 11.7%. BASE-DWHC and BASE-SFE have similar scores (74.7 and 74.9), indicating that hierarchical extraction and dynamic clustering are complementary, and neither alone can maximize ontology quality.
5.2.3. Logical Consistency and Reasoning Capability Evaluation
In addition to the LLM-based quality metrics, we also evaluated the logical consistency and reasoning capability of the generated ontologies using the Pellet OWL reasoner. We checked three key aspects: 1. Class satisfiability: Whether all classes in the ontology can have instances without logical contradiction. 2. Ontology consistency: Whether the ontology as a whole is logically consistent. 3. Entailment capability: Whether the ontology can correctly infer implicit relationships that are not explicitly stated.
The results are shown in Table 9.
All ontologies generated by our method are fully consistent and satisfiable. The high entailment accuracy demonstrates that our ontologies not only capture explicit knowledge but also support reasoning about implicit relationships, which is critical for practical applications such as intelligent decision-making and task scheduling. Notably, GPT-3.5 achieves good consistency but lower entailment accuracy than our method, indicating that while general-purpose LLMs can generate logically consistent structures, they lack the specialized reasoning capabilities required for task ontology modeling.
5.2.4. Comparison with Existing Domain Ontologies
We compared our automatically generated emergency rescue ontology with an existing manually constructed emergency ontology: Disaster Response Ontology [43], an ontology for sharing and managing information in disaster response. The comparison results are shown in Table 10.
Our method generates an ontology with broader coverage of emergency tasks while maintaining the same level of logical consistency.
5.2.5. Application Validation of Constructed Ontology
To verify real-world utility, the ontology constructed by the proposed method is applied to an emergency rescue scenario. Figure 6 illustrates the application workflow, and key steps are detailed below:
(1) Text Input and Task Class Matching
Input Text: A 7.7-magnitude earthquake struck central Myanmar on March 28, 2025. The Myanmar NDMC activated the highest emergency response, collaborating with the Chinese International Search and Rescue Team and ASEAN AHA Centre for urban rubble search, and coordinating the International Medical Corps to set up temporary medical points. USGS aftershock data and Mandalay building safety reports were required continuously.
The ontology’s semantic matching engine localized the task class via: 1. Trigger word extraction: "earthquake", "emergency response", "rubble search", "temporary medical points". 2. Hierarchy matching: Cross-Border Emergency Rescue → Myanmar Earthquake Rescue → Urban Rubble Search Subtask. 3. Element verification: Preliminary identification of subjects (NDMC, Chinese Rescue Team) and time (2025-03-28) to confirm matching accuracy.
(2) Task Elements Parsing
Based on the matched "Urban Rubble Search Subtask", the ontology automatically parsed: Bottom-Level Features: - Subject: Myanmar NDMC, Chinese International Search and Rescue Team, ASEAN AHA Centre, International Medical Corps. - Time: 72-hour golden rescue period post-earthquake. - Environment: Urban rubble in central Myanmar, aftershock risks, road disruptions. - Goal: Rescue trapped people, treat injuries, relocate victims, ensure team safety.
Top-Level Features: - Task Activities: Rubble reconnaissance, life signal detection, breaking-rescue, cross-border medical transfer, temporary shelter setup. - Information Requirements: USGS aftershock data, Mandalay building safety reports, ASEAN disaster images, rescue route plans.
(3) Four Core Relationships Extraction
The ontology structured relationships from the text: - Task Decomposition: "Myanmar Earthquake Rescue", "Rubble Search", "Cross-Border Medical Aid", "Victim Relocation", "Material Allocation". - Task Division: NDMC (rescue coordination), Chinese Team (breaking-rescue), ASEAN AHA Centre (resource coordination), Medical Corps (medical points). - Task Collaboration: Teams synchronize progress every 25 minutes; medical teams prepare treatment plans based on life signals. - Information Interaction: USGS joint command (aftershock warnings), ASEAN AHA Centre teams (disaster plans), Medical Corps command (resource needs).
This application verified the ontology’s ability to automate "text-to-knowledge" conversion, laying a foundation for upper-level tasks like intelligent decision-making and scenario retrieval.
5.3. Cross-Domain Generalization Experiment
To evaluate the generalization capability of our method, we conducted additional experiments in two other domains: intelligent healthcare and intelligent manufacturing. We constructed datasets for each domain following the same procedure as the emergency rescue dataset. The results are shown in Table 11.
The results show that our method achieves consistent good performance across all three domains, with only a slight decrease in performance in the healthcare and manufacturing domains compared to the emergency rescue domain. This provides strong empirical support for our claim of strong generalization capability.
5.4. Reusability and Integration with Semantic Web Systems
The ontologies generated by our method are fully compliant with the OWL 2 DL standard, which endows them with native compatibility with all mainstream Semantic Web tools and systems, laying a solid foundation for their widespread reuse and integration across different application scenarios. Specifically, these ontologies can be incorporated as modular components into larger-scale domain ontologies through the standard OWL import mechanism, facilitating the hierarchical construction and incremental extension of knowledge systems. Meanwhile, with the assistance of widely adopted ontology matching tools such as Alignment API and LogMap, they can be semantically aligned with existing domain ontologies to resolve semantic heterogeneity among disparate knowledge frameworks, thereby enabling cross-system knowledge sharing and interoperability. In addition, the structured task knowledge encoded in these ontologies provides a robust foundation for semantic search engines, supporting task retrieval based on intrinsic semantic meaning rather than superficial keyword matching, which substantially improves both the precision and recall of information retrieval. Furthermore, these ontologies can be seamlessly integrated into various decision support systems, leveraging their structured knowledge representation and inherent reasoning capabilities to deliver context-aware task recommendations and intelligent decision support for upper-level applications.
6. Conclusions
To address key challenges in traditional task ontology construction, this study proposes an LLM-driven automatic construction method. Focusing on the core requirements of full-process automation, hierarchical semantic consistency, and dynamic scenario adaptability, the proposed method consists of six functional modules and constructs an end-to-end technical framework covering corpus collection and preprocessing, hierarchical feature extraction, core relationship modeling, hierarchical structure construction, OWL ontology generation, and dynamic ontology update.
In terms of technical implementation, a customized LLM prompt engineering strategy enhances the model’s semantic understanding of task attributes and abstract features for improved extraction accuracy. A hierarchical mechanism combined with dynamic weight hierarchical clustering builds an accurate task hierarchical structure aligned with practical domain task execution logic. A core relationship semantic modeling module systematically describes dynamic relationships including task decomposition, division of labor, collaboration, and information interaction. Additionally, an event gap identification and incremental update mechanism ensures the ontology’s dynamic adaptability to domain task iterations.
Experiments with real-world emergency rescue data verify the method’s advantages in construction efficiency, accuracy, and dynamism. Results show the method enables full automation of task ontology construction, outperforms traditional BiLSTM-CRF and two core module ablation models in key dimensions, and exhibits good generalization across different LLMs. Compared with traditional methods, it significantly reduces manual annotation costs while improving construction efficiency and quality, effectively supporting task knowledge application in intelligent decision-making and multi-agent collaboration. This study provides a technical path for automated task ontology construction based on hierarchical semantics and dynamic clustering. Future work will focus on optimizing LLMs’ cross-domain generalization ability and refining update mechanisms to support more dynamic scenarios.
Author Contributions
Conceptualization, Z.L. and J.Y.; methodology, Z.L.; software, Z.L.; validation, Z.L., J.Y., J.W. and Y.L.; formal analysis, Z.L.; investigation, Z.L.; resources, J.Y.; data curation, Z.L.; writing—original draft preparation, Z.L.; writing—review and editing, J.Y., Y.L., C.Z. and J.W.; visualization, Z.L.; supervision, J.Y.; project administration, J.Y.; funding acquisition, J.Y. All authors have read and agreed to the published version of the manuscript.
Funding
This work was partially supported by the National Natural Science Foundation of China (Grant No. 62402505) and the Independent Innovation Science Fund of National University of Defense Technology (Grant No. 22-ZZCX-055).
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The data presented in this study are available on request from the corresponding author.
Acknowledgments
The authors thank the anonymous reviewers for their valuable comments and suggestions. During the preparation of this manuscript, the authors used DeepSeek V4.1, Doubao, and ERNIE Bot for the purposes of text generation assistance and semantic embedding computation. The authors have reviewed and edited the output and take full responsibility for the content of this publication.
Conflicts of Interest
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Appendix A. Authority Score Assignment for Different Data Sources
Table A1 shows the authority scores assigned to different types of data sources. These scores are based on the credibility and reliability of the source.
Table A1.
Authority Score Assignment.
| Data Source Type | Authority Score |
|---|---|
| Peer-reviewed academic journals | 1.0 |
| Government and international organization websites | 0.9 |
| Reputable news media (BBC, CNN, Reuters, etc.) | 0.8 |
| University and research institute websites | 0.8 |
| Industry white papers and technical reports | 0.7 |
| Personal blogs and social media | 0.3 |
Appendix B. Example OWL Ontology Snippet
Below is a simplified OWL-Turtle snippet of the earthquake rescue task ontology generated from our running example:
- fullflexible
- @prefix rdf: <[http://www.w3.org/1999/02/22-rdf-syntax-ns](http://www.w3.org/1999/02/22-rdf-syntax-ns)#> .
- @prefix rdfs: <[http://www.w3.org/2000/01/rdf-schema](http://www.w3.org/2000/01/rdf-schema)#> .
- @prefix owl: <[http://www.w3.org/2002/07/owl](http://www.w3.org/2002/07/owl)#> .
- @prefix task: <[http://example.org/task-ontology](http://example.org/task-ontology)#> .
- task:EarthquakeSearchAndRescue a owl:Class ;
- rdfs:subClassOf task:EmergencyResponseTask .
- task:SearchAndRescueTeam a owl:Class ;
- rdfs:subClassOf task:Organization .
- task:CISAR a task:SearchAndRescueTeam ;
- task:hasSubTeam task:SearchGroup, task:RescueGroup, task:MedicalGroup .
- task:SearchGroup a task:Team ;
- task:performs task:LocateSurvivors .
- task:LocateSurvivors a task:Task ;
- task:requiresInformation task:SurvivorLocation ;
- task:usesEquipment task:Drone .
Appendix C. Context Correlation Scoring Prompt Template
The following fixed prompt template is used for all context correlation judgments between bottom-level features and task goals. The temperature is strictly set to 0.0 to ensure deterministic and fully reproducible outputs across all runs.
Table A2.
Standardized prompt template for context correlation scoring.
| Prompt Template |
|---|
| Please judge the execution context correlation between the given feature description and task goal. Output a score between 0.0 and 1.0 (retain two decimal places) strictly following the scoring criteria below: |
| Scoring Criteria: |
| - 1.0: Perfectly relevant. The feature description is a direct and necessary component for completing the task goal. |
| - 0.7: Moderately relevant. The feature has a clear logical connection to the task execution context but is not a core component of the goal. |
| - 0.3: Weakly relevant. Only superficial lexical overlap exists, with no meaningful execution context association. |
| - 0.0: Irrelevant. No logical connection between the feature and the task goal. |
| Feature Description: {feature_text} |
| Task Goal: {goal_text} |
| Output Format (strict JSON only): |
| { |
| "correlation_score": 0.00 |
| } |
This unified template ensures that all three LLMs (Deepseek, Doubao, ERNIE Bot) follow the same scoring rubric, eliminating subjective variations and ensuring the comparability of correlation scores across different models and experimental conditions.
References
- Studer, R.; Benjamins, V.R.; Fensel, D. Knowledge engineering: Principles and methods. Data Knowl. Eng. 1998, 25, 161–197. [Google Scholar] [CrossRef]
- Wielinga, B.J.; Schreiber, A.T.; Breuker, J.A. KADS: A modelling approach to knowledge engineering. Knowl. Acquis. 1992, 4, 5–53. [Google Scholar] [CrossRef]
- Fox, M.S.; Barbuceanu, M.; Teigen, R. TOVE: Towards a common sense model of the enterprise. In Enterprise Integration and Modeling Technology; Springer: Boston, MA, USA, 1998; pp. 1–22. [Google Scholar]
- Martin, D.; Burstein, M.; Hobbs, J.; et al. OWL-S: Semantic markup for web services. W3C Note 2004, 22, 1–29. [Google Scholar]
- Gangemi, A.; Presutti, V. Ontology design patterns. In Handbook on Ontologies; Springer: Berlin, Heidelberg, Germany, 2009; pp. 221–243. [Google Scholar]
- Caufield, K.; Wang, S.; Zhou, Z.; et al. SPIRES: Structured prompt interrogation and recursive extraction of semantics for knowledge graph construction from scientific literature. J. Biomed. Inform. 2024, 149, 104587. [Google Scholar]
- Caufield, K.; Zhou, Z.; Wang, S.; et al. OntoGPT: A flexible framework for ontological knowledge extraction with large language models. arXiv 2023, arXiv:2304.02711. [Google Scholar]
- Guarino, N. Formal ontology in information systems. In Proceedings of FOIS’98; IOS Press: Trento, Italy, 1998; pp. 3–15. [Google Scholar]
- Scherp, A.; Franz, T.; Saathoff, C.; et al. F—A model of events based on the foundational ontology DOLCE+DnS Ultralight. In Proceedings of the 5th International Conference on Knowledge Capture; ACM: Redondo Beach, CA, USA, 2009; pp. 137–144. [Google Scholar]
- Dionisio, J.D.N.; Kimball, W.C.; Martinez, A. Task ontologies for knowledge sharing and reuse. In Proceedings of the 12th International Workshop on Distributed Artificial Intelligence, Hidden Valley, PA, USA, 1993; pp. 21–38. [Google Scholar]
- Grenon, P.; Smith, B. SNAP and SPAN: Towards dynamic spatial ontology. Spat. Cogn. Comput. 2004, 4, 69–103. [Google Scholar] [CrossRef]
- Mizoguchi, R.; Vanwelkenhuysen, J.; Ikeda, M. Task ontology for reuse of problem solving knowledge. In Proceedings of the 10th Knowledge Acquisition for Knowledge-Based Systems Workshop, Banff, Canada, 1995; pp. 1–16. [Google Scholar]
- Uschold, M.; Gruninger, M. Ontologies: Principles, methods and applications. Knowl. Eng. Rev. 1996, 11, 93–155. [Google Scholar] [CrossRef]
- Fernández-López, M.; Gómez-Pérez, A.; Juristo, N. METHONTOLOGY: From ontological art towards ontological engineering. In Proceedings of the 14th National Conference on Artificial Intelligence and 9th Innovative Applications of Artificial Intelligence Conference; AAAI Press: Providence, RI, USA, 1997; pp. 333–338. [Google Scholar]
- Noy, N.F.; McGuinness, D.L. Ontology development 101: A guide to creating your first ontology. Stanf. Knowl. Syst. Lab. Tech. Rep. 2001, SMI-2001-0880. [Google Scholar]
- Yusuke, K.; Riichiro, M.; Toru, I. Task ontology design for service navigation systems. In Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology; IEEE: Sydney, Australia, 2008; pp. 421–424. [Google Scholar]
- Noy, N.F.; Fergerson, R.W.; Musen, M.A. The knowledge model of Protégé-2000: Combining interoperability and flexibility. In Proceedings of the 12th International Conference on Knowledge Engineering and Knowledge Management; Springer: Angers, France, 2000; pp. 17–32. [Google Scholar]
- Hepp, M. Ontologies: State of the art and future challenges. IEEE Trans. Knowl. Data Eng. 2007, 19, 1630–1643. [Google Scholar]
- Klein, M.; Fensel, D.; Kiryakov, A.; et al. Ontology learning and its application to automated terminology translation. IEEE Intell. Syst. 2002, 17, 22–32. [Google Scholar] [CrossRef]
- Maedche, A.; Staab, S. Ontology learning for the semantic web. IEEE Intell. Syst. 2001, 16, 72–79. [Google Scholar] [CrossRef]
- Sungmin, K.; Jaehyun, P.; Hyunbo, C. Rule-based ontology construction for intelligent building manufacturing task management. J. Comput. Des. Eng. 2018, 5, 387–398. [Google Scholar]
- Stojanovic, L.; Stojanovic, N.; Volz, R. Migrating data-intensive web sites into the semantic web. In Proceedings of the 13th International Conference on Database and Expert Systems Applications; Springer: Aix-en-Provence, France, 2002; pp. 414–424. [Google Scholar]
- Pan, J.Z.; Horrocks, I. OWL-Eu: Adding customised datatypes into OWL. J. Web Semant. 2007, 5, 27–34. [Google Scholar] [CrossRef]
- Shamsfard, M.; Barforoush, A.A. The state of the art in ontology learning: A comparative framework. Knowl. Eng. Rev. 2004, 18, 293–316. [Google Scholar] [CrossRef]
- Buitelaar, P.; Cimiano, P.; Magnini, B. Ontology Learning from Text: Methods, Evaluation and Applications; IOS Press: Amsterdam, The Netherlands, 2005; pp. 1–220. [Google Scholar]
- Hao, Y.; Zhang, Y.; Liu, K.; et al. Event extraction with distant supervision via CNN-based feature representation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing; Association for Computational Linguistics: Beijing, China, 2015; pp. 1456–1465. [Google Scholar]
- Wang, X.; Liu, Y.; Zhang, Z. Geological task ontology construction based on BiLSTM-CRF model. Geol. Bull. China 2020, 39, 1752–1760. [Google Scholar]
- Joachims, T. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the 10th European Conference on Machine Learning; Springer: Chemnitz, Germany, 1998; pp. 137–142. [Google Scholar]
- Ritter, A.; Mausam; Etzioni, O.; et al. Open domain event extraction from Twitter. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; ACM: Beijing, China, 2012; pp. 1060–1068. [Google Scholar]
- Zhang, C.; Li, Y.; Niu, Z. A survey of ontology learning methods. Chin. J. Comput. 2021, 44, 587–611. [Google Scholar]
- Li, Q.; Ji, H. Incremental event extraction via supervised topic adaptation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing; Association for Computational Linguistics: Doha, Qatar, 2014; pp. 1374–1384. [Google Scholar]
- Zhang, N.; Deng, S.; Sun, Z.; et al. Deep learning for knowledge graph construction: A survey. J. Comput. Sci. Technol. 2022, 37, 572–595. [Google Scholar]
- Chen, X.; Jia, S.; Xiang, Y. Knowledge reasoning over knowledge graphs: A review. Expert Syst. Appl. 2020, 141, 112948. [Google Scholar] [CrossRef]
- Ho, T.; Pham, T.; Nguyen, L. BERT-based task ontology element extraction for emergency response. In Proceedings of the 14th International Conference on Knowledge and Systems Engineering; IEEE: Nha Trang, Vietnam, 2022; pp. 1–6. [Google Scholar]
- Gan, L.; Wang, H.; Zhang, L. BERT-CRF based task entity extraction for ship condition monitoring. J. Mar. Eng. Technol. 2023, 22, 112–120. [Google Scholar]
- Lin, Y.; Ji, H.; Huang, L.; et al. A joint neural model for event extraction with document-level context. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020; Association for Computational Linguistics: Online; pp. 6303–6313. [Google Scholar]
- Xie, J.; Liu, S.; Wang, Y. Multi-branch BiGRU-CRF for geological disaster task ontology construction. Comput. Eng. Appl. 2024, 60, 189–197. [Google Scholar]
- Pedram, M.; Pourreza, H.; Davoodi, R. Few-shot prompt engineering for task knowledge extraction from unstructured text. arXiv 2023, arXiv:2305.12723. [Google Scholar]
- Wei, J.; Wang, X.; Schuurmans, D.; et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Red Hook, NY, USA, 2022; pp. 24824–24837. [Google Scholar]
- Liu, Z.; Zhang, X.; Li, J. Task ontology construction framework integrating LLaMA and BERT. Comput. Appl. 2024, 44, 456–463. [Google Scholar]
- Devlin, J.; Chang, M.W.; Lee, K.; et al. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018, arXiv:1810.04805. [Google Scholar]
- Brown, T.B.; Mann, B.; Ryder, N.; et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Red Hook, NY, USA, 2020; pp. 1877–1901. [Google Scholar]
- Khantong, S.; Arch-int, S.; Arch-int, N. Disaster response ontology for information sharing and management. Sustainability 2020, 12, 8127. [Google Scholar] [CrossRef]
Figure 1.
Fully automatic task ontology construction framework.

Figure 2.
Schematic diagram of task core process relationship modeling.

Figure 3.
Bar chart for bottom-level feature extraction performance comparison.

Figure 4.
Bar chart for top-level feature extraction performance comparison.

Figure 5.
Bar chart for ontology quality evaluation results.

Figure 6.
Schematic diagram of task ontology application in emergency rescue.

Table 1.
Construction results of the emergency rescue domain corpus database.
| Source | Content |
|---|---|
| Academic Papers | 1. Task Offloading for Post-disaster Rescue in Vehicular Fog Computing-assisted UAV Networks |
| 2. Emergency Communication Systems for Natural Disaster Response Using Delay-Tolerant Networks | |
| … | |
| Official Websites | 1. The United Nations Office for the Coordination of Humanitarian Affairs (OCHA) plays a pivotal role in natural disaster emergency…–United Nations Office for the Coordination of Humanitarian Affairs (OCHA) |
| 2. In the face of wildfires, the Australian Government’s Department of Home Affairs mobilizes extensive resources…–Australian Department of Home Affairs Official Website, Natural Disaster Emergency Section | |
| … | |
| News Reports | 1. Flood Rescue Efforts Intensify in New South Wales, Australia —-ABC News Report |
| 2. Japan’s Self-Defense Forces Join Typhoon Rescue Operations—-NHK World News | |
| … |
Table 2.
Prompt engineering input for bottom-level feature extraction.
| Prompt Engineering Input |
|---|
| Please extract the bottom-level features of the task from the following text, including the four contents of subject, time, environment, and task goal. If a certain type of feature is not mentioned, mark it as "None". |
| Text Content: {text} |
| Output Format: { "task_bottom_features": [{ "participants": ["Subject 1 Name", "Subject 2 Name"], "time": "Task Execution Time Description", "environment": "Task Execution Environment Description", "task_goal": "Task Goal Description" }] } |
| payload = { "messages": [{ "role": "user", "content": prompt }], "model": "deepseek-chat", "temperature": 0.1, "response_format": { "type": "json_object" } } |
Table 3.
Prompt engineering input for top-level feature extraction.
| Prompt Engineering Input |
|---|
| Please refine the top-level features of the task cluster based on the following bottom-level features of the task cluster, including task activities and information requirements. The task activities should reflect the logic of operation steps, and the information requirements should clarify the key information types. |
| Bottom-Level Features of the Task Cluster: {cluster_bottom_features} |
| Output Format: { "task_top_features": [{ "task_activities": ["Activity 1 Description", "Activity 2 Description"], "information_needs": ["Information 1 Type", "Information 2 Type"] }] } |
| payload = { "messages": [{ "role": "user", "content": prompt }], "model": "deepseek-chat", "temperature": 0.3, "response_format": { "type": "json_object" } } |
Table 4.
Design of prompt engineering input for task ontology core process relationship extraction.
| Prompt Engineering Input |
|---|
| Please analyze and generate task division, task decomposition, task collaboration, and task information relationships based on the following structured elements of the task subunit. The relationship descriptions should conform to the task execution logic. |
| Task Subunit Elements: {task_unit_features} |
| Output Format: { "task_core_relations": [{ "task_division": "Description of the correspondence between subjects and responsible activities", "task_decomposition": "Logical description of task decomposition into sub-activities", "task_collaboration": "Description of the cooperation mode between subjects", "task_information_relation": "Description of information acquisition and transmission by subjects" }] } |
| payload = { "messages": [{ "role": "user", "content": prompt }], "model": "deepseek-chat", "temperature": 0.4, "max_tokens": 1500, "response_format": { "type": "json_object" } } |
Table 5.
Results of Evaluation System Construction (Hyperparameter Weights).
| Model | : Accuracy | : Rationality | : Richness | : Domain | : Extensibility |
|---|---|---|---|---|---|
| DeepSeek | 0.25 | 0.15 | 0.20 | 0.30 | 0.10 |
| Doubao | 0.28 | 0.22 | 0.20 | 0.18 | 0.12 |
| ERNIE Bot | 0.30 | 0.25 | 0.20 | 0.15 | 0.10 |
| Average | 0.28 | 0.21 | 0.20 | 0.21 | 0.10 |
Table 6.
Experimental results of bottom-level feature extraction.
| Method | Precision (P) | Recall (R) | F1-Score |
|---|---|---|---|
| BASE1 (Proposed) | 0.91 | 0.87 | 0.89 |
| BASE2 (Proposed) | 0.93 | 0.89 | 0.91 |
| BASE3 (Proposed) | 0.90 | 0.86 | 0.88 |
| GPT-3.5 (LLM Baseline) | 0.87 | 0.82 | 0.84 |
| BERT-CRF (Pre-LLM SOTA) | 0.83 | 0.78 | 0.80 |
| BiLSTM-CRF (Traditional Baseline) | 0.79 | 0.73 | 0.76 |
| BASE-SFE (Ablation) | 0.85 | 0.81 | 0.83 |
Table 7.
Experimental results of top-level feature extraction.
| Method | Precision (P) | Recall (R) | F1-Score |
|---|---|---|---|
| BASE1 (Proposed) | 0.88 | 0.83 | 0.85 |
| BASE2 (Proposed) | 0.90 | 0.85 | 0.87 |
| BASE3 (Proposed) | 0.87 | 0.82 | 0.84 |
| GPT-3.5 (LLM Baseline) | 0.82 | 0.77 | 0.79 |
| BERT-CRF (Pre-LLM SOTA) | 0.75 | 0.69 | 0.72 |
| BiLSTM-CRF (Traditional Baseline) | 0.65 | 0.58 | 0.61 |
| BASE-SFE (Ablation) | 0.73 | 0.66 | 0.69 |
Table 8.
Ontology quality evaluation results.
| Method | ||||||
|---|---|---|---|---|---|---|
| BASE1 (Proposed) | 85 | 90 | 90 | 85 | 87 | 86.4 |
| BASE2 (Proposed) | 81 | 85 | 92 | 80 | 89 | 85.5 |
| BASE3 (Proposed) | 80 | 82 | 89 | 85 | 86 | 83.9 |
| GPT-3.5 (LLM Baseline) | 80 | 78 | 87 | 78 | 75 | 80.2 |
| BERT-CRF (Pre-LLM SOTA) | 76 | 72 | 83 | 72 | 68 | 75.1 |
| BiLSTM-CRF (Traditional Baseline) | 68 | 67 | 81 | 65 | 63 | 69.2 |
| BASE-DWHC (Ablation) | 75 | 75 | 82 | 70 | 70 | 74.9 |
| BASE-SFE (Ablation) | 70 | 80 | 80 | 75 | 65 | 74.7 |
Table 9.
Logical Consistency and Reasoning Results.
| Method | Class Satisfiability | Ontology Consistency | Entailment Accuracy |
|---|---|---|---|
| BASE1 | 100% | 100% | 92% |
| BASE2 | 100% | 100% | 89% |
| BASE3 | 100% | 100% | 87% |
| GPT-3.5 (LLM Baseline) | 99% | 98% | 82% |
| BERT-CRF (Pre-LLM SOTA) | 96% | 94% | 72% |
| BiLSTM-CRF (Traditional Baseline) | 94% | 91% | 68% |
| BASE-SFE (Ablation) | 98% | 97% | 79% |
| BASE-DWHC (Ablation) | 97% | 96% | 76% |
Table 10.
Comparison with Existing Emergency Ontologies.
| Metric | ATOC-LLM | Disaster Response Ontology |
|---|---|---|
| Number of Classes | 127 | 76 |
| Number of Properties | 42 | 28 |
| Coverage of Emergency Tasks | 92% | 72% |
| Logical Consistency | 100% | 100% |
Table 11.
Cross-Domain Generalization Results.
| Domain | Bottom-level F1 | Top-level F1 | Overall Quality Score |
|---|---|---|---|
| Emergency Rescue | 0.91 | 0.87 | 85.5 |
| Intelligent Healthcare | 0.88 | 0.83 | 82.1 |
| Intelligent Manufacturing | 0.87 | 0.82 | 81.3 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.