Named Entity Recognition Based on Contrastive Learning and Enhanced Lexicon for Pig Diseases of Chinese Corpus

1. Introduction

In the context of large-scale and intensive pig breeding practices, it is of great significance to establish intelligent diagnostic and preventive measures for pig diseases. Early prevention and timely diagnosis are pivotal for maintaining swine health and mitigating potential losses. Named Entity Recognition (NER) assumes a critical role in this endeavor by identifying specific entities within textual corpora, serving as the cornerstone for numerous downstream tasks in natural language processing. These tasks include but are not limited to information retrieval, intelligent question answering, and knowledge graph construction. However, the existing entity recognition methods mostly focus on recognition of person, location and organization, etc. Given the pressing need to bolster disease surveillance and management in swine, there arises an urgent imperative to develop specialized NER methodologies tailored to the specific lexicon of pig disease terminology in Chinese.

The early NER methods include rule-based recognition methods and statistics-based machine learning recognition methods. In recent years, with the rapid development of neural networks, methods of deep learning are more suitable for the task of NER and become the mainstream method [1,2,3,4,5].

The rule-based NER method requires the rules which are formulated manually by experts. This method has high accuracy when dealing with small datasets, but it is difficult to expand it on a large scale and apply it in different domains because the rules are based on manual construction, which is a time-consuming task [6].

The statistics-based NER method select the appropriate training model according to the specific research background. Commonly used statistical models include hidden Markov models(HMM), conditional random field model(CRF), branch support vector machine (SVM) and maximum entropy model (ME), etc. Compared to the rule-based model, this method omits many tedious rule designs and are fast, portable and convenient to use [7,8]. However, the statistics-based method requires a large number of manually labeled datasets to train model parameters, which is gradually replaced by deep learning method.

The deep learning based NER method can learn more complex features and achieve good results. In contrast to the preceding two approaches, deep learning-based NER methods do not necessitate an abundance of artificial features. Therefore, the deep learning-based methods has been widely concerned by researchers. Common deep learning models include convolutional neural network (CNN), recurrent neural network (RNN), graph neural network (GNN), deep neural network (DNN), generative adversarial network (GAN), long short-term memory network (LSTM), Transformer and BERT(bi-directional encode representation from transformers) and so on [1,9]. Compared to the rule-based and statistics-based models, deep learning models are dominant and achieve state-of-the-art results in NER. However, the scalability of deep learning models applied in specific domain remains a significant challenge.

The lexicon-based NER method can effectively avoid segmentation errors and improve the accuracy of entity boundary recognition by integrating potential word information into feature vectors. Currently, a large number of lexicon enhanced Chinese entity extraction methods have been proposed, with better performance than methods based on character embedding or word embedding. Lattice-LSTM [10] has achieved new benchmark results on several public Chinese NER datasets. However, the Lattice-LSTM model architecture is complex, which limits its application in many industrial areas requiring real-time NER responses. A convolutional neural network based method that incorporates lexicons using a rethinking mechanism was proposed,which can model all the characters and potential words that match the sentence in parallel [11]. A lexicon-based graph neural network with global semantics was proposed to tackle word ambiguities. In this model, the lexicon knowledge is used to connect characters to capture the local composition, while a global relay node can capture global sentence semantics and long-range dependency [12]. A Lexicon Enhanced BERT (LEBERT) for Chinese sequence labeling was put forward [13]. The model integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer and achieves better performance than both lexicon enhanced models and BERT baseline in Chinese datasets. More character-word association models have been proposed, such as SoftLexicon [14],FLAT [15],PLTE [16].

The pre-trained model-based NER method effectively leverages deep bidirectional contextual information. It demonstrates superior performance with shorter training times, reduced labeling data requirements, and improved results compared to traditional models. Currently, BERT [17] is widely used, followed by ELMo [18], RoBERTA [19], ERNIE [20], ALBERT [21], and others. At present, the pre-trained models and lexicon are integrated by utilizing their respective strengths. Li proposed Flat-LAttice Transformer for Chinese NER, which converts the lattice structure into a flat structure consisting of spans [15]. Li proposed the LEBERT-BiLSTM-CRF model for elementary mathematics text NER, which integrates external lexicon knowledge into BERT layers directly by a lexicon adapter layer and performs better than other NER models [22] .

Contrastive learning acquires feature representations of samples by comparing positive and negative samples in feature space. This approach has garnered significant attention in the fields of computer vision (CV) and natural language processing (NLP). ConSERT (Contrastive Framework for Self-supervised Sentiment Representation Transfer) and SimCSE(Simple Contrastive Learning of Sentiment Embedding) model, which use different data enhancement methods and comparative learning loss function to learn the representation of sentences, obtain SOTA results on the task of text semantic similarity [23,24]. COntrastive learning with Prompt guiding for few-shot NER (COPNER) was proposed and outperforms state-of-the-art models with a significant margin in most cases. This method introduces category specific words COPNER composed of prompts as supervised signals for contrastive learning to optimize entity token representation [25]. Moreover, Named Entity Recognition in low-resource scenarios based on contrastive learning has also received considerable attention [26,27,28]. He proposed a novel prompt-based contrastive learning method for few-shot NER without template construction and label word mappings [26]. Li proposed a multi-task learning framework CLINER for Few-Shot NER [27].

In the field of livestock husbandry, text mining, Named Entity Recognition (NER), intelligent question-and-answer systems, and artificial intelligence (AI) technologies have been gradually applied. However, this field faces numerous challenges, including the prevalence of technical terms, complex knowledge structures, fine knowledge granularity, and a lack of labeled datasets [29]. Seok created a BERT-DIS-NER model that adds a CRF layer to BERT for the disease named entity recognition and used syllable unit-based named entity recognition that can reflect the characteristics of disease names. The F1-score is 0.81 trained with human data and fine-tuned with animal data [30]. Kung designed and implemented an intelligent knowledge question-and-answer system for pig farming based on bi-GRU and SNN methods, combined with the LTSM deep-learning method [31] .

NER methods have been found extensive applications in the agricultural domain [32,33,34,35,36,37]. Nonetheless, there remains a apparent gap in current research concerning the accurate recognition of named entities within the domain of pig diseases in Chinese. Pig disease data is characterized by complex entities, fuzzy boundaries and domain-specific vocabulary. Unlike conventional NER tasks focusing on common entities such as person and organization names, pig disease data encompasses specialized terminologies drawn from the domains of animal husbandry and veterinary science.

Furthermore, the resources in the field of pig diseases are confined and dispersed, exacerbating the scarcity of publicly available benchmark corpora and labeled datasets specific to this domain in Chinese. While considerable research has been devoted to NER systems in human medicine [38,39], it remains impractical to directly transfer such models to the domain of pig diseases due to the domain-specific rules and vocabulary governing this domain. Hence, named entity recognition in the field of pig diseases needs to be further explored. A model of Pig Disease Chinese Named Entity Recognition(PDCNER) is proposed in this paper. The main contributions of the paper are as follows:

(1)A named entity recognition model that integrates contrastive learning and enhanced lexicon was proposed for pig diseases corpus in Chinese, which achieved the best recognition results.

(2)To enrich the contextual understanding and semantic representation of pig disease data, we employed Lexicon-enhanced BERT. This approach facilitated the direct integration of external lexicon knowledge in pig disease domain into BERT layers via a Lexicon Adapter layer, seamlessly combining the characteristics of both characters and words. Furthermore, we used contrastive learning to maximize the agreement between representations within the same batch while ensuring their distinctiveness from other representations. This approach enhances model robustness and facilitates more effective feature extraction and representation learning.

(3)We constructed a comprehensive Chinese corpus and lexicon for identifying specific terms in pig diseases domain. Moreover, we built an annotated datasets encompassing 25 distinct types of pig diseases and 6 entity categories, comprising a total of 7518 annotated entities.

The remainder of the paper is organized as follows: Section 2 introduces the data set and method proposed in this paper. The experiments and results are described in Section 3. Section 4 compares the method proposed in this paper with other commonly used methods and provides an analysis of the experimental results. Finally, the conclusion are presented in Section 5.

2. Materials and Methods

2.1. Materials

2.1.1. Corpus Collection and Pre-Processing

Due to the lack of NER public benchmark datasets in pig disease domain, a new Chinese pig disease corpus was constructed and annotated under the guidance of animal disease experts and veterinarians.To ensure the quality of data, we collected information on pig diseases from professional books, published standards, Baidu Encyclopedia and official websites. The data source details are mentioned in Appendix A.

After the data acquisition, we performed basic data per-processing steps. Firstly, the optical character recognition(OCR) technology was used to convert the books and standards into text format. Secondly, useless data such as garbled characters or special symbols, were manually deleted and wrong words were modified in raw text. Thirdly, the duplicate data and invalid data were removed. Ultimately, an comprehensive and effective text corpus of containing 1.45 million characters was obtained(Corpus Ⅰ of pig disease).

2.1.2. Corpus Annotation

152,596 characters were selected from Corpus Ⅰ to form Corpus Ⅱ for entity labeling. Label Studio tool and BIEO labeling method were used to label entities. B represents the start position of the entity, I represents the inside of the entity, E represents the end position of the entity, and O represents the other. Six types of pig disease such as pig type, disease name, body part, symptom, medicine, prevention and control measures were labeled in the corpus text. Finally, the annotated pig disease corpus containing 7518 entities was obtained under the guidance of pig disease experts. The statistical information of labeled entities is presented in Table 1.

2.1.3. Construction of Lexicon and Pre-Training Word Embedding

We constructed lexicon in pig disease domain based on Corpus Ⅱ and professional books. Firstly, the most commonly used professional terms were extracted from the Corpus Ⅱ by word segmentation and frequency statistics. Then, some professional words in the glossary of professional books were manually added to the dictionary under the guidance of veterinarians, such as “猪副嗜血杆菌病(Haemophilus parasuis)”,“噻苯达唑(Thiabendazole)”,“内阿米巴原虫(Entamoeba spp)”. Finally, the pig disease lexicon comprising 2391 professional terms was obtained for understanding the specific words and technical term. Subsequently, this lexicon was incorporated into the built-in dictionary of Jieba to avoid incorrect segmentation of words.

To obtain a high-quality embedded representation of pig diseases, we trained Corpus Ⅰ and the lexicon.The Gensim tool was used to train Word2Vec model with a word vector dimension of 200. The construction process of the lexicon and the pre-training of word embeddings are illustrated in Figure 1.

2.2. Methods

2.2.1. Framework

The framework of PDCNER model is shown in Figure 2. Firstly, the Chinese sentences in the pig disease corpus are converted into a character-words pair sequence, and both Chinese character features and lexicon features are used as inputs. Secondly, a lexicon adapter is added between Transformer layers, which is used to dynamically extract the most relevant matching items. The word of each character uses the bi-linear attention mechanism from character to word, and the lexicon adapter is applied between adjacent Transformer in BERT. The lexicon features and BERT representations are fully interacted through multi-layer encoders in BERT, so that lexicon knowledge can be effectively integrated into BERT. The contrastive loss layer is above the Lexicon Enhanced BERT encoder, ensuring that similar samples are as close as possible, while dissimilar samples are as far apart as possible. Embeddings of the same type of entity are treated as positive samples, whereas embeddings of different types of entities are treated as negative samples. Considering the correlation between consecutive labels, a Conditional Random Field (CRF) layer is employed to label the sequence.

2.2.2. Char-Words Pair Sequence

According to the Lexicon Enhanced BERT in [13], we firstly expand the character sequence into a sequence of character-word pairs for applying lexical information of pig disease.

A Chinese sentence with n characters, ${\mathrm{S}}_{\mathrm{C}}=\left\{{{\mathrm{c}}_1,\mathrm{c}}_2,\cdots ,{\mathrm{c}}_{\mathrm{n}}\right\}$. We identify all potential words within the sentence by comparing the character sequence with the lexicon of pig disease. To achieve this, we first create a Trie data structure based on the lexicon. Then we examine all character subsequences within the sentence, matching them with the Trie to identify all potential words. For instance, the truncated sentence "非洲猪瘟 (African swine fever)" as an example. We can identify five distinct words: "非洲 (Africa)", "非洲猪 (African pig)", "猪瘟 (swine fever)", "瘟 (epidemic disease)" and "非洲猪瘟 (African swine fever)" . In the field of pig disease, African swine fever is a complete disease name and should not be separated. Following this, for each matched word, we associate it with the characters that compose it. In conclusion, we pair each character with its associated words and transform the Chinese sentence into a sequence of character-word pairs, represented as:

$${\mathrm{S}}_{\mathrm{C}\mathrm{w}}=\left\{({{\mathrm{c}}_1,{\mathrm{w}\mathrm{s}}_1),(\mathrm{c}}_2,{\mathrm{w}\mathrm{s}}_2),\cdots ,({\mathrm{c}}_{\mathrm{n}},{\mathrm{w}\mathrm{s}}_{\mathrm{n}})\right\}$$

(1)

where ${\mathrm{c}}_{\mathrm{i}}$ denotes the i-th character in the sentence, and ${\mathrm{w}\mathrm{s}}_{\mathrm{i}}$ signifies the words matched and assigned to ${\mathrm{c}}_{\mathrm{i}}$.

2.2.3. Lexicon Adapter

Using the lexicon adapter proposed in LEBERT [13], the pig disease lexicon information is directly injected into BERT for integrating lexical features.

For the i-th character in a character-word sequence, the input is$\left(h_{\mathrm{i}}^{\mathrm{c}},{\mathrm{x}}_{\mathrm{i}}^{\mathrm{w}\mathrm{s}}\right)$, $h_{\mathrm{i}}^{\mathrm{c}}$ represents the character vector, the output of a transformation layer in BERT. ${\mathrm{x}}_{\mathrm{i}}^{\mathrm{w}\mathrm{s}}=\left\{{\mathrm{x}}_{\mathrm{i}1}^{\mathrm{w}},{\mathrm{x}}_{\mathrm{i}2}^{\mathrm{w}},\cdots ,{\mathrm{x}}_{\mathrm{i}\mathrm{m}}^{\mathrm{w}}\right\}$ represents a group of words embeddings. The j-th word in ${\mathrm{x}}_{\mathrm{i}}^{\mathrm{w}\mathrm{s}}$ is represented as following:

$${\mathrm{x}}_{\mathrm{i}\mathrm{j}}^{\mathrm{w}}={\mathrm{e}}^{\mathrm{w}}\left({\mathrm{w}}_{\mathrm{i}\mathrm{j}}\right)$$

(2)

where ${\mathrm{e}}^{\mathrm{w}}$ is the pre-trained word embedding list and ${\mathrm{w}}_{\mathrm{i}\mathrm{j}}$ represents the j-th word in${\mathrm{w}\mathrm{s}}_{\mathrm{i}}$.

To align these two different representations, a nonlinear transformation is used for each word vector:

$${\mathrm{v}}_{\mathrm{i}\mathrm{j}}^{\mathrm{w}}={\mathrm{W}}_2\left(\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathrm{W}}_1{\mathrm{x}}_{\mathrm{i}\mathrm{j}}^{\mathrm{w}}+{\mathrm{b}}_1\right)\right)+{\mathrm{b}}_2$$

(3)

where ${\mathrm{W}}_1\in {\mathrm{R}}^{{\mathrm{d}}_{\mathrm{c}}\times {\mathrm{d}}_{\mathrm{w}}}$, ${\mathrm{W}}_2\in {\mathrm{R}}^{{\mathrm{d}}_{\mathrm{c}}\times {\mathrm{d}}_{\mathrm{c}}}$, ${\mathrm{d}}_{\mathrm{w}} \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{d}}_{\mathrm{c}}$represent the dimension of word embedding and BERT's hidden size respectively. ${\mathrm{b}}_1 \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{b}}_2$ are scaler bias.

Each character is associated with a variety of words, but the degree of contribution from each word differs. For instance, in the field of pig diseases, the words "非洲(Africa)" and "猪瘟(swine fever)" are more important than "非洲猪(African pigs)" and "瘟(epidemic disease)".In order to find the most relevant words, the character-word attention mechanism is used. The correlation of each word is calculated as follows:

$${\mathrm{a}}_{\mathrm{i}}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\mathrm{h}}_{\mathrm{i}}^{\mathrm{c}}{\mathrm{W}}_{\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{n}}{\mathrm{V}}_{\mathrm{i}}^{\mathrm{T}}\right)$$

(4)

where ${\mathrm{W}}_{\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{n}}\in {\mathrm{R}}^{{\mathrm{d}}_{\mathrm{c}}\times {\mathrm{d}}_{\mathrm{c}}}$ is the bi-linear attention mechanism. The all ${\mathrm{v}}_{\mathrm{i}\mathrm{j}}^{\mathrm{w}}$ assigned to i-th character ${\mathrm{V}}_{\mathrm{i}}=\left({\mathrm{v}}_{\mathrm{i}1}^{\mathrm{w}},\cdots ,{\mathrm{v}}_{\mathrm{i}\mathrm{m}}^{\mathrm{w}}\right)$, where m is the total number of assigned words.

Lastly, the lexicon information is integrated into the vector representation of the character.

$${{\tilde{\mathrm{h}}}_{\mathrm{i}}=\mathrm{h}}_{\mathrm{i}}^{\mathrm{c}}+\sum _{\mathrm{j}=1}^{\mathrm{m}}{\mathrm{a}}_{\mathrm{i}\mathrm{j}}{\mathrm{v}}_{\mathrm{i}\mathrm{j}}^{\mathrm{w}}$$

(5)

2.2.4. Lexicon Enhanced BERT

The lexicon adapter is attached between transformer layers in BERT so that the knowledge of pig disease lexicon can be injected into BERT. A sequence of characters $\left\{{{\mathrm{c}}_1,\mathrm{c}}_2,\cdots ,{\mathrm{c}}_{\mathrm{n}}\right\}$ is input into the input embedding of BERT and then $\mathrm{E}=\left\{{{\mathrm{e}}_1,\mathrm{e}}_2,\cdots ,{\mathrm{e}}_{\mathrm{n}}\right\}$ is obtained by adding token, segmentation and position embedding . After that , E is input into the Transformer encoders. Each layer is as follows.

$$\begin{array}{c}\mathrm{G}=\mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left({\mathrm{H}}^{\mathrm{l}-1}+Multiheadattention\left({\mathrm{H}}^{\mathrm{l}-1}\right)\right)\\ {\mathrm{H}}^{\mathrm{l}}=\mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{G}+\mathrm{F}\mathrm{F}\mathrm{N}\left(\mathrm{G}\right)\right)\end{array}$$

(6)

Where ${\mathrm{H}}^{\mathrm{l}}=\left\{{\mathrm{h}}_1^{\mathrm{l}},{\mathrm{h}}_2^{\mathrm{l}},\cdots ,{\mathrm{h}}_{\mathrm{n}}^{\mathrm{l}}\right\}$ represents the output of l-th layer and ${\mathrm{H}}^0$=E. FFN represents the two-layer feed-forward network with RELU as the hidden activation function.

The lexicon information were input between the k-th and (k+1)-th layer Transformer. ${\mathrm{H}}^{\mathrm{k}}=\left\{h_1^{\mathrm{k}},h_2^{\mathrm{k}},\cdots ,h_{\mathrm{n}}^{\mathrm{k}}\right\}$ are got first after k consecutive Transformers layers. Subsequently, each character-word pair $\left(h_{\mathrm{i}}^{\mathrm{k}},{\mathrm{x}}_{\mathrm{i}}^{\mathrm{w}\mathrm{s}}\right)$ was processed through the lexicon adapter to obtain a new hidden layer representation and the ${\mathrm{i}}_{\mathrm{t}h}$ pair was converted into ${\tilde{h}}_{\mathrm{i}}^{\mathrm{k}}$ accordingly.

$${\tilde{h}}_i^k=\mathrm{L}\mathrm{e}\mathrm{x}\mathrm{i}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{A}\mathrm{d}\mathrm{a}\mathrm{p}\mathrm{t}\mathrm{e}\mathrm{r}\left(h_i^k,x_i^{ws}\right)$$

(7)

${\mathrm{H}}^{\mathrm{k}}=\left\{h_1^{\mathrm{k}},h_2^{\mathrm{k}},\cdots ,h_{\mathrm{n}}^{\mathrm{k}}\right\}$ are input into the remaining (L-K) Transformer as there are 12 layers of Transformers in BERT. Finally, the output of L-th Transformer ${\mathrm{H}}^{\mathrm{L}}$ used for the name entity recognition task is obtained.

2.2.5. Lexicon Enhanced Contrastive Learning

The normalized temperature-scaled cross-entropy loss, denoted as NT-Xent was used as our contrastive loss function [40]. For each training iteration, we randomly select N texts from the datasets to form a mini-batch, which yields 2N feature representations. The model is then trained to identify each data point's corresponding pair among the 2(N-1) negative samples present within the batch.

$${\mathcal{L}}_{\mathrm{i},\mathrm{j}}=-\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\right({\mathrm{r}}_{\mathrm{i}},{\mathrm{r}}_{\mathrm{j}})/\mathsf{{\rm T}})}{{\sum }_{\mathrm{k}}^{2\mathrm{N}}{\mathrm{I}}_{\left[\mathrm{k}\ne \mathrm{i}\right]}\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\right({\mathrm{r}}_{\mathrm{i}},{\mathrm{r}}_{\mathrm{k}})/\mathsf{{\rm T}})}$$

(8)

Where r_i, r_j represent the embedding for entities of the same type, whereas r_i, r_k denote the embedding for entities of different types. s() refers to the cosine similarity function, where $\mathsf{{\rm T}}$ acts as the temperature parameter, and I serves as an indicator function. Ultimately, we calculate the final contrastive loss by averaging the classification losses of all 2N instances within the batch.

3. Experiment

We carry out experiments to investigate the effectiveness of PDCNER. This section presents the results and evaluation of the proposed PDCNER model illustrated on pig disease corpus.

3.1. Evaluation

To identify a named entity for pig diseases, it is necessary to correctly identify both the boundaries of the entity and its corresponding categories. The proposed PDCNER model is evaluated using standard measures, Precision(P), Recall(R), and F1, which are computed using the Eqs. (9), (10) and (11).

$$P=\frac{T_P}{T_P+F_P}\times 100\%$$

(9)

$$R=\frac{T_P}{T_P+F_n}\times 100\%$$

(10)

$$F1=\frac{2PR}{P+R}\times 100\%$$

(11)

where Tp represents the number of positive samples that are accurately predicted, while Fp denotes the count of positive samples that are inaccurately predicted. Additionally, Fn stands for the number of negative samples that are incorrectly predicted.

3.2. Experimental Settings

Experiments. The hardware environment that the experimental research relied on was Intel(R) Xeon(R) Silver4116 CPU@2.10GHz, GPU@NVIDIA Tesla P100. The software environment was Python3.8 and tensorflow 2.0.0. The model parameters were set as follows: based on BERT_BASE (Devlin et al., 2019) [17] version, with 12 transformer layers, 768 hidden layers, and 12 multi-head attention mechanisms. The lexicon corresponds to the vocabulary of pre-trained word embeddings in field of pig disease. During the training process, we incorporate the Lexicon Adapter between the first and second Transformer layers of BERT [13]. Meanwhile, the parameters of BERT and the pre-trained word embedding were fine-tuned.

Hyperparameters.The learning rate was 1e-5, the training batch_size was 16, the dropout was 0.5, the optimizer chose Adam, and the number of iterations was 100.

Dataset.This study randomly divided the datasets into training, validation, and test sets according to a ratio of 7:2:1.

3.3. Results

We evaluate the effectiveness of PDCNER against widely-used NER models on the pig disease corpus. The overall findings of our experiments are presented in Table 2. As shown in Table 2, the recognition effect based on the model proposed in this study was significantly better than the other models, with a micro average F1-score of 85.99% in recognizing six major entities.

Compared with the baseline model BERT_BILSTM_CRF, F1-score was increased by 2.67 percentage points. This is due to the fact that PDCNER makes full use of Lexicon enhanced BERT with the help of Lexicon Adapter layer and obtains optimization of feature representation with the help of contrastive learning.

Obviously, the F1-score of the BILSTM_CRF model is the lowest among all the results. Generally, the F1 values of pre-trained models are higher than those of the BILSTM_CRF model, as the pre-trained models have learned more language features and possess a deeper network structure. Moreover, the pre-trained model incorporating lexicon information demonstrates superior performance due to the additional lexical knowledge.

4. Discussion

4.1. Performance Analysis of the Proposed Model

For better understanding of the proposed approach, we evaluate the PDCNER model separately on the six entities type, disease, body parts, symptom, medicine, control, which are presented in Figure 3 and Table 3.

We found that the F1-scores for type, disease, and medicine all exceeded 90%, with the F1-score for type being the highest at 95.41%. Conversely, the lowest F1-score was for the entity of control, at only 63.16%. The primary reason for this disparity is that the boundaries of pig type and disease entities are very clear, whereas the boundaries of control measures entities are more ambiguous. For instance, type entities typically end with terms like 'pigs (猪)' (e.g., sick pigs (患病猪), conservation pigs (保育猪), fattening pigs (育肥猪)), while disease entities usually end with terms such as 'disease (病),' 'inflammation (炎),' and 'plague (瘟)' (e.g., Porcine blue ear disease (猪蓝耳病), Necrotic enteritis (坏死性肠炎), African swine fever (非洲猪瘟)). In contrast, control measure entities are generally composed of verbs and nouns, such as 'isolating infected pigs(隔离感染猪群)' and 'reducing environmental stress factors(减少环境应激因素)'. The second reason is the uneven distribution of entities. Control entities in the training set are significantly fewer than other categories, comprising only 7.29%. Consequently, the model could not fully learn their contextual features. Additionally, the average length of control entities is 11 Chinese characters, which contributes to a low overall recognition rate.

On the other hand, the F1-scores of disease entities and medicine entities were 92.96% and 90.05%, respectively. Both disease and medicine entities include a large number of technical terms, yet the method proposed in this paper achieves a good recognition effect on these two entities. The results demonstrate that PDCNER fully utilizes both Chinese character features and lexicon knowledge in the pig disease domain at the input level, and the lexicon adapter can effectively leverage pig disease knowledge.

4.2. Comparison of Common Pre-Trained and Lexicon-Based Model

In contrast to models such as BERT-BiLSTM-CRF, BERT-BILSTM-CRF-SoftLexicon, RoBERTa and LEBERT, PDCNER has demonstrated significant advancements, confirming its efficiency. PDCNER holds a distinct advantage over other pre-trained and lexicon-based models, illustrating the value of incorporating pig disease-related lexicon features directly into the BERT representation from the bottom layer and using the contrastive learning method.

For the comparison of other models, we present the recognition results for six entity types in Figure 4. It can be clearly seen from the Figure 4 that PDCNER has achieved the best results in 4 entities such as type, body parts, symptom and medicine, which exhibit robust domain-specific features.

(1)

Effectiveness of the lexicon Enhanced BERT

Comparative analysis with the results of BERT-BiLSTM-CRF reveals notable improvement in the precision, recall, and F1-score of PDCNER, with improvements of 5.98 percentage points, 0.04 percentage points, and 3.05 percentage points, respectively. PDCNER leverages the lexicon adapter to make full use of pig disease feature information, seamlessly integrating it into the BERT architecture. Specifically, the Lexicon Adapter is attached between certain transformers within BERT, facilitating the infusion of pig disease lexicon knowledge into the model's representation.

(2)

Effectiveness of the contrastive learning

Through comparative evaluation utilizing the same dataset and downstream model, PDCNER demonstrates superior accuracy in identifying pig disease entities compared to LEBERT, exhibiting improvements in precision, recall, and F1-score by 0.45 percentage points, 0.44 percentage points, and 0.44 percentage points, respectively. This underscores the efficacy of the loss function employed in contrastive learning, which enhances the model's capacity for semantic representation of text. Consequently, this optimization contributes to superior performance across various NER tasks.

4.3. Analysis of Results for Few-Shot

In order to verify the reliability and robustness of PDCNER in the condition of scarce data for few-shot entity recognition, we used 1%, 10% and 30% of pig disease corpus for experimentation. The results can be found in Table 4. The result shows that the PDCNER model has obvious improvement compared to BERT-BILSTM-CRF and LEBERT.

The F1-score of PDCNER reaches 84.77% when the sample size is 10%, which is only 1.22% lower than that of the full sample. As the sample size increases to 30%, the F1-score of the PDCNER model further improves to 85.39%, showing a marginal decrease of only 0.6% compared to the full sample. Moreover, it outperforms the BERT-BiLSTM-CRF and LEBERT models by 6.38% and 8.42%, respectively. These results demonstrate the PDCNER model's capability to achieve higher recognition accuracy even under data scarcity scenarios. The incorporation of lexical information in the bottom layer of BERT enables efficient utilization of BERT's representational capabilities. Additionally, the adoption of contrastive learning enhances the semantic representation space, facilitating effective feature capture without extensive training.

4.4. Experiments on Public Datasets

To assess the generalization capability of PDCNER, we conducted evaluations across three public datasets: Weibo, Ontonotes, and Resume. As illustrated in Table 5, the PDCNER model achieved the highest F1 score across all three datasets. These results indicate that PDCNER exhibits superior performance not only on the pig disease corpus but also demonstrates a degree of generalizability to other domains.

5. Conclusions

High-quality extraction of knowledge related to pig diseases is critical for intelligent consultation, question answering, technical recommendations, and other application scenarios.

In this study, we constructed a corpus, labeled datasets and lexicon for Chinese named entity recognition specific to pig diseases, encompassing 152,596 characters, 7,518 entities and 2,391 professional terms. To tackle the challenges of entity identification in the pig disease domain, such as the scarcity of annotated data, numerous technical terms, and fuzzy boundaries, we propose the PDCNER model. This model integrates lexicon information from the pig disease domain into the BERT's Transformer layers at the lower level and employs contrastive learning to enhance representation quality and generalization capability. The results indicate that the PDCNER model surpasses the performance of BERT-BiLSTM-CRF and other mainstream models in extracting named entities related to pig diseases, achieving accuracy, recall, and F1-score of 86.92%, 85.08%, and 85.99%, respectively. This demonstrates high-quality entity recognition in the field of pig diseases. Moreover, few-shot experiments confirm that our model remains robust with limited data, and experiments on public datasets verify its generalization ability.

In future work, we plan to utilize additional datasets from other related animal diseases, such as chicken and cow diseases, to further test the scalability and generalization ability of the model.