Version 1
: Received: 26 September 2023 / Approved: 27 September 2023 / Online: 28 September 2023 (03:01:30 CEST)
How to cite:
Marcińczuk, M. Improving Stability of Transformer-based Named Entity Recognition Models with Combined Data Representation. Preprints2023, 2023091859. https://doi.org/10.20944/preprints202309.1859.v1
Marcińczuk, M. Improving Stability of Transformer-based Named Entity Recognition Models with Combined Data Representation. Preprints 2023, 2023091859. https://doi.org/10.20944/preprints202309.1859.v1
Marcińczuk, M. Improving Stability of Transformer-based Named Entity Recognition Models with Combined Data Representation. Preprints2023, 2023091859. https://doi.org/10.20944/preprints202309.1859.v1
APA Style
Marcińczuk, M. (2023). Improving Stability of Transformer-based Named Entity Recognition Models with Combined Data Representation. Preprints. https://doi.org/10.20944/preprints202309.1859.v1
Chicago/Turabian Style
Marcińczuk, M. 2023 "Improving Stability of Transformer-based Named Entity Recognition Models with Combined Data Representation" Preprints. https://doi.org/10.20944/preprints202309.1859.v1
Abstract
This study leverages transformer-based models and focuses on data representation strategies in the named entity recognition task, including "single" (one sentence per vector), "merged" (multiple sentences per vector), and "context" (sentences joined with attention to context). Performance analysis reveals that models trained with a single strategy may not perform well on different data representations. A combined training procedure is proposed to address this limitation, using all three strategies to enhance the stability and adaptability of the model. The results of this approach are presented and discussed for various datasets for four languages (English, Polish, Czech, and German), demonstrating the effectiveness of the combined strategy.
Keywords
named entity recognition; deep learning; transformers; data augmentation; pre-trained language models; PLM
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
