Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Exploring the Cognitive Neural Basis of Factuality in Abstractive Text Summarization Models: Interpretable Insights from EEG Signals

Zhejun Zhang
,
Yingqi Zhu
,
Yubo Zheng
,
Yingying Luo
ORCID logo ,
Hengyi Shao
,
Shaoting Guo
ORCID logo ,
Liang Dong
ORCID logo ,
Lin Zhang
* ,
Lei Li
* ORCID logo
Version 1 : Received: 27 November 2023 / Approved: 27 November 2023 / Online: 27 November 2023 (14:58:56 CET)

A peer-reviewed article of this Preprint also exists.

Zhang, Z.; Zhu, Y.; Zheng, Y.; Luo, Y.; Shao, H.; Guo, S.; Dong, L.; Zhang, L.; Li, L. Exploring the Cognitive Neural Basis of Factuality in Abstractive Text Summarization Models: Interpretable Insights from EEG Signals. Appl. Sci. 2024, 14, 875. Zhang, Z.; Zhu, Y.; Zheng, Y.; Luo, Y.; Shao, H.; Guo, S.; Dong, L.; Zhang, L.; Li, L. Exploring the Cognitive Neural Basis of Factuality in Abstractive Text Summarization Models: Interpretable Insights from EEG Signals. Appl. Sci. 2024, 14, 875.

Abstract

(1) Background: Information overload challenges decision-making in the Industry 4.0 era. While Natural Language Processing (NLP), especially Automatic Text Summarization (ATS), offers solutions, issues with factual accuracy persist. This research bridges cognitive neuroscience and NLP, aiming to improve model interpretability. (2) Methods: This research examined four fact extraction techniques: dependency relation, named entity recognition, part-of-speech tagging, and TF-IDF, in order to explore their correlation with human EEG signals. Representational Similarity Analysis (RSA) was applied to gauge the relationship between language models and brain activity. (3) Results: Named entity recognition showed the highest sensitivity to EEG signals, marking the most significant differentiation between factual and non-factual words with a score of -0.99. The dependency relation followed with -0.90, while part-of-speech tagging and TF-IDF resulted in 0.07 and -0.52, respectively. Deep language models such as GloVe, BERT, and GPT-2 exhibited noticeable influences on RSA scores, highlighting the nuanced interplay between brain activity and these models. (4) Conclusions: Our findings emphasize the crucial role of named entity recognition and dependency relations in fact extraction and demonstrate the independent effects of different models and TOIs on RSA scores. These insights aim to refine algorithms to reflect human text processing better, thereby enhancing ATS models’ factual integrity.

Keywords

natural language processing (NLP); abstractive summarization (ABS); factual extraction; Electroencephalography (EEG); Representational similarity analysis (RSA)

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download PDF

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0
Get PDF Cite Share 0
Bookmark BibSonomy Mendeley Reddit Delicious
×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.