Version 1
: Received: 24 July 2023 / Approved: 25 July 2023 / Online: 25 July 2023 (13:09:30 CEST)
How to cite:
Galitsky, B.A. Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations. Preprints2023, 2023071723. https://doi.org/10.20944/preprints202307.1723.v1
Galitsky, B.A. Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations. Preprints 2023, 2023071723. https://doi.org/10.20944/preprints202307.1723.v1
Galitsky, B.A. Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations. Preprints2023, 2023071723. https://doi.org/10.20944/preprints202307.1723.v1
APA Style
Galitsky, B.A. (2023). Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations. Preprints. https://doi.org/10.20944/preprints202307.1723.v1
Chicago/Turabian Style
Galitsky, B.A. 2023 "Truth-O-Meter: Collaborating with LLM in Fighting its Hallucinations" Preprints. https://doi.org/10.20944/preprints202307.1723.v1
Abstract
A text obtained by a Large Language Model (LLM) such as GPT4 usually has issues in terms of incorrectness and hallucinations. We build a fact-checking system 'Truth-O-Meter' which identifies wrong facts, comparing the generation results with the web and other sources of information, and suggests corrections. Text mining and web mining techniques are leveraged to identify correct corresponding sentences; also, the syntactic and semantic generalization procedure adopted to the content improvement task. To handle inconsistent sources while fact-checking, we rely on an argumentation analysis in the form of defeasible logic programming. We compare our fact checking engine with competitive approach based on reinforcement learning on top of LLM or token-based hallucination detection. It is observed that LLM content can be substantially improved for factual correctness and meaningfulness.
Keywords
Large Language Model; hallucination; fact-checking; multiple inconsistent sources
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
