Farhat, F.; Sohail, S.S.; Madsen, D.Ø. How Trustworthy Is ChatGPT? The Case of Bibliometric Analyses. Cogent Engineering 2023, 10, doi:10.1080/23311916.2023.2222988.
Farhat, F.; Sohail, S.S.; Madsen, D.Ø. How Trustworthy Is ChatGPT? The Case of Bibliometric Analyses. Cogent Engineering 2023, 10, doi:10.1080/23311916.2023.2222988.
Farhat, F.; Sohail, S.S.; Madsen, D.Ø. How Trustworthy Is ChatGPT? The Case of Bibliometric Analyses. Cogent Engineering 2023, 10, doi:10.1080/23311916.2023.2222988.
Farhat, F.; Sohail, S.S.; Madsen, D.Ø. How Trustworthy Is ChatGPT? The Case of Bibliometric Analyses. Cogent Engineering 2023, 10, doi:10.1080/23311916.2023.2222988.
Abstract
The introduction of the AI-powered chatbot ChatGPT by OpenAI has sparked much interest and debate among academic researchers. Commentators from different scientific disciplines have raised many concerns and issues, especially related to the ethics of using these tools in scientific writing and publications. In addition, there has been discussions about whether ChatGPT is trustworthy, effective, and useful in increasing researchers’ productivity. Therefore, in this paper, we evaluate ChatGPT’s performance on tasks related to bibliometric analysis, by comparing the output provided by the chatbot with a recently conducted bibliometric study on the same topic. The findings show that there are large discrepancies and ChatGPT’s trustworthiness is low in this particular area. Therefore, researchers should exercise caution when using ChatGPT as a tool in bibliometric studies.
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Received:
31 March 2023
Commenter:
Deep Kumar Kirtania
The commenter has declared there is no conflict of interests.
Comment:
Very well done, congratulation to all of you for this excellent research paper.I have also done two similar paper about this topic. If you are interested, you can view these two papers. Thank youOpenAI ChatGPT Generated Content and Similarity Index: A study of selected terms from the Library & Information Science (LIS) disciplinehttps://www.qeios.com/read/FO1CP6 ChatGPT as a Tool for Bibliometrics Analysis: Interview with ChatGPThttps://papers.ssrn.com/sol3/papers.cfm?abstract_id=4391794
The commenter has declared there is no conflict of interests.
Comment:
As far as I know, GPT is not currently connected to internet resources (ChatGPT and GPT 3.X). Therefore, a task focused on bibliometric analysis using web resources such as WoS or Scopus is actually pointless, as GPT cannot accomplish this in the version used.
The commenter has declared there is no conflict of interests.
Comment:
Ofcourse. ChatGPT can access these internet resources like WOS and SCOPUS databases upto 2021 and we have tasked ChatGPT to fetch the data upto 2021 only. Indeed it can fetch very relevant data from these databases but when inquire for bibliometric analysis it glitched somehow or couldn't sum up the data for bibliometric analysis. Moreover, it never declines the query even it couldn't able to process right information and repetitively fetched the same incorrect data.
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Commenter: Deep Kumar Kirtania
The commenter has declared there is no conflict of interests.
Commenter:
The commenter has declared there is no conflict of interests.
Commenter:
The commenter has declared there is no conflict of interests.
Commenter:
The commenter has declared there is no conflict of interests.