Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Generative Artificial Intelligence in Chemistry Problem Solving across Versions and Languages

Version 1 : Received: 18 February 2024 / Approved: 19 February 2024 / Online: 19 February 2024 (16:15:53 CET)

How to cite: Daher, W.; Diab, H.; Rayan, A. Generative Artificial Intelligence in Chemistry Problem Solving across Versions and Languages. Preprints 2024, 2024021007. https://doi.org/10.20944/preprints202402.1007.v1 Daher, W.; Diab, H.; Rayan, A. Generative Artificial Intelligence in Chemistry Problem Solving across Versions and Languages. Preprints 2024, 2024021007. https://doi.org/10.20944/preprints202402.1007.v1

Abstract

Generative artificial intelligence tools are flourishing across various sciences, particularly in the educational domain. ChatGPT, with its versions 3.5 and 4.0, stands out for its multilingual capabilities, catering to both students and educators. This research aimed to evaluate the performance differences between these versions, and importantly, to compare the accuracy of ChatGPT's responses in English versus Arabic. In our study, we presented ChatGPT with 39 chemistry problems related to 6th-7th grade curriculum, including twenty-five open questions and fourteen multiple-choice questions. Each response was categorized as accurate, partially accurate, or inaccurate. Our analysis, focused on trends across versions and languages, revealed significant enhancements in version 4.0, particularly in its ability to process Arabic. However, despite these improvements, our findings indicate that responses in English consistently outperformed those in Arabic in terms of accuracy. In light of this discrepancy, we suggest that ChatGPT either incorporates additional Arabic databases into its training regimen or develops a code that adopts the following strategy to ensure more accurate responses in science education: questions should first be translated from Arabic to English, addressed, and then retranslated into Arabic before being presented to the user. This approach takes advantage of the greater accuracy of responses in English to benefit Arabic-speaking users, and it may also improve outcomes for speakers of other languages who experience less precise answers in the fields of chemistry or science education.

Keywords

Artificial intelligence; generative artificial intelligence; chemistry; problem solving; ChatGPT

Subject

Social Sciences, Education

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.