GPT-4 Can't Reason

Konstantine Arkoudas

doi:10.20944/preprints202308.0148.v2

Submit preprint
How it Works
Instructions for Authors
Subject Areas
Advisory Board
screening preprints
About
Awards
Log in/Register

Instructions for Authors
Awards
About
FAQ
Submit Log in/Register

Share this article with

Create alert

Email:

Load new image

preprints.org > computer science and mathematics > artificial intelligence and machine learning > doi: 10.20944/preprints202308.0148.v2

Preprint Article Version 2 Preserved in Portico This version is not peer-reviewed

GPT-4 Can't Reason

Konstantine Arkoudas

Version 1 : Received: 27 July 2023 / Approved: 2 August 2023 / Online: 2 August 2023 (04:10:57 CEST)
Version 2 : Received: 3 August 2023 / Approved: 4 August 2023 / Online: 7 August 2023 (07:13:00 CEST)

How to cite: Arkoudas, K. GPT-4 Can't Reason. Preprints 2023, 2023080148. https://doi.org/10.20944/preprints202308.0148.v2 Arkoudas, K. GPT-4 Can't Reason. Preprints 2023, 2023080148. https://doi.org/10.20944/preprints202308.0148.v2

Cite

Export citation file: Bib TeX | RIS

MDPI and ACS Style

Arkoudas, K. GPT-4 Can't Reason. Preprints 2023, 2023080148. https://doi.org/10.20944/preprints202308.0148.v2

APA Style

Arkoudas, K. (2023). GPT-4 Can't Reason. Preprints. https://doi.org/10.20944/preprints202308.0148.v2

Chicago/Turabian Style

Arkoudas, K. 2023 "GPT-4 Can't Reason" Preprints. https://doi.org/10.20944/preprints202308.0148.v2

Abstract

GPT-4 was released in March 2023 to wide acclaim, marking a very substantial improvement across the board over GPT-3.5 (OpenAI's previously best model, which had powered the initial release of ChatGPT). Despite the genuinely impressive improvement, however, there are good reasons to be highly skeptical of GPT-4's ability to reason. This position paper discusses the nature of reasoning; criticizes the current formulation of reasoning problems in the NLP community and the way in which the reasoning performance of LLMs is currently evaluated; introduces a collection of 21 diverse reasoning problems; and performs a detailed qualitative analysis of GPT-4's performance on these problems. Based on the results of this analysis, the paper argues that, despite the occasional flashes of analytical brilliance, GPT-4 at present is utterly incapable of reasoning.

Keywords

GPT-4; LLM; AI; reasoning; inference

Subject

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download PDF Download Supplementary

Comments (1)

Comment 1

Received: 7 August 2023
Commenter: Konstantine Arkoudas
Commenter's Conflict of Interests: Author

Comment: Fixed a few typos and spelling errors that had slipped through the cracks in the first version.

+ Respond to this comment

Not displayed online.

Importance: How significant is the paper to the field?

Outstanding/highlight paper

Significant contribution

Incremental contribution

No contribution

I am not qualified to judge

Soundness of evidence/arguments presented:

Conclusions well supported

Most conclusions supported (minor revision needed)

Incomplete evidence (major revision needed)

Hypothesis, unsupported conclusions, or proof-of-principle

I am not qualified to judge

Please leave your comment to this article below.

Mathematical equations can be typed in either LaTeX formats \\[ ... \\] or $$ ... $$, or MathML format <math> ... </math>. Try the LaTeX or MathML example.

Type equation:

Preview:

Copy equation into message below

Close menu

Please click a symbol to insert it into the message box below:

Please enter the link here:

Please enter a valid URL

Optionally, you can enter text that should appear as linked text:

Copy link into message below

Close menu

Please enter or paste the URL to the image here (please only use links to jpg/jpeg, png and gif images):

Please enter a valid URL

Copy image into message below

Close menu

Type author name or keywords to filter the list of references in this group (you can add a new citation under Bibliography):

No existing citations in Discussion Group

Wikify editor is a simple editor for wiki-style mark-up. It was written by MDPI for Sciforum in 2014. The rendering of the mark-up is based on Wiky.php with some tweaks. Rendering of mathematical equations is done with MathJax. Please send us a message for support or for reporting bugs.

Close menu

Please declare any conflicts of interest you may have with this paper, financial or otherwise.

I do not have a conflict of interest

I would like to declare a conflict of interest

Display my name on the website Send my name and email address to the authors

Captcha

Renew

I accept the following conditions:

Comments must follow the standards of professional discourse and should focus on the scientific content of the article. Insulting or offensive language, personal attacks and off-topic remarks will not be permitted. Comments must be written in English. Preprints reserves the right to remove comments without notice. Readers who post comments are obliged to declare any competing interests, financial or otherwise.

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)

* All users must log in before leaving a comment

what’s this?

Add a record of this review to Publons to track and showcase your reviewing expertise across the world’s journals.

Downloads 0

Comments 1

Metrics 0

Get PDF Supplementary Files Cite Share 3

Bookmark BibSonomy Mendeley Reddit Delicious

Alerts

Notify me about updates to this article or when a peer-reviewed version is published.

Preprints.org is a free preprint server subsidized by MDPI in Basel, Switzerland.

MDPI Initiatives

SciProfiles
Sciforum
Encyclopedia
MDPI Books
Scilit
Proceedings
JAMS

Important links

How it Works
Advisory Board
FAQ
Friendly Journals
Instructions for Authors
About
Statistics

Choose the area that interest you and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.

RSS feed for

Preprints: The Multidisciplinary Preprint Platform

Paste the URL in your RSS reader:

https://www.preprints.org/rss

If you want to subscribe our website in subject-wide, you could find the URL by choosing in the following:

https://www.preprints.org/rss

Please note that by using ScienceDirect RSS feeds, you agree to our Terms of Use and Privacy Policy.