Submitted:
27 March 2023
Posted:
28 March 2023
You are already at the latest version
Abstract
Keywords:
1. Introduction
1.1. Aims and Contributions
- Investigated the opportunities and possible threats of using ChatGPT in educational settings, particularly in programming education
- Presented threat mitigation strategies in the presence of AI tools such as ChatGPT
- Conducted experiments with ChatGPT to illustrate how this tool can be used to support programming learning
- Discussed future educational plans and curriculum in light of such revolutionary AI tools
2. Related Literature
3. Opportunities with ChatGPT
3.1. Opportunities for Learners
3.2. Opportunities for Educators
3.3. Opportunities for Researchers
4. Programming Learning with ChatGPT
4.1. Conceptual Understanding
4.2. Solution Code Generation
4.3. Error Checking and Debugging in Code
4.4. Solution Code Optimization
5. Threats and Strategies
“ChatGPT is incredibly limited but good enough at some things to create a misleading impression of greatness. It’s a mistake to be relying on it for anything important but a preview of progress. We have lots of work to do on robustness and truthfulness.”— Sam Altman, CEO of OpenAI
6. Limitation
7. Conclusion
References
- Rahman, M.M.; Watanobe, Y.; Nakamura, K. A neural network based intelligent support model for program code completion. Scientific Programming 2020, 2020, 1–18. [Google Scholar] [CrossRef]
- Rahman, M.M.; Watanobe, Y.; Nakamura, K. A bidirectional LSTM language model for code evaluation and repair. Symmetry 2021, 13, 247. [Google Scholar] [CrossRef]
- Rahman, M.M. Data Analysis and Code Assessment Using Machine Learning Techniques for Programming Activities. PhD Thesis, The University of Aizu, Japan, 2022. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser; Polosukhin, I. Attention is all you need. Advances in neural information processing systems 2017, 30. [Google Scholar]
- Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. Advances in neural information processing systems 2020, 33, 1877–1901. [Google Scholar]
- Floridi, L.; Chiriatti, M. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines 2020, 30, 681–694. [Google Scholar] [CrossRef]
- OpenAI-Team. ChatGPT: Optimizing language models for dialogue. 2022. Available online: https://openai.com/blog/chatgpt/ (accessed on 11 March 2023).
- Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. Training language models to follow instructions with human feedback. arXiv preprint 2022, arXiv:2203.02155 2022. [Google Scholar]
- Kasneci, E.; Seßler, K.; Küchemann, S.; Bannert, M.; Dementieva, D.; Fischer, F.; Gasser, U.; Groh, G.; Günnemann, S.; Hüllermeier, E.; et al. ChatGPT for good? On opportunities and challenges of large language models for education 2023. [Google Scholar]
- Qadir, J. Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education 2022.
- Thunstrom, A.O. We asked GPT-3 to write an academic paper about itself: Then we tried to get it published. Scientific American 2022, 30. [Google Scholar]
- Stokel-Walker, C. AI bot ChatGPT writes smart essays-should academics worry? Nature 2022. [Google Scholar] [CrossRef]
- Welsh, M. The End of Programming. Commun. ACM 2022, 66, 34–35. [Google Scholar] [CrossRef]
- Susnjak, T. ChatGPT: The End of Online Exam Integrity? arXiv preprint 2022, arXiv:2212.09292 2022. [Google Scholar]
- Parslow, G.R. Commentary: How the internet is changing the way we think, read and remember. Biochemistry and Molecular Biology Education 2011, 39, 228. [Google Scholar] [CrossRef] [PubMed]
- Pappano, L. The Year of the MOOC. The New York Times 2012, 2, 2012. [Google Scholar]
- Wollny, S.; Schneider, J.; Di Mitri, D.; Weidlich, J.; Rittberger, M.; Drachsler, H. Are we there yet?-A systematic literature review on chatbots in education. Frontiers in artificial intelligence 2021, 4, 654924. [Google Scholar] [CrossRef] [PubMed]
- Rahman, M.M.; Watanobe, Y.; Rage, U.K.; Nakamura, K. A novel rule-based online judge recommender system to promote computer programming education. In Proceedings of the Advances and Trends in Artificial Intelligence. From Theory to Practice: 34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2021, Kuala Lumpur, Malaysia, 26–29 July 2021; Proceedings, Part II 34. Springer, 2021; pp. 15–27. [Google Scholar]
- Rahman, M.M.; Watanobe, Y.; Nakamura, K. Source code assessment and classification based on estimated error probability using attentive LSTM language model and its application in programming education. Applied Sciences 2020, 10, 2973. [Google Scholar] [CrossRef]
- Rahman, M.M.; Watanobe, Y.; Kiran, R.U.; Kabir, R. A stacked bidirectional lstm model for classifying source codes built in mpls. In Proceedings of the Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2021, Virtual Event, 13-17 September 2021; Proceedings, Part II. Springer, 2022; pp. 75–89. [Google Scholar]
- Litman, D. Natural language processing for enhancing teaching and learning. In Proceedings of the Proceedings of the AAAI conference on artificial intelligence, 2016, Vol. 30. 30.
- M Alshater, M. Exploring the role of artificial intelligence in enhancing academic performance: A case study of ChatGPT. Available at SSRN 2022. [Google Scholar] [CrossRef]
- Dowling, M.; Lucey, B. ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters, 2023; 103662. [Google Scholar]
- Rudolph, J.; Tan, S.; Tan, S. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 2023, 6. [Google Scholar] [CrossRef]
- Frieder, S.; Pinchetti, L.; Griffiths, R.R.; Salvatori, T.; Lukasiewicz, T.; Petersen, P.C.; Chevalier, A.; Berner, J. Mathematical capabilities of chatgpt. arXiv preprint 2023, arXiv:2301.13867 2023. [Google Scholar]
- Kung, T.H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepaño, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health 2023, 2, e0000198. [Google Scholar] [CrossRef]
- Gilson, A.; Safranek, C.; Huang, T.; Socrates, V.; Chi, L.; Taylor, R.A.; Chartash, D. How Well Does ChatGPT Do When Taking the Medical Licensing Exams? The Implications of Large Language Models for Medical Education and Knowledge Assessment. medRxiv, 2022; 2022–12. [Google Scholar]
- Aydın, Ö.; Karaarslan, E. OpenAI ChatGPT generated literature review: Digital twin in healthcare. Available at SSRN 4308687. 2022. [Google Scholar]
- Watanobe, Y.; Rahman, M.M.; Amin, M.F.I.; Kabir, R. Identifying algorithm in program code based on structural features using CNN classification model. Applied Intelligence, 2022; 1–27. [Google Scholar]
- Li, Y.; Choi, D.; Chung, J.; Kushman, N.; Schrittwieser, J.; Leblond, R.; Eccles, T.; Keeling, J.; Gimeno, F.; Dal Lago, A.; et al. Competition-level code generation with alphacode. Science 2022, 378, 1092–1097. [Google Scholar] [CrossRef]
- Castelvecchi, D. Are ChatGPT and AlphaCode going to replace programmers? Nature 2022. [Google Scholar] [CrossRef] [PubMed]
- Sobania, D.; Briesch, M.; Hanna, C.; Petke, J. An analysis of the automatic bug fixing performance of chatgpt. arXiv preprint, 2023; arXiv:2301.08653 2023. [Google Scholar]
- Jalil, S.; Rafi, S.; LaToza, T.D.; Moran, K.; Lam, W. ChatGPT and Software Testing Education: Promises & Perils. arXiv preprint, 2023; arXiv:2302.03287 2023. [Google Scholar]
- Avila-Chauvet, L.; Mejía, D.; Acosta Quiroz, C.O. Chatgpt as a Support Tool for Online Behavioral Task Programming. Available at SSRN 4329020. 2023. [Google Scholar]
- Antaki, F.; Touma, S.; Milad, D.; El-Khoury, J.; Duval, R. Evaluating the performance of chatgpt in ophthalmology: An analysis of its successes and shortcomings. medRxiv, 2023; 2023–01. [Google Scholar]
- Rao, A.S.; Kim, J.; Kamineni, M.; Pang, M.; Lie, W.; Succi, M. Evaluating ChatGPT as an adjunct for radiologic decision-making. medRxiv, 2023; 2023–02. [Google Scholar]
- Wenzlaff, K.; Spaeth, S. Smarter than Humans? Validating how OpenAI’s ChatGPT model explains Crowdfunding, Alternative Finance and Community Finance. Validating how OpenAI’s ChatGPT model explains Crowdfunding, Alternative Finance and Community Finance.(December 22, 2022). 22 December 2022. [Google Scholar]
- Zaremba, A.; Demir, E. ChatGPT: Unlocking the Future of NLP in Finance. Available at SSRN 4323643. 2023. [Google Scholar]
- Choi, J.H.; Hickman, K.E.; Monahan, A.; Schwarcz, D. Chatgpt goes to law school. Available at SSRN. 2023. [Google Scholar]
- Jiao, W.; Wang, W.; Huang, J.t.; Wang, X.; Tu, Z. Is ChatGPT a good translator? A preliminary study. arXiv preprint 2023, arXiv:2301.08745 2023. [Google Scholar]
- Beth, M. AI and the future of undergraduate writing. Available online: https://www.chronicle.com/article/ai-and-the-future-of-undergraduate-writing (accessed on 12 March 2023).
- Sharples, M. Automated essay writing: an AIED opinion. International Journal of Artificial Intelligence in Education 2022, 32, 1119–1126. [Google Scholar] [CrossRef]
- Rudolph, J.; Tan, S.; Tan, S. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 2023, 6. [Google Scholar] [CrossRef]
- Rahman, M.M.; Watanobe, Y.; Kiran, R.U.; Thang, T.C.; Paik, I. Impact of practical skills on academic performance: A data-driven analysis. IEEE Access 2021, 9, 139975–139993. [Google Scholar] [CrossRef]
- Zhang, Q.; Fang, C.; Ma, Y.; Sun, W.; Chen, Z. A Survey of Learning-based Automated Program Repair. arXiv preprint 2023, arXiv:2301.03270 2023. [Google Scholar] [CrossRef]
- Rahman, M.M.; Watanobe, Y.; Nakamura, K. Evaluation of source codes using bidirectional lstm neural network. In Proceedings of the 2020 3rd IEEE international conference on knowledge innovation and invention (ICKII). IEEE; 2020; pp. 140–143. [Google Scholar]
- Sobania, D.; Briesch, M.; Rothlauf, F. Choose your programming copilot: a comparison of the program synthesis performance of github copilot and genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference; 2022; pp. 1019–1027. [Google Scholar]
- Watanobe, Y.; Rahman, M.M.; Matsumoto, T.; Rage, U.K.; Ravikumar, P. Online judge system: requirements, architecture, and experiences. International Journal of Software Engineering and Knowledge Engineering 2022, 32, 917–946. [Google Scholar] [CrossRef]
- Cotton, D.R.; Cotton, P.A.; Shipway, J.R. Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT 2023. [Google Scholar]
- Stokel-Walker, C. ChatGPT listed as author on research papers: many scientists disapprove. Nature. [CrossRef]
- Pavlik, J.V. Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. Journalism & Mass Communication Educator, 2023; 10776958221149577. [Google Scholar] [CrossRef]
- Elkins, K.; Chun, J. Can GPT-3 pass a Writer’s turing test? Journal of Cultural Analytics 2020, 5. [Google Scholar] [CrossRef]
- Gao, C.A.; Howard, F.M.; Markov, N.S.; Dyer, E.C.; Ramesh, S.; Luo, Y.; Pearson, A.T. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv, 2022; 2022–12. [Google Scholar]
- Dehouche, N. Plagiarism in the age of massive Generative Pre-trained Transformers (GPT-3). Ethics in Science and Environmental Politics 2021, 21, 17–23. [Google Scholar] [CrossRef]
- Kalhan, R. ChatGPT banned from New York City public schools’ devices and networks. 2023. Available online: https://www.nbcnews.com/tech/tech-news/new-york-city-public-schools-ban-chatgpt-devices-networks-rcna64446 (accessed on 12 March 2023).










| Category | Article | # Citation | Description |
|---|---|---|---|
| Programming support | Jalil et al. [33] | - | Solving questions of software testing curriculum |
| Sobania et al. [32] | - | Automatic bug fixing in code | |
| Qadir [10] | 5 | Application in engineering education | |
| Laurent et al. [34] | 1 | Support tool for HTML, CSS, and JavaScript code | |
| Matt Welsh [13] | - | Future of common programming practices | |
| Medical Education and Exam | Kung et al. [26] | 14 | AI-assisted medical education |
| Gilson et al. [27] | 4 | Medical education | |
| Antaki et al. [35] | 1 | Ophthalmology question-answering | |
| Rao et al. [36] | - | Adjunct for Radiologic Decision-Making | |
| Finance Education and Research | Muneer [22] | 2 | Enhance the performance of economy and finance research |
| Dowling et al. [23] | 3 | Research on finance | |
| Karsten [37] | 1 | Explain alternative and community finance and crowdfunding | |
| Adam and Demir [38] | 2 | Financial application | |
| Mathematics | Simon et al. [25] | 2 | Presented mathematical capabilities |
| Law | Jonathan et al. [39] | 2 | Examine the performance for law school exams |
| Translator | Wenxiang et al. [40] | 6 | Performance as a machine translation |
| Case No. | Problem Name | Evaluation on Basic Compiler | Evaluation on AOJ | ||
|---|---|---|---|---|---|
| Success | Failed | Success | Failed | ||
| 1 | Insertion Sort (IS) 4 | 3 | 0 | 2 | 1 (WA) |
| 2 | GCD 5 | 3 | 0 | 3 | 0 |
| 3 | Prime Numbers (PN) 6 | 3 | 0 | 3 | 0 |
| 4 | Reverse Polish Notation (RPN) 7 | 3 | 0 | 3 | 0 |
| 5 | Round-Robin Scheduler (RRS) 8 | 2 | 1 | 2 | 1 (RE) |
| 6 | Binary Search (BS) 9 | 3 | 0 | 2 | 1 (TLE) |
| 7 | Merge Sort (MS) 10 | 3 | 0 | 1 | 2 (TLE, WA) |
| 8 | Depth First Search (DFS) 11 | 3 | 0 | 2 | 1 (WA) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).