Submitted:
13 March 2024
Posted:
28 March 2024
You are already at the latest version
Abstract
Keywords:
1. Introduction
1.1. Introduction
1.2. Problem Statement and Research Questions
2. Materials and Methods:
2.1. Design, Participants, and Procedures
- experimental group 1 received 15 sessions of 90 minutes each with chatbot-based language learning support,
- experimental group 2 received 15 sessions of 90 minutes each with adaptive learning algorithms for language acquisition,
- experimental group 3 received 15 sessions of 90 minutes each with virtual reality language immersion experience, and
- control group 4 underwent regular computer-assisted language learning activities without the incorporation of chatbot-based language learning support, adaptive learning algorithms, or virtual reality language immersion.
2.2. Instruments
2.2.1. Self-Regulated Learning Scale
2.2.2. Language Proficiency (Pretest-Posttest)
2.2.3. Participants’ Perceptions of the Administered Treatments
2.2.4. Interviews
2.3. Data Analysis
3. Results
4. Discussion
5. Implications and Limitations, and Future Steps
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
Appendix A. Self-regulated Learning Scale Items and Descriptions
| Item | Focus | Statement | M | SD |
| 1 | Goal Setting | I set clear and achievable learning goals for myself. | 3.76 | 1.244 |
| 2 | Time Management | I effectively manage my time when engaging in language learning activities. | 3.77 | 1.259 |
| 3 | Self-Monitoring | I regularly monitor my progress and adjust my learning strategies accordingly. | 3.80 | 1.267 |
| 4 | Adaptive Learning Strategies | I adapt my learning approach based on my understanding of the language material. | 6.89 | .314 |
| 5 | Goal Setting | I set specific targets for improving my language skills. | 5.74 | .750 |
| 6 | Time Management | I allocate my study time efficiently to cover different language learning tasks. | 6.29 | .472 |
| 7 | Self-Monitoring | I keep track of my language learning progress and make changes as needed. | 6.89 | .314 |
| 8 | Adaptive Learning Strategies | I modify my learning methods to better suit my language learning needs. | 5.74 | .750 |
| 9 | Goal Setting | I establish measurable objectives to enhance my language proficiency. | 6.29 | .472 |
Appendix B. Perceptions of language learning support and immersion technologies
| No. | Question/Item | Focus | M | SD |
| 1. | I believe that the chatbot-based language learning support facilitated my language skills by providing personalized language practice and feedback. | Chatbot-Based Support | 5.89 | .313 |
| 2. | I believe that the chatbot-based language learning support facilitated my self-regulated learning by offering interactive and real-time language support. | Chatbot-Based Support | 5.45 | .327 |
| 3. | I believe that the adaptive learning algorithms enhanced my language learning experience by adapting to my learning pace and providing targeted exercises. | Adaptive Learning Algorithms | 5.67 | .320 |
| 4. | I believe that the adaptive learning algorithms enhanced my self-regulated learning by tracking my progress and offering tailored learning resources. | Adaptive Learning Algorithms | 5.89 | .313 |
| 5. | I believe that the virtual reality language immersion improved my language skills by creating immersive and interactive language learning environments. | Virtual Reality Immersion | 5.45 | .327 |
| 6. | I believe that the virtual reality language immersion improved my self-regulated learning by providing realistic and engaging language learning scenarios. | Virtual Reality Immersion | 5.67 | .320 |
| 7. | I believe that the chatbot-based language learning support enhanced my vocabulary acquisition by providing targeted word practice and explanations. | Chatbot-Based Support | 5.89 | .313 |
| 8. | I believe that the adaptive learning algorithms improved my language comprehension by adjusting the difficulty of learning materials based on my performance. | Adaptive Learning Algorithms | 5.45 | .327 |
| 9. | I believe that the virtual reality language immersion enriched my cultural understanding by simulating authentic language and cultural experiences. | Virtual Reality Immersion | 5.67 | .320 |
| 10. | I believe that the chatbot-based language learning support increased my motivation to learn by offering engaging and interactive learning activities. | Chatbot-Based Support | 5.89 | .313 |
| 11. | I believe that the adaptive learning algorithms promoted my autonomy in learning by providing opportunities for self-assessment and self-directed learning. | Adaptive Learning Algorithms | 5.45 | .327 |
| 12. | I believe that the virtual reality language immersion enhanced my language fluency by creating opportunities for real-time language use and communication. | Virtual Reality Immersion | 5.67 | .320 |
Appendix C. Interview questions and focus
| No. | Question/Item | Focus | M | SD |
| 1. | In your experience, did the chatbot-based language learning support contribute to your language skills? | Chatbot-Based Support | 3.94 | .254 |
| 2. | Can you share your perspective on how the chatbot-based language learning support impacted your self-regulated learning? | Chatbot-Based Support | 3.65 | .288 |
| 3. | How would you rate the effectiveness of the adaptive learning algorithms in enhancing your language learning experience? | Adaptive Learning Algorithms | 3.94 | .254 |
| 4. | From your experience, how did the adaptive learning algorithms contribute to your self-regulated learning? | Adaptive Learning Algorithms | 3.65 | .288 |
| 5. | Can you assess the impact of virtual reality language immersion on improving your language skills? | Virtual Reality Immersion | 3.94 | .254 |
| 6. | How would you rate the effectiveness of virtual reality language immersion in enhancing your self-regulated learning? | Virtual Reality Immersion | 3.65 | .288 |
| 7. | Based on your experience, how effective was the chatbot-based language learning support in enhancing your vocabulary acquisition? | Chatbot-Based Support | 3.94 | .254 |
| 8. | How do you assess the impact of the adaptive learning algorithms on improving your language comprehension? | Adaptive Learning Algorithms | 3.65 | .288 |
| 9. | From your perspective, how did virtual reality language immersion contribute to enriching your cultural understanding? | Virtual Reality Immersion | 3.94 | .254 |
| 10. | Can you evaluate whether the chatbot-based language learning support increased your motivation to learn? | Chatbot-Based Support | 3.45 | .2898 |
References
- Kondurkar, I., Raj, A., & Lakshmi, D. (2024). Modern Applications With a Focus on Training ChatGPT and GPT Models: Exploring Generative AI and NLP. In A. Obaid, B. Bhushan, M. S., & S. Rajest (Eds.), Advanced Applications of Generative AI and Natural Language Processing Models (pp. 186-227). IGI Global. [CrossRef]
- Tyagi, A. K. (2024). Transformative Effects of ChatGPT on the Modern Era of Education and Society: From Society’s and Industry’s Perspectives. In P. Baby Maruthi, S. Prasad, & A. Tyagi (Eds.), Machine Learning Algorithms Using Scikit and TensorFlow Environments (pp. 374-387). IGI Global. [CrossRef]
- Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28(1), 973-1018. [CrossRef]
- Al-Abdullatif, A. M., Al-Dokhny, A. A., & Drwish, A. M. (2023). Implementing the Bashayer chatbot in Saudi higher education: measuring the influence on students’ motivation and learning strategies. Frontiers in Psychology, 14, 1129070. [CrossRef]
- Deng, X., & Yu, Z. (2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15(4), 2940. [CrossRef]
- Osadcha, K., Osadchyi, V., Semerikov, S., Chemerys, H., & Chorna, A. V. (2020). The review of the adaptive learning systems for the formation of individual educational trajectory. In CEUR Workshop Proceedings 2732, 547-558.https://elibrary.kdpu.edu.ua/handle/123456789/4130.
- Yang, F.. C. O., Lo, F. Y. R., Hsieh, J. C.,& Wu, W. C. V. (2020). Facilitating communicative ability of EFL learners via high-immersion virtual reality. Journal of Educational Technology & Society, 23(1), 30-49.https://www.jstor.org/stable/26915405.
- Ciekanski, M., Kalyaniwala, C., Molle, N., & Privas-Bréauté, V. (2020). Real and perceived affordances of Immersive Virtual Environments in a language teacher-training context: effects on the design of learning tasks. Revista Docência e Cibercultura, 4(3), 83-111. [CrossRef]
- Grassini, S. (2023). Shaping the future of education: exploring the potential and consequences of AI and ChatGPT in educational settings. Education Sciences, 13(7), 692. [CrossRef]
- Zhang, R., Zou, D., & Cheng, G. (2023). A review of chatbot-assisted learning: pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments, 1-29. [CrossRef]
- Merelo, J.J. et al. (2022). Exploring the Role of Chatbots and Messaging Applications in Higher Education: A Teacher’s Perspective. In: Zaphiris, P., Ioannou, A. (eds) Learning and Collaboration Technologies. Novel Technological Environments. HCII 2022. Lecture Notes in Computer Science, vol 13329. Springer, Cham. [CrossRef]
- Ramandanis, D., & Xinogalos, S. (2023). Investigating the Support Provided by Chatbots to Educational Institutions and Their Students: A Systematic Literature Review. Multimodal Technologies and Interaction, 7(11), 103. [CrossRef]
- Yang, Y., Wen, Y., & Song, Y. (2023). A systematic review of technology-enhanced self-regulated language learning. Educational Technology & Society, 26(1), 31-44. https://www.jstor.org/stable/48707965.
- Zhao, Y., & Lai, C. (2023). Technology and second language learning: Promises and problems. In Y. Zhao & C. Lai (Eds) Technology-mediated learning environments for young English learners (pp. 167-206). London: Routledge. [CrossRef]
- Wu, J.G., Zhang, D. & Lee, S.M. (2024) Into the Brave New Metaverse: Envisaging Future Language Teaching and Learning. in IEEE Transactions on Learning Technologies, 17, 44-53. [CrossRef]
- Authors et al. (2023).
- Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive science, 12(2), 257-285. [CrossRef]
- Dix, A., Finlay, J., Abowd, G. & Beale, R. (2004). Human-computer interaction. Harlow: Pearson Education.
- Author (2023).
- Authors et al. (2022).
- Jia, C., & Hew, K. F. (2023). Meeting the challenges of decoding training in English as a foreign/second language listening education: current status and opportunities for technology-assisted decoding training. Computer assisted language learning, 36(5-6), 1116-1145. [CrossRef]
- Chen, J. C., & Kent, S. (2020). Task engagement, learner motivation and avatar identities of struggling English language learners in the 3D virtual world. System, 88, 102168. [CrossRef]
- Suzuki, Y. (Ed.). (2023). Practice and automatization in second language research: Perspectives from skill acquisition theory and cognitive psychology. New York: Routledge.
- Han, S., Liu, M., Pan, Z., Cai, Y., & Shao, P. (2023). Making FAQ chatbots more Inclusive: an examination of non-native English users’ interactions with new technology in massive open online courses. International Journal of Artificial Intelligence in Education, 33(3), 752-780. [CrossRef]
- Kerimbayev, N., Umirzakova, Z., Shadiev, R., & Jotsov, V. (2023). A student-centered approach using modern technologies in distance learning: a systematic review of the literature. Smart Learning Environments, 10(1), 61. [CrossRef]
- Song, C., Shin, S. Y., & Shin, K. S. (2023). Optimizing Foreign Language Learning in Virtual Reality: A Comprehensive Theoretical Framework Based on Constructivism and Cognitive Load Theory (VR-CCL). Applied Sciences, 13(23), 12557. [CrossRef]
- Şahin Kızıl, A., & Savran, Z. (2018). Assessing self-regulated learning: The case of vocabulary learning through information and communication technologies. Computer Assisted Language Learning, 31(5-6), 599-616. [CrossRef]
- Miller Jr, R. G. (1997). Beyond ANOVA: basics of applied statistics. Florida: CRC press.
- Lawless, B., & Chen, Y. W. (2019). Developing a method of critical thematic analysis for qualitative communication inquiry. Howard Journal of Communications, 30(1), 92-106. [CrossRef]
- Silitonga, L. M., Hawanti, S., Aziez, F., Furqon, M., Zain, D. S. M., Anjarani, S., & Wu, T. T. (2023, August). The Impact of AI Chatbot-Based Learning on Students’ Motivation in English Writing Classroom. In International Conference on Innovative Technologies and Learning (pp. 542-549). Cham: Springer Nature Switzerland. [CrossRef]
- Chiu, T. K., Moorhouse, B. L., Chai, C. S., & Ismailov, M. (2023). Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot. Interactive Learning Environments, 1-17. [CrossRef]
- Ait Baha, T., El Hajji, M., Es-Saady, Y., & Fadili, H. (2023). The impact of educational chatbot on student learning experience. Education and Information Technologies, 1-24. [CrossRef]
- Shaikh, S., Yayilgan, S. Y., Klimova, B., & Pikhart, M. (2023). Assessing the usability of ChatGPT for formal English language learning. European Journal of Investigation in Health, Psychology and Education, 13(9), 1937-1960. [CrossRef]
- Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237-257. [CrossRef]
- Kohnke, L. (2023). A pedagogical chatbot: A supplemental language learning tool. RELC Journal, 54(3), 828-838. [CrossRef]
| Descriptives | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| N | Mean | Std. Deviation | Std. Error | 95% Confidence Interval for Mean | Minimum | Maximum | |||
| Lower Bound | Upper Bound | ||||||||
| Language Proficiency Pretest | Group 1 | 136 | 48.03 | 5.074 | .435 | 47.17 | 48.89 | 39 | 58 |
| Group 2 | 136 | 47.81 | 5.165 | .443 | 46.93 | 48.68 | 39 | 58 | |
| Group 3 | 137 | 48.07 | 5.039 | .431 | 47.22 | 48.92 | 39 | 58 | |
| Group 4 | 137 | 48.28 | 5.033 | .430 | 47.43 | 49.13 | 39 | 58 | |
| Total | 546 | 48.05 | 5.067 | .217 | 47.62 | 48.47 | 39 | 58 | |
| Language Proficiency Posttest | Group 1 | 136 | 93.39 | 5.492 | .471 | 92.46 | 94.32 | 76 | 100 |
| Group 2 | 136 | 70.74 | 4.566 | .392 | 69.96 | 71.51 | 66 | 89 | |
| Group 3 | 137 | 70.83 | 4.362 | .373 | 70.10 | 71.57 | 66 | 89 | |
| Group 4 | 137 | 47.93 | 4.810 | .411 | 47.12 | 48.75 | 39 | 58 | |
| Total | 546 | 70.68 | 16.790 | .719 | 69.27 | 72.09 | 39 | 100 | |
| Self-regulated learning scale pretest | Group 1 | 136 | 3.76 | 1.244 | .107 | 3.55 | 3.97 | 1 | 6 |
| Group 2 | 136 | 3.77 | 1.259 | .108 | 3.56 | 3.99 | 1 | 6 | |
| Group 3 | 137 | 3.80 | 1.267 | .108 | 3.58 | 4.01 | 1 | 6 | |
| Group 4 | 137 | 3.91 | 1.228 | .105 | 3.71 | 4.12 | 1 | 6 | |
| Total | 546 | 3.81 | 1.248 | .053 | 3.70 | 3.91 | 1 | 6 | |
| Self-regulated learning scale posttest | Group 1 | 136 | 6.89 | .314 | .027 | 6.84 | 6.94 | 6 | 7 |
| Group 2 | 136 | 5.74 | .750 | .064 | 5.62 | 5.87 | 4 | 7 | |
| Group 3 | 137 | 6.29 | .472 | .040 | 6.21 | 6.37 | 5 | 7 | |
| Group 4 | 137 | 3.74 | 1.213 | .104 | 3.54 | 3.95 | 1 | 6 | |
| Total | 546 | 5.66 | 1.410 | .060 | 5.55 | 5.78 | 1 | 7 | |
| ANOVA | ||||||
|---|---|---|---|---|---|---|
| Sum of Squares | df | Mean Square | F | Sig. | ||
| Language Proficiency Pretest | Between Groups | 15.120 | 3 | 5.040 | .195 | .899 |
| Within Groups | 13975.642 | 542 | 25.785 | |||
| Total | 13990.762 | 545 | ||||
| Language Proficiency Posttest | Between Groups | 141022.186 | 3 | 47007.395 | 2018.801 | .000 |
| Within Groups | 12620.364 | 542 | 23.285 | |||
| Total | 153642.549 | 545 | ||||
| Self-regulated learning scale pretest | Between Groups | 2.038 | 3 | .679 | .435 | .728 |
| Within Groups | 846.153 | 542 | 1.561 | |||
| Total | 848.190 | 545 | ||||
| Self-regulated learning scale posttest | Between Groups | 763.947 | 3 | 254.649 | 431.692 | .000 |
| Within Groups | 319.718 | 542 | .590 | |||
| Total | 1083.665 | 545 | ||||
| ANOVA Effect Sizesa,b | ||||
|---|---|---|---|---|
| Point Estimate | 95% Confidence Interval | |||
| Lower | Upper | |||
| Language Proficiency Pretest | Eta-squared | .001 | .000 | .006 |
| Epsilon-squared | -.004 | -.006 | .000 | |
| Omega-squared Fixed-effect | -.004 | -.006 | .000 | |
| Omega-squared Random-effect | -.001 | -.002 | .000 | |
| Language Proficiency Posttest | Eta-squared | .918 | .907 | .926 |
| Epsilon-squared | .917 | .906 | .926 | |
| Omega-squared Fixed-effect | .917 | .906 | .926 | |
| Omega-squared Random-effect | .787 | .762 | .806 | |
| Self-regulated learning scale pretest | Eta-squared | .002 | .000 | .011 |
| Epsilon-squared | -.003 | -.006 | .006 | |
| Omega-squared Fixed-effect | -.003 | -.006 | .006 | |
| Omega-squared Random-effect | -.001 | -.002 | .002 | |
| Self-regulated learning scale posttest | Eta-squared | .705 | .667 | .734 |
| Epsilon-squared | .703 | .665 | .733 | |
| Omega-squared Fixed-effect | .703 | .664 | .733 | |
| Omega-squared Random-effect | .441 | .398 | .477 | |
| a. Eta-squared and Epsilon-squared are estimated based on the fixed-effect model. | ||||
| b. Negative but less biased estimates are retained, not rounded to zero. | ||||
| Multiple Comparisons | |||||||
|---|---|---|---|---|---|---|---|
| Tukey HSD | |||||||
| Dependent Variable | (I) group | (J) group | Mean Difference (I-J) | Std. Error | Sig. | 95% Confidence Interval | |
| Lower Bound | Upper Bound | ||||||
| Language Proficiency Pretest | Group 1 | Group 2 | .221 | .616 | .984 | -1.37 | 1.81 |
| Group 3 | -.044 | .615 | 1.000 | -1.63 | 1.54 | ||
| Group 4 | -.248 | .615 | .978 | -1.83 | 1.34 | ||
| Group 2 | Group 1 | -.221 | .616 | .984 | -1.81 | 1.37 | |
| Group 3 | -.264 | .615 | .973 | -1.85 | 1.32 | ||
| Group 4 | -.469 | .615 | .871 | -2.05 | 1.12 | ||
| Group 3 | Group 1 | .044 | .615 | 1.000 | -1.54 | 1.63 | |
| Group 2 | .264 | .615 | .973 | -1.32 | 1.85 | ||
| Group 4 | -.204 | .614 | .987 | -1.79 | 1.38 | ||
| Group 4 | Group 1 | .248 | .615 | .978 | -1.34 | 1.83 | |
| Group 2 | .469 | .615 | .871 | -1.12 | 2.05 | ||
| Group 3 | .204 | .614 | .987 | -1.38 | 1.79 | ||
| Language Proficiency Posttest | Group 1 | Group 2 | 22.654* | .585 | .000 | 21.15 | 24.16 |
| Group 3 | 22.558* | .584 | .000 | 21.05 | 24.06 | ||
| Group 4 | 45.455* | .584 | .000 | 43.95 | 46.96 | ||
| Group 2 | Group 1 | -22.654* | .585 | .000 | -24.16 | -21.15 | |
| Group 3 | -.097 | .584 | .998 | -1.60 | 1.41 | ||
| Group 4 | 22.801* | .584 | .000 | 21.30 | 24.31 | ||
| Group 3 | Group 1 | -22.558* | .584 | .000 | -24.06 | -21.05 | |
| Group 2 | .097 | .584 | .998 | -1.41 | 1.60 | ||
| Group 4 | 22.898* | .583 | .000 | 21.40 | 24.40 | ||
| Group 4 | Group 1 | -45.455* | .584 | .000 | -46.96 | -43.95 | |
| Group 2 | -22.801* | .584 | .000 | -24.31 | -21.30 | ||
| Group 3 | -22.898* | .583 | .000 | -24.40 | -21.40 | ||
| Self-regulated learning scale pretest | Group 1 | Group 2 | -.015 | .152 | 1.000 | -.41 | .38 |
| Group 3 | -.038 | .151 | .994 | -.43 | .35 | ||
| Group 4 | -.155 | .151 | .735 | -.54 | .23 | ||
| Group 2 | Group 1 | .015 | .152 | 1.000 | -.38 | .41 | |
| Group 3 | -.024 | .151 | .999 | -.41 | .37 | ||
| Group 4 | -.140 | .151 | .790 | -.53 | .25 | ||
| Group 3 | Group 1 | .038 | .151 | .994 | -.35 | .43 | |
| Group 2 | .024 | .151 | .999 | -.37 | .41 | ||
| Group 4 | -.117 | .151 | .866 | -.51 | .27 | ||
| Group 4 | Group 1 | .155 | .151 | .735 | -.23 | .54 | |
| Group 2 | .140 | .151 | .790 | -.25 | .53 | ||
| Group 3 | .117 | .151 | .866 | -.27 | .51 | ||
| Self-regulated learning scale posttest | Group 1 | Group 2 | 1.147* | .093 | .000 | .91 | 1.39 |
| Group 3 | .598* | .093 | .000 | .36 | .84 | ||
| Group 4 | 3.145* | .093 | .000 | 2.91 | 3.38 | ||
| Group 2 | Group 1 | -1.147* | .093 | .000 | -1.39 | -.91 | |
| Group 3 | -.549* | .093 | .000 | -.79 | -.31 | ||
| Group 4 | 1.998* | .093 | .000 | 1.76 | 2.24 | ||
| Group 3 | Group 1 | -.598* | .093 | .000 | -.84 | -.36 | |
| Group 2 | .549* | .093 | .000 | .31 | .79 | ||
| Group 4 | 2.547* | .093 | .000 | 2.31 | 2.79 | ||
| Group 4 | Group 1 | -3.145* | .093 | .000 | -3.38 | -2.91 | |
| Group 2 | -1.998* | .093 | .000 | -2.24 | -1.76 | ||
| Group 3 | -2.547* | .093 | .000 | -2.79 | -2.31 | ||
| *. The mean difference is significant at the 0.05 level. | |||||||
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).