Submitted:
10 May 2025
Posted:
12 May 2025
You are already at the latest version
Abstract
Keywords:
1. Introduction
2. Literature Review
2.1. AI-Assisted Academic Research
2.2. AI and Questionnaires
3. Research Methods
3.1. Experimental Design
3.2. Single Model Questionnaire Generation
3.3. Integrated Model Questionnaire Generation
3.4. Expert Evaluation Design
3.5. Deep Interviews
4. Results
4.1. Quantitative Research Results
4.2. Qualitative Research Results
4.2.1. Qualitative Research Results for Single Theoretical Model
4.2.2. Qualitative Research Results for Integrated Theoretical Model
5. Discussion
5.1. Performance of AI in Generating Questionnaires Based on Single Theoretical Model
5.2. Performance of AI in Generating Questionnaires Based on Integrated Theoretical Models
6. Conclusion
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Wang, C. AI-driven digital image art creation: Methods and case analysis[J]. Chinese Journal of Intelligent Science and Technology 2023, 5, 406–414. [Google Scholar]
- Zhang, C.; Lu, Y. Study on artificial intelligence: The state of the art and future prospects[J]. Journal of Industrial Information Integration 2021, 23, 100224. [Google Scholar]
- Min, B.; Ross, H.; Sulem, E.; et al. Recent advances in natural language processing via large pre-trained language models: A survey[J]. ACM Computing Surveys 2023, 56, 1–40. [Google Scholar] [CrossRef]
- Xiangdong, L.I.; Hanfei, X.I.A.; Yifei, S.; et al. Opportunities and challenges: automatic generation technologies for graphical user interfaces[J]. Journal of Graphics 2024, 45, 409. [Google Scholar]
- Genelza, G.G. A systematic literature review on AI voice cloning generator: A game-changer or a threat? [J]. Journal of Emerging Technologies 2024, 4, 54–61. [Google Scholar]
- Ayyagari, K.C. Rapid Video Prototyping and Content Creation Using Generative Artificial Intelligence[J]. 2024.
- Liang, J.; Shan, X.; Chung, J. A Study on Process of Creating 3D Models Using the Application of Artificial Intelligence Technology[J]. International Journal of Advanced Culture Technology 2023, 11, 346–351. [Google Scholar]
- Zhang, D.; Yu, Y.; Dong, J.; et al. Mm-llms: Recent advances in multimodal large language models[J]. arXiv preprint arXiv:2401.13601, 2024.
- AlZaabi, A.; ALamri, A.; Albalushi, H.; et al. ChatGPT applications in academic research: a review of benefits, concerns, and recommendations[J]. Biorxiv 2023, 2023.08. 17.553688.
- CURTIS N, CHATGPT, 2023. To ChatGPT or not to ChatGPT? the impact of artificial intelligence on academic publishing [J]. The Pediatric Infectious Disease Journal, 42(4), 275.
- ISKENDER A, 2023. Holy or unholy? interview with open AI’s ChatGPT [J]. European Journal of Tourism Research, 34, 3414-3414.
- LI X, HU L, SHANG X, et al. Reconstructing the Paradigm of Sports Science Research through Large Language Models: Applications and Insights—A Case Study of ChatGPT[J]. Journal of Xi'an Physical Education University 2023, 40, 405–415. [CrossRef]
- Wang, Y.; Guo, X.; Liu, Z. Detection and comparative study of differences between AI-generated and scholar-written Chinese abstracts[J]. J Intell 2023, 42, 127–134. [Google Scholar]
- Lebrun, B.; Temtsin, S.; Vonasch, A.; et al. Detecting the corruption of online questionnaires by artificial intelligence[J]. Frontiers in Robotics and AI 2024, 10, 1277635. [Google Scholar] [CrossRef] [PubMed]
- Olivos, F.; Liu, M. ChatGPTest: opportunities and cautionary tales of utilizing AI for questionnaire pretesting[J]. Field Methods 2024, 1525822X241280574. [Google Scholar] [CrossRef]
- Abu-Dalbouh, H.M. A questionnaire approach based on the technology acceptance model for mobile tracking on patient progress applications[J]. J. Comput. Sci. 2013, 9, 763–770. [Google Scholar] [CrossRef]
- Putz, D.; Schilling, J.; Kluge, A.; et al. Measuring organizational learning from errors: Development and validation of an integrated model and questionnaire[J]. Management learning 2013, 44, 511–536. [Google Scholar] [CrossRef]
- Goodfellow, I.; Pouget⁃Abadie J, Mirza M.; et al. Generative adversarial nets[J]. Advances in neural information processing systems 2014, 27, 2672–2680.
- Clark, A.; Donahue, J.; Simonyan, K. Adversarial video generation on complex datasets[J]. arXiv preprint, arXiv:1907.06571, 2019.
- Tsidylo, I.M.; Sena, C.E. Artificial intelligence as a methodological innovation in the training of future designers: Midjourney tools[J]. Information Technologies and Learning Tools 2023, 97, 203. [Google Scholar] [CrossRef]
- OpenAI GPT - 4 [ EB/ OL]. [2023 - 04 - 27] https: / / openai.com/ research / gpt-4.
- Dowling, M.; Lucey, B. ChatGPT for (finance) research: The Bananarama conjecture[J]. Finance Research Letters 2023, 53, 103662. [Google Scholar] [CrossRef]
- Ramos, M.C.; Collison, C.J.; White, A.D. A review of large language models and autonomous agents in chemistry[J]. Chemical Science, 2025.
- ZUO B, WU J, YANG Y. ChatGPT as a Questionnaire Tool for Tourism Research: Advantages and Limitations[J/OL]. Tourism Science,1-11[2025-02-24]. [CrossRef]
- Zhao, H.; Wang, Q.; Zhou T. et, a.l. Research on willingness of medical students to use virtual simulation experiment platform based on the UTAUT model. Disease Prevention and Control Bulletin, 1-5. [Online]. [CrossRef]
- WANGC Study on the Willingness of Chinese University Students to Learn AI Painting Tools Based on an Extended UTAUT Model[J/OL]. Packaging Engineering, 1-13 [2025-02-24]. http://kns.cnki.net/kcms/detail/50.1094.TB.20241210.1330.002.html.
- Meyer, M.A.; Booker, J.M. Eliciting and analyzing expert judgment: a practical guide[M]. Society for industrial and applied mathematics, 2001.
- BAUM S D, GOERTZEL B, GOERTZEL T G. How long until human-level AI? Results from an expert assessment[J].
- Technological Forecasting and Social Change, 2011, 78(1): 185-195.Olson K. An examination of questionnaire evaluation by expert reviewers[J]. Field methods 2010, 22, 295–318.
- Zou, Z.; Mubin, O.; Alnajjar, F.; et al. A pilot study of measuring emotional response and perception of LLM-generated questionnaire and human-generated questionnaires[J]. Scientific reports 2024, 14, 2781. [Google Scholar] [CrossRef] [PubMed]
- Kunst, J.R.; Bierwiaczonek, K. Utilizing AI questionnaire translations in cross-cultural and intercultural research: Insights and recommendations[J]. International Journal of Intercultural Relations 2023, 97, 101888. [Google Scholar] [CrossRef]
| Dimension | Items |
| Performance Expectancy | 1. The application of virtual simulation can compensate for scenarios that cannot be realized in real teaching (such as face-to-face interviews with patients). 2. Using virtual simulation helps me develop practical operation skills. 3. Virtual simulation experiments can give me a sense of learning achievement. |
| Effort Expectancy | 1. The system function options of the virtual simulation platform are simple and easy to use. 2. The page response speed is fast during the use of the virtual simulation platform. 3. The course content organization structure of the virtual simulation platform is clear and has a strong sense of hierarchy. |
| Social Influence |
1. Surrounding classmates are all using the virtual simulation platform, and I cannot fall behind. 2. Feel proud of being able to complete course learning from the virtual simulation platform. 3. Recognize that learning on the virtual simulation platform is one of the main learning modes for future careers. |
| Facilitating Conditions | 1. The learning time and location of the virtual simulation platform are somewhat limited. 2. The virtual simulation platform is inconvenient to switch between other programs or applications. 3. The virtual simulation platform requires network and computer, which is very inconvenient for you. |
| Behavioral Intention | 1. I plan to continue trying to use the educational resources of the virtual simulation platform. 2. I am willing to recommend the teaching resources of the virtual simulation platform to classmates, friends, etc. 3. I actively use the virtual simulation platform for learning. |
| Dimension | Items |
| Performance Expectancy | 1. I believe the virtual simulation experiment platform can help me improve my academic performance and experimental skills. 2. I believe using the virtual simulation experiment platform can enhance my experimental operation abilities. 3. I believe the virtual simulation experiment platform provides practical help for my medical studies. |
| Effort Expectancy | 1. I believe the virtual simulation experiment platform is easy to operate. 2. I believe the interface design of the virtual simulation experiment platform is simple and intuitive. 3. I believe the process of learning how to use the virtual simulation experiment platform is not too complicated. |
| Social Influence |
1. My classmates and teachers recommend using the virtual simulation experiment platform. 2. I feel that my peers' use of the virtual simulation experiment platform influences my willingness to use it. 3. I believe that using the virtual simulation experiment platform in learning is widely accepted and promoted. |
| Facilitating Conditions | 1. I believe the school provides sufficient resources (such as computer equipment, network support) to use the virtual simulation experiment platform. 2. I believe technical support and help for the virtual simulation experiment platform is easily accessible. 3. I believe I have sufficient knowledge and skills to use the virtual simulation experiment platform. |
| Behavioral Intention | 1. I am willing to continue using the virtual simulation experiment platform for learning. 2. I plan to frequently use the virtual simulation experiment platform in my future studies. 3. Even without mandatory requirements, I would choose to use the virtual simulation experiment platform for learning. |
| Use Behavior |
1. In the past three months, I have used the virtual simulation experiment platform for learning. 2. In the past three months, I have conducted experimental operations through the virtual simulation experiment platform. 3. I frequently use the virtual simulation experiment platform for medical experimental operations. |
| Dimension | Items |
| Performance Expectancy | 1. I believe learning AI painting tools can enhance my overall abilities. 2. I think using AI painting tools can improve my creative efficiency. 3. I believe AI painting technology will be very useful in the future. |
| Effort Expectancy | 1. I find it easy to learn how to use AI painting tools. 2. I find using AI painting tools is not complicated/does not require much mental effort. 3. I can easily master using AI painting tools. |
| Social Influence |
1. Friends around me occasionally discuss and exchange information about AI painting tools. 2. My friends/teachers recommend that I use AI painting tools. 3. My company/industry is gradually beginning to use AI painting tools. |
| Facilitating Conditions | 1. I have sufficient resources to use AI painting tools. 2. I have enough time to learn and use AI painting tools. 3. I can access the information needed to learn AI painting tools. |
| Individual Innovativeness | 1. I like to try new information technologies. 2. Among my peers, I am usually the first to try new information technologies. 3. If I hear about a new information technology, I will try to experiment with it. |
| AI Anxiety |
1. I worry that AI painting tools might generate content that violates privacy. 2. I am concerned that as AI painting technology develops, the skills I have previously learned may become less important. 3. I worry that the widespread application of AI painting tools may affect my career prospects. |
| Behavioral Intention | 1. I plan to use AI painting tools in my future research. 2. I will recommend others to use AI painting tools. 3. I am willing to follow the development and updates of AI painting tools. |
| Dimension | Items |
| Performance Expectancy | 1. I believe using AI painting tools can improve my design efficiency. 2. Using AI painting tools can help me create more creative design works. 3. AI painting tools can bring substantial help to my design process. |
| Effort Expectancy | 1. I believe that operating AI painting tools is simple and easy to understand. 2. I believe learning to use AI painting tools won't take too much time and effort. 3. I believe the functional interface design of AI painting tools is intuitive and easy to use. |
| Social Influence |
1. My classmates encourage me to use AI painting tools. 2. My teachers recommend using AI painting tools to improve design abilities. 3. Many people in my social circle use AI painting tools, and I feel their user experience is positive. |
| Facilitating Conditions | 1. I can access sufficient resources (such as computer equipment, software, etc.) to use AI painting tools. 2. I have adequate technical support to use AI painting tools. 3. My school provides the necessary facilities and environment for using AI painting tools. |
| Individual Innovativeness | 1. I am willing to try new technologies and tools to enhance my design abilities. 2. I like to try unfamiliar technologies, even if they bring some uncertainty. 3. I usually become an early adopter, using newly launched technologies or tools. |
| AI Anxiety |
1. I worry that AI painting tools might replace my creativity and design thinking. 2. I feel uncertain and anxious about how AI painting tools process my creations. 3. I am concerned that AI painting tools might infringe on my personal creative privacy or leak my design data. |
| Behavioral Intention | 1. I plan to use AI painting tools in future design projects. 2. I am willing to use AI painting tools in my future learning process. 3. I believe AI painting tools will be an important part of my future design work. |
| Dimension | Items |
| Accuracy | The questions accurately reflect the variables intended to be studied. |
| Comprehensiveness | The questionnaire questions comprehensively reflect each variable. |
| Clarity | The language of the questions is straightforward, concise, without ambiguity, and without complex expressions or difficult terminology. |
| Redundancy | Each question expresses a different meaning, with no repetition between different questions. |
| Order and Structure | The questions in the questionnaire are arranged in a logical order, guiding respondents to answer smoothly. |
| Objectivity | The questionnaire does not contain questions that create emotional or cognitive bias in respondents; all questions are neutral and fair. |
| Ethics and Privacy Protection | The questionnaire fully considers ethics and respondent privacy protection. |
| Dimension | Type | Mean±SD | t | p |
| Accuracy | Manual | 4.00±0.85 | 5.322 | 0.000** |
| AI | 2.17±0.83 | |||
| Comprehensiveness | Manual | 4.00±0.74 | 6.127 | 0.000** |
| AI | 2.08±0.79 | |||
| Clarity | Manual | 4.08±0.67 | 5.409 | 0.000** |
| AI | 2.25±0.97 | |||
| Redundancy | Manual | 4.17±0.58 | 1.787 | 0.088 |
| AI | 3.67±0.78 | |||
| Order & Structure | Manual | 4.58±0.51 | 0.793 | 0.436 |
| AI | 4.42±0.51 | |||
| Objectivity | Manual | 4.33±0.65 | -0.348 | 0.731 |
| AI | 4.42±0.51 | |||
| Ethics & Privacy Protection | Manual | 4.75±0.45 | 0.842 | 0.409 |
| AI | 4.58±0.51 |
| Dimension | Type | Mean±SD | t | p |
| Accuracy | Manual | 4.25±0.87 | 4.811 | 0.000** |
| AI | 2.42±1.00 | |||
| Comprehensiveness | Manual | 4.33±0.65 | 5.244 | 0.000** |
| AI | 2.67±0.89 | |||
| Clarity | Manual | 4.50±0.52 | 9.601 | 0.000** |
| AI | 2.25±0.62 | |||
| Redundancy | Manual | 4.08±0.51 | 8.509 | 0.000** |
| AI | 2.33±0.49 | |||
| Order & Structure | Manual | 4.42±0.67 | 0.632 | 0.534 |
| AI | 4.25±0.62 | |||
| Objectivity | Manual | 4.08±0.79 | -0.327 | 0.747 |
| AI | 4.17±0.39 | |||
| Ethics & Privacy Protection | Manual | 4.58±0.51 | 0.793 | 0.436 |
| AI | 4.42±0.51 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).