Artıfıcıal Intellıgence Applıcatıons ın Prımary Educatıon: A Quantıtatıvely-Supported Mıxed-Meta Method Study

1. Introductıon

By the end of the 20th century, advancements in technology and technical fields caused profound changes in societies. Today, information technologies lie at the heart of constantly evolving education systems (Candan & Başaran, 2023). During this period, one of the most notable developments among existing technologies is artificial intelligence. Artificial intelligence can be defined as a technology capable of mimicking certain abilities of computer systems (Russell & Norvig, 2009). This technology, characterized as machines that can autonomously think, analyze large datasets, recognize patterns, learn, and solve problems, has been shaping the future of computer systems and has become an indispensable part of daily life (Akerkar, 2014; Ginsenberg, 2012). Thanks to the advancing intelligence of computers, continuous innovations in software and hardware are emerging, integrating various technologies like robots, smart home systems, and autonomous vehicles into our daily lives (Komalavalli et al., 2020).

Artificial intelligence has a wide range of applications across fields such as healthcare, finance, manufacturing, and education. For instance, in the healthcare sector, artificial intelligence can analyze hospital data to create diagnostic and treatment plans and provide preventive healthcare services (Topol, 2019). In financial services, artificial intelligence can be used to prevent fraud and enhance customer service (Litan, 2021). In the industrial sector, it offers advantages such as lower-cost and widespread production, along with safer transportation options (Hamet & Tremblay, 2017; Li et al., 2017). The rapid progress of artificial intelligence is deeply transforming our lifestyles, learning methods, and ways of conducting business. The proliferation of innovative artificial intelligence applications has significant implications for society and future generations. Artificial intelligence has moved beyond professional and academic research to become a vital component of everyday life. Thus, artificial intelligence education needs to evolve from being limited to experts and be extended to the general public (Chiu & Chai, 2020).

While artificial intelligence topics are generally addressed in higher education, they have begun to be implemented in primary and secondary school classrooms worldwide as part of a strategic initiative (Pedró et al., 2019). This initiative aims to educate future generations. Throughout artificial intelligence education, extending even to secondary education levels, children not only become aware of emerging technologies and how they work but also receive inspiration to become the future users, designers, software developers, and researchers of artificial intelligence (Pedró et al., 2019).

1.1. Artificial Intelligence in Education

As in many fields, artificial intelligence (AI) is being utilized in education. The significance of applying AI in the education sector became evident during the COVID-19 pandemic, which resulted in a shift from traditional classroom teaching to digital learning methods (Maqbool et al., 2021; Mijwil et al., 2022; Pantelimon et al., 2021). In fact, studies on the use of AI in educational settings began as early as the 1980s (Self, 2016). Over the past five decades, research in the literature has focused on AI-supported educational applications (Bracaccio et al., 2019), language teaching processes in educational environments, systems for evaluating and supporting individual performance (Santos, 2016), and intelligent teaching systems and environments (Roscoe, Walker, & Patchan, 2018).

These studies indicate that using AI in education can provide more effective learning experiences by personalizing learning, helping students discover their abilities, fostering creativity, and reducing teachers’ workload (Liang & Chen, 2018; Xue & Li, 2018). Additionally, the literature highlights that AI can support students with special needs, offer content tailored to different learning styles, analyze students’ learning processes to focus on their individual needs, and guide them in career planning (Mu, 2019). These findings align with the three paradigms defined by Ouyang and Jiao (2021) for the use of AI in education: guiding, supporting, and integrating students into enhanced practices.

While such findings showcase the benefits of AI for students, studies examining teachers’ perceptions of AI reveal that teachers often see AI as a professional threat that could replace their jobs (Luckin et al., 2016). However, recent research indicates that these perceptions are changing, as teachers now expect AI to bring significant advancements to education in various settings (Panigrahi, 2020). Teachers’ perceptions of AI systems depend on factors such as their pedagogical beliefs, teaching experience, and history of using educational technology. These factors can influence their willingness to adopt new educational technologies (Ryu & Han, 2018).

Studies exploring teachers’ perceptions of AI in education (AIE) reveal that teachers see AI as a tool that can contribute to the teaching-learning process by providing digitized learning materials and facilitating human-computer interaction (Jia et al., 2020). They also perceive AI as a means to address learning difficulties faced by students in overcrowded classrooms (Holmes et al., 2019). Moreover, research indicates that teachers believe AIE can reduce their administrative workload by taking over repetitive and simple tasks (Qin et al., 2020).

1.2. Artificial Intelligence in Elementary Schools

Artificial intelligence (AI) is becoming increasingly significant in our daily lives, with applications in disaster prevention, agriculture, civil engineering, and healthcare, among others (Avci et al., 2021; Jain et al., 2020; Nichols et al., 2019; Sharma et al., 2020). Educating all age groups, from elementary school students to the elderly, about AI and how to use it is crucial for fostering a society where AI is effectively utilized. AI education should not be limited to higher education but should be integrated into all levels, starting from elementary school through secondary education (Su & Yang, 2022).

In 2019, the United Nations Educational, Scientific and Cultural Organization (UNESCO) evaluated educational programs provided by governments and institutions, adopting a policy to support research aimed at developing standardized AI curricula for K-12 education (UNESCO, 2019). This initiative has led to an increase in research on K-12 education and the emergence of conferences and journals dedicated to the topic (Yue et al., 2022).

As children increasingly interact with new technological tools and engage with learning processes through AI systems, the importance of AI education tailored to children is growing. For instance, Francis et al. (2021) found significant improvements in elementary students’ spatial reasoning skills after participating in short- or long-term robotic interventions.

Chu et al. (2014) developed a Model-Tracing Intelligent Tutor (MIT) to interpret students’ mathematical problem-solving behaviors, diagnose learning difficulties, and provide individual feedback. Their findings showed that the model-tracing-based learning diagnosis system significantly outperformed traditional web-based tests in helping students learn mathematics.

Calero et al. (2019) investigated the impact of robotics-based education on students’ mental rotation abilities. Their study revealed that robotics-based education significantly improved boys’ mental rotation skills compared to the control group, while no significant difference was observed for girls. Although the study provides evidence of the potential of robotics-based education, it also highlighted the need for future research to deeply analyze gender differences in learning outcomes achieved through educational robotics.

Yang (2022) emphasized the importance of introducing children to AI concepts at an early age and proposed a curriculum design focusing on why, what, and how to teach AI. Similarly, Holmes et al. (2019) echoed this perspective, underscoring the value of early AI education for children.

1.3. Purpose and Significance of the Study

The 21st century has witnessed revolutionary transformations in technology, reshaping the landscape of education. These changes have shifted the acquisition of knowledge beyond individual memory or traditional classroom settings, incorporating interactions through online networks and communities. Consequently, the learning process is now conceptualized as a network structure where knowledge is constantly updated by connecting various nodes of information.

The connectivism theory emerges within this context, emphasizing its effectiveness in fields where knowledge evolves rapidly and requires continuous updating, such as technology and digital literacy. Connectivism perceives learning as a multi-layered process occurring across networks (Siemens, 2004). In other words, it aims to redefine teaching and learning processes in knowledge societies shaped by emerging technologies. Furthermore, connectivism is particularly relevant in explaining learning within e-learning environments (Goldie, 2016) and contributes to the modernization of traditional education by integrating e-learning scenarios (Guerra, 2022). In light of these perspectives, connectivism provides significant contributions to educators aiming to support modern educational settings with digital technology, particularly through AI applications. Artificial intelligence (AI) is defined as the capacity of a digital device to perform tasks typically attributed to humans (Chiu, 2021). In today’s era, innovative technologies like AI are increasingly playing a critical role in classroom environments and students’ learning processes (Humble & Mozelius, 2019; Kaplan-Rakowski et al., 2023). AI-related studies focus on various aspects, including enhancing student learning, supporting teaching practices, evaluating student performance, and improving school administration (Chiu et al., 2023). Among these, research on student learning has centered on academic performance, 21st-century skills, and student motivation and engagement (Fu et al., 2020; Li & Su, 2020; Luo, 2018).

When viewed holistically, it becomes evident that there is a gap in studies examining the effects of AI applications on various variables in elementary education, particularly using a mixed-meta approach that integrates quantitative analysis. This study aims to explore the influence of AI applications on different variables in three dimensions;

• Quantitative dimension: Meta-analysis
• Qualitative dimension: Meta-thematic analysis
• Quantitative teacher feedback: Rasch measurement model

The study intends to provide a unique perspective to the literature by integrating these methods. Accordingly, in line with a document analysis framework based on a mixed-meta methodology:

Meta-Analysis

• Identifying the general effect size of different variables on AI applications.
• Determining the effect size levels of AI use in terms of courses, implementation duration, and sample size.

Meta-Thematic Analysis

3.

Exploring the effects of AI applications on learning environments, identifying potential challenges in the application process, and offering solutions.

Rasch Measurement Model (Teacher Feedback)

4.

Analyzing teachers’ general opinions on AI applications.

5.

Examining jury tendencies towards strictness or leniency in evaluations.

6.

Conducting item difficulty analysis for criteria related to AI applications.

2. Method

The methodology of a scientific study encompasses every stage of the research process, from its justification and content to the concepts used, measurement techniques applied, the analysis of collected data, and the interpretation of results (Koçel, 2017). Methodology can be classified into qualitative, quantitative, and mixed-methods approaches (Toraman, 2021). Among these, methodological pluralism, which combines quantitative and qualitative methods, has gained significant traction in fields such as education and sociology (Molina-Azorín et al., 2012).

Methodological pluralism involves the holistic analysis of quantitative and qualitative data through document analysis (Creswell & Sözbilir, 2017). Mixed-methods research, as defined by Creswell et al. (2003), aligns with this concept of methodological pluralism. It involves the integration of qualitative and quantitative approaches, either concurrently or sequentially, to create a unified dataset.

This study adopts a methodological pluralism framework to evaluate the use of AI in primary schools, encompassing both qualitative and quantitative analyses. It employs a mixed-meta approach, which integrates quantitative and qualitative methods within a single research process. Specifically, the methodological process includes:

• Meta-analysis: A quantitative synthesis of data to determine the effect size of AI applications.
• Meta-thematic analysis: A qualitative examination of recurring themes in the literature, focusing on the effects of AI applications in educational contexts.
• Rasch measurement model: A quantitative analysis of participant opinions, providing insights into teacher perspectives and evaluating response consistency.
• This combination of methods ensures a scientifically robust and holistic research process. The research integrates findings from meta-analysis, meta-thematic analysis, and the Rasch model to comprehensively analyze AI applications in education. A visual representation of this methodological framework is presented in Figure 1.

The mixed-meta method involves the holistic analysis of quantitative and qualitative data based on document examination. In other words, mixed-meta allows for the analysis of quantitative data using statistical programs such as CMA/MetaWin, and qualitative data using software like Nvivo/Maxqda, thereby enabling the integration of both data sets into a single study framework. This makes the method a comprehensive and rich approach to research (Batdı, 2021). The Multiple Complementary Approach (McA), a design within this method, integrates meta-thematic analysis with the quantitative research process to address the shortcomings of meta-thematic findings, support the obtained results, and provide a more holistic perspective (Batdı, 2024b). The analysis of scientifically valuable quantitative and qualitative data is crucial in this research process. In this context, the mixded-meta method integrates meta-analysis, meta-thematic analysis, and Rasch analysis based on participant views in the quantitative dimension. This three-phase approach enhances the depth of the research and ensures the generation of more comprehensive and valid results.

Method Phases:

Meta-Analysis: Statistical analysis of quantitative data, determining the impact of different variables on AI applications.

Meta-Thematic Analysis: Thematic analysis of qualitative data, exploring the impact of artificial intelligence applications on education.

2.1. Meta-Analysis Process

In the first dimension of the research, meta-analysis has been used. Glass (1976) defined the concept of meta-analysis as a statistical analysis process aimed at integrating research findings by combining the analysis results obtained from individual studies. Meta-analysis is a statistical method that, instead of examining individual studies one by one, aims to gather various studies effectively and reliably to produce broader and more meaningful results (Tsagris and Fragkos, 2018). In this dimension of the research, the meta-analysis process combines the general effect size of studies involving artificial intelligence applications at the elementary school level, as well as the effect sizes related to lessons, application duration, and sample size. By doing so, the results of various studies conducted at the elementary school level are effectively and validly integrated, leading to a holistic conclusion (Tsagris and Fragkos, 2018).

2.1.1. Data Collection and Analysis

Research on the use of artificial intelligence in elementary schools has been conducted in both English and Turkish using the keywords “impact/effectiveness of artificial intelligence use/in elementary schools” in the literature. During this study, databases such as YÖK, Google Scholar, Web of Science, Taylor & Francis Online, Science Direct, and ProQuest Dissertations & Theses Global were searched. The search was conducted according to the inclusion criteria listed below. These criteria are shown in Table 1.

Table 1 outlines the inclusion criteria used to select studies for the meta-analysis. Studies that did not meet these criteria were excluded from the analysis. In this context, studies that lacked access permission, did not contain quantitative data, lacked the necessary data for analysis, were found in multiple databases, or did not involve an experimental process were excluded from the analysis. The number of studies included and excluded, along with the reasons for exclusion, are shown in the PRISMA flow diagram (Moher et al., 2009) in Figure 2.

As shown in Figure 2, as a result of the screenings, in the first phase, N=1475 studies examining the impact of various variables on the use of artificial intelligence were identified. Out of these studies, 182 were excluded due to duplication, 486 due to irrelevant topics, 437 due to failure to meet inclusion criteria identified through abstract reading (NMA=437), and 64 due to insufficient digital data (NMA=64). As a result, as presented in Figure 2, 12 studies were included in the meta-analysis. Additionally, inter-rater reliability was calculated using the formula [agreement / (agreement + disagreement) × 100] proposed by Miles and Huberman (1994), and the reliability level of the research was determined to be .90. The data were analyzed using the CMA 2.0 program.

2.1.2. Effect Size and Model Selection

Meta-analysis procedures were carried out using CMA 2.0 software. The effect size (g) obtained from the analyses was interpreted according to the effect levels defined by Thalheimer and Cook (2002). The data were assessed and interpreted within the framework of the random effects model (REM). Schmidt et al. (2009) pointed out that the use of the fixed effects model (SEM) is limited in most cases, emphasizing that REM is a more appropriate option. Therefore, in this research, REM was preferred.

2.1.3. Moderator Analysis

In the meta-analysis of studies on artificial intelligence applications, a heterogeneity test was performed, and an I² value of 85.90 was found. This value indicates that the overall effect size of artificial intelligence applications, as well as different variables, could be evaluated. Values with heterogeneity rates of 75% or higher are considered high heterogeneity, so it is recommended to perform moderator analysis (Cooper et al., 2009). Therefore, in this research, to deepen the meta-analysis, various factors influencing artificial intelligence applications were examined, and conducting moderator analysis was deemed necessary.

2.1.4. Publication Bias

In meta-analyses, reliability is a key aspect. In studies where effect size is calculated, the inclusion of published studies or those with significant differences can raise concerns about publication bias. For this reason, certain calculations are made to assess publication bias in meta-analyses. Figure 3 shows the reliability calculations related to this in a funnel plot.

Figure 3 shows a funnel plot, which is presented as a visual summary of the meta-analysis dataset and also illustrates the potential for publication bias (Cooper et al., 2009). Funnel plots are used to detect publication bias in meta-analyses (Duval & Tweedie, 2000). In this plot, the horizontal axis represents effect size, and the vertical axis represents sample size, variance, or standard error values. The funnel plot highlights the relationship between effect size and sample size. As sample size increases, studies tend to cluster closer to the average effect size at the top of the plot (Sterne & Harbord, 2004). If there is no publication bias, the plot should form a symmetrical inverted funnel shape, as seen in Figure 3 (Rodriguez, 2001). Upon examining Figure 3, it can be observed that there is a balanced distribution of studies on both the left and right sides of the symmetry axis. This indicates that no publication bias was found in the research.

To test for publication bias, the calculation of Rosenthal’s fail-safe N is a method used for this purpose (Rosenthal, 1979). The N value indicates how many unpublished null studies would be required to invalidate the existing effect. A high N value suggests that the results are valid (Borenstein et al., 2009). In this research, the safe N value is 841. Comparing this with the number of studies included in the analysis, it can be said that this number is quite high, meaning no bias was detected.

2.2. Meta-Thematic Analysis Process

As part of the mixed-meta method, the second dimension of this study applied the meta-thematic analysis process. Meta-thematic analysis can be defined as a type of analysis where participants’ views (raw data) from qualitative research on a particular topic are reevaluated, and themes and codes are extracted (Batdı, 2021). In this study, qualitative research on the use of artificial intelligence at the primary school level was reviewed, and the meta-thematic analysis process was evaluated. In meta-thematic analysis, it is not the quantity of data that is important, but rather working with enough data to reach saturation. This type of analysis involves developing new codes and themes from data obtained through document analysis. In other words, meta-thematic analysis is a process based on text or verbal data that combines qualitative findings to generate new themes and codes. The themes and codes obtained in this process are included in the study to provide broader and more comprehensive results. In the qualitative dimension of this study, themes and codes related to the use of artificial intelligence at the primary school level were created using the meta-thematic analysis process (Batdı, 2019).

2.2.1. Data Collection and Review

Qualitative studies on the use of artificial intelligence at the primary school level were gathered using the document review method. These studies were reviewed after a preliminary reading, and the studies that were considered appropriate were selected through four stages: initial reading, full reading, and review of the findings section. The review phase involved reading the title, short summary, and expanded summary before moving to the findings section. After examining the findings section, it was decided whether the study would be included in the current research (Batdı, 2024a). As indicated in the PRISMA diagram, the studies were selected after a four-stage process. Ultimately, four studies were reviewed in the meta-thematic analysis process. Document analysis is defined as the process of collecting, examining, evaluating, and analyzing different documents as primary sources of research data (Sak et al., 2021). In other words, document analysis includes a series of processes involving the examination and evaluation of printed and digital (computer-based and internet-accessible) data (Bowen, 2008). In the processing of meta-thematic data obtained through qualitative research using document analysis, the Maxqda program was used. The data was transferred into the program, and content analysis, a commonly used method (Bowen, 2008), was applied to analyze indicators, comments, and discourses (Bryman, 2016). In this context, the participant opinions obtained in this research were reanalyzed to generate different codes and themes.

2.2.2. Coding Process

An important phase of the meta-thematic analysis process is coding. The opinions of participants were reformulated, and codes were created, grouping similar codes under common themes. In this phase, the findings of the meta-thematic analysis process, which were based on the opinions of participants from the reviewed studies, included the themes and codes determined for the study’s main objectives. The reliability of the coding process is demonstrated by ensuring that the generated codes are consistent with each other (Mayring, 2000). In this study, two themes were created at the end of the meta-thematic analysis process: “the impact of AI applications on learning environments” and “problems and solutions encountered in AI applications.” Two coders independently created the themes and codes. One of the coders was the researcher, and the other was an academic expert in the field. After completing the coding process, the themes and codes created by both coders were compared to check for consistency and agreement. Similar themes and codes were recorded jointly, while discrepancies were discussed until agreement was reached between the coders. The reliability between coders was calculated using Cohen’s Kappa coefficient (Cohen, 1960). The Kappa value for the theme “Impact of AI applications on learning environments” was 0.79, and for the theme “Problems and solutions encountered in AI applications” it was 0.86 (Appendix A). The Kappa values ranging from 0.79 to 0.86 indicate a good to very good level of agreement between the coders.

2.2.3. Reliability in the Meta-Thematic Analysis Process

Reliability methods used in qualitative research were also applied to the meta-thematic analysis in this study. During this phase, the concept of researcher triangulation (Streubert & Carpenter, 2011) was used, where two researchers collaborated throughout the meta-thematic analysis process. Furthermore, direct quotes from participant expressions were included to provide the raw data source while creating the themes and codes. The related quotes were expressed with codes indicating the study and page number from which the quotes were taken. For example, the numeric expression M12-s.15 refers to M (article), 12 (study number), and “s.15” (page number). The codes obtained from the YÖK National Thesis Center were also abbreviated using thesis numbers. Through the meta-analysis of studies conducted on primary school AI use, effect sizes were identified, and codes and themes were created through the meta-thematic analysis. To support these findings and provide a holistic table, the opinions of class teachers were analyzed using the Rasch measurement model.

2.3. Rasch Measurement Model Analysis Process

In the third dimension of the research, quantitative data related to teacher opinions on the use of AI in primary schools were analyzed using the Rasch measurement model, developed by Linacre (1993; 2008). This model is based on objectivity (Semerci, 2011). It also evaluates other variable sources that may affect the test results, such as item difficulty levels, raters, scoring keys, conditions, tasks, and scoring criteria (İlhan, 2016; Lynch & McNamara, 1998). The Multi-Faceted Rasch model establishes necessary linear relationships between different facets (such as AI application usage levels, evaluation item characteristics, and rater behaviors) and forms these connections (Hambleton & Swaminathan, 1985). In the current research, the Rasch model includes three facets: 21 primary school teachers, 18 items related to AI use in education, and four educational programs (Turkish, Mathematics, Life Science, and Science). To collect data, the researchers created the “Primary School Teachers’ AI Application Usage Evaluation Form.”

2.3.1. Study Group

The study group consisted of 21 classroom teachers working in state primary schools during the 2024-2025 academic year. The raters independently evaluated the teachers’ AI application usage levels using the 18-item evaluation form.

2.3.2. Research Data and Analysis

To collect data for the study, the “Primary School Teachers’ AI Application Usage Evaluation Form” was used (Appendix B). The evaluation form, developed within the framework of the Rasch measurement model and consisting of 18 items, assesses teachers’ knowledge acquisition, knowledge deepening, and knowledge creation under the headings of curriculum and assessment, pedagogy, and the application of digital skills. The form includes evaluations of the Turkish, Mathematics, Science, and Life Science curricula. The evaluation form was created based on a literature review (Ministry of National Education [MEB], 2024; UNESCO, 2018) and expert opinions. Afterward, the form was re-examined by experts for verification, and necessary additions and deletions were made. This form contains a total of 18 criteria. The content validity index (CVI) of the items was calculated using the Content Validity Ratio (CVR) formula developed by Lawshe (1975) and found to be 0.81 (Appendix C). According to Veneziano and Hooper (1997), this value is statistically significant at the 0.05 level. The analysis of the AI application evaluation form was conducted using the FACETS analysis program developed by Linacre (1993: 2-15). FACETS is an analysis program frequently used in the multi-faceted Rasch measurement model. This program generally includes three main facets: rater, ability, and task (Güler & Gelbal, 2010).

3. Findings

This section interprets the findings obtained through meta-analysis, meta-thematic analysis, and participant opinions on various variables related to the use of AI applications in both classroom and out-of-class teaching environments. In the first phase of the study, meta-analytic findings based on document analysis are presented; in the second phase, results of the meta-thematic analysis; and in the third phase, participant opinions supported by quantitative data.

3.1. Meta-Analysis Findings on AI Applications

When examining the meta-analysis findings in Table 2, it was determined that the effect size of academic achievement scores (AA) for AI-supported applications, calculated using the REM method, was g = .51 [.28; .74]. This effect size is classified as moderate according to Thalheimer and Cook’s (2002) classification, indicating that AI applications have a positive and beneficial effect on these variables. Additionally, a significant difference was found between the scores based on test types (p < .05).

When examining the results of the heterogeneity test in Table 2, it is seen that the effect sizes of the scores for AI applications exhibit a heterogeneous distribution (QB = 163.11; p < .05). The I² value was calculated as 85.90%, indicating that the observed 86% variance originates from the true variance among the studies. According to Cooper et al. (2009), an I² value of 25% is considered low, around 50% is moderate, and 75% or higher is high heterogeneity. The calculated I² value of 85.90% in this study points to a high level of heterogeneity (Higgins et al., 2003). This suggests the presence of moderator variables affecting the total effect size. In other words, the detection of high heterogeneity indicates the need for a moderator analysis (Borenstein et al., 2009).

As presented in Table 3, the duration of implementation and sample size have been selected as moderator analyses. In the significance test, a significant difference was found in the duration of implementation (QB = 7.69; p < .05); however, no significant differences were observed in terms of lessons (QB = 1.7; p > .05) and sample size (QB = 1.29; p > .05). Nevertheless, when the analysis results are generally evaluated, it can be stated that various variables have a moderate effect on artificial intelligence applications.

3.2. Meta-Thematic Findings Regarding Artificial Intelligence Applications

In this part of the study, the themes and codes obtained through meta-thematic analysis of the studies accessed regarding the effects of various variables on artificial intelligence applications are presented. The data related to the effects of different variables on artificial intelligence applications were analyzed, and two themes were formed: the impact of artificial intelligence applications on educational settings and challenges encountered and solutions proposed. The themes and codes are illustrated in models. Additionally, direct quotations are provided as reference sentences within the comments.

When examining Figure 4, the contributions of artificial intelligence applications to learning environments can be seen in the model as codes. Some of the codes in this model can be explained as providing support for challenging subjects, presenting topics through visualization, enhancing problem-solving skills, fostering intrinsic and extrinsic motivation, supporting socio-emotional development, and improving presentation skills. Expressions and quotations that can serve as references to these codes are stated in the studies as follows:

“I like to learn English with it (the AI coach) as it helps improve my English competence,” and

“You can get (virtual) flowers and awards if you practice English with the AI coach every day and achieve good performance”.

(M1-p. 6)

In another study: “He also started to improve his oral skills, and finally gave presentations in front of large audiences”.

(M2-p. 1890)

Additionally, in the study coded (M5-p. 12), it is stated as: “I thought it was interesting because I was able to actually control the AI to play rock-paper-scissors with only two fingers.”

Furthermore, it can be stated that artificial intelligence applications help students develop critical thinking skills, promote learning through exploration, enhance problem-solving skills, and foster the acquisition of various other competencies. However, some challenges may be encountered during the implementation of these applications. The problems faced by practitioners and the proposed solutions are presented in the figure below.

When examining Figure 5, the problems encountered in learning environments during the artificial intelligence (AI) application process and the corresponding solutions are visualized. Some of these issues are indicated with the following codes: AI applications being deemed insufficient, difficulty in understanding questions asked, lack of knowledge about AI, inability to produce adequate outcomes, concerns about personal data being compromised, lack of emotional support from AI applications, and anxiety related to AI. Expressions related to these codes include:

“In my opinion, artificial intelligence is software that humans create, that we decide on the different things it learns and, after that, the computer adds them to other things it learned, like when we trained it with pictures, we also showed some pictures, and not always the computer was right, so we tried to give it more pictures and teach it things”.

(M3, p. 187)

In the study (M1, p. 10), the limitations of AI in sentence formation and vocabulary memorization were stated as: “For instance, one student indicated that ‘… it (the AI coach) improves our oral English through several ways, such as English shadowing, mimicking picture books, and memorizing vocabulary…’ Another student wrote, ‘it (the AI coach) contains many resources linked to our textbooks, picture books, and movie clips for budding practice.’”

Additionally, concerns about AI were expressed as follows: “I had concerns about the potential for AI to dominate the world, given its ability to complete tasks in just a few steps” (M5, p. 12).

During the analysis process, solution suggestions for potential problems in AI applications were identified. Some of the proposed solutions include: Students should actively engage in learning efforts with AI; AI applications should be used through teamwork; AI application activities should be conducted interactively and actively; security measures should be prioritized in AI applications.

Quotations referencing these suggestions include the following:

In the study (M2, p. 1822), “Students building basic robotic models benefit when they are working individually; meanwhile, students might work better in teams (ideally, two to three members per team) when working on advanced robotic models that include writing code (programming).”

Another example states: “In order to reduce such fraud, I think it is more convenient and accurate to use AI than humans” (M4, p. 12).

In conclusion, while certain issues may arise during AI applications, these problems can be resolved by utilizing the proposed solutions.

Findings derived from classroom teachers’ opinions have been tabulated and expressed using the Rasch measurement model. This approach provided quantitative support to the data obtained through the mixed-meta process, presenting a detailed interaction of all the dynamics of the study.

3.4. Findings Related to the Rasch Measurement Model for Artificial Intelligence Applications

Since the research was conducted using a quantitatively supported mixed-meta method, this section presents the findings related to AI applications in primary school education using the multi-faceted Rasch measurement model. When analyzing the AI applications, the surfaces used in the study (teachers’ AI applications, judges’ strictness/generosity, and the appropriateness of the items used) and general information about these surfaces are provided in the calibration map shown in Figure 6.

In the findings, four curricula (Turkish, mathematics, life sciences, and science education curricula), 21 judges, and 18 evaluation items related to the program content were taken into consideration. In this calibration map, it was found that the use of AI applications was at a high level in the science and mathematics curricula (Science Education Program: FBP-4 and Mathematics Program: MP-2), while it was observed to be at a low level in the life sciences and Turkish education programs (Life Sciences Program: HBP-3 and Turkish Program: TP-1).

Among the judges (raters), J2 was identified as the most generous rater, while J11 was determined to be the strictest rater.

When the column containing the items related to the use of AI applications was examined, the item “I can enable students to prepare a presentation describing their environment using AI tools” (I17) was found to be the most difficult item, whereas the item “I know how to efficiently use AI applications in classroom activities” (I14) was identified as the easiest item.

After evaluating the findings from the calibration map, the analysis report prepared for the curricula is presented in Figure 7.

As a result of the Rasch analysis, the reliability coefficient was determined to be 0.91. This coefficient value indicates the reliability with which the curricula were ranked. Furthermore, when the data in Figure 7 were analyzed, it was observed that there were statistically significant differences between the curricula (X²=44.4; df=3; p=0.00). Additionally, the standard error (RMSE) of the logit values related to the curricula was found to be 0.07, indicating a very low level of error.

This error rate, with its adjusted standard deviation, is below the critical value of 1.0, which is considered acceptable. In this context, the order of AI usage rates in the curricula from high-quality to low-quality can be listed as follows:

FBP-4 (Science Education Curriculum)

MP-2 (Mathematics Education Curriculum)

HBP-3 (Life Sciences Curriculum)

TP-1 (Turkish Education Curriculum)

In Rasch analysis, the quality control limits for “infit” (internal consistency) and “outfit” (outlier-sensitive consistency) values are accepted to be between 0.6 and 1.4 (Wright & Linacre, 1996). During the decision-making process, the “infit” values are used for unexpected responses from judges, while “outfit” values are used for unexpected distant responses (Baş Türk & Işıkoğlu, 2008; Batdı, 2014a). Upon examining the values in Figure 7, it is understood that these limits are appropriate.

In Figure 8, information regarding the strictness/leniency of the judges in relation to their use of AI in secondary education is presented. In Figure 7, it is observed that the judges’ scores are ranked from the strictest to the most lenient. The judge identified with the code J2 is seen to be the most lenient, while the judge identified with the code J11 is the strictest. Additionally, the judge separation index was observed to be 6.44, and the reliability coefficient was noted as 0.98. A statistically significant difference was also found between the strictness/leniency of the judges’ scores (X²=872.8; df=20; p=0.00).

As a result, it was determined that the score observed in J2, with 283, was the most lenient, while the score observed in J11, with 150, was the strictest.

When the infit (internal consistency) and outfit (outlier-sensitive consistency) values of the surfaces in Figure 8 were evaluated, it was found that the judge identified with the code J15 did not meet the infit and outfit criteria (range of 0.6 to 1.4). In this case, it can be stated that the mean of the infit and outfit squares for the judge coded as J15 fell outside the defined limits. This can be interpreted as the lenient judge (J15) not exhibiting consistent scoring behavior while evaluating AI applications.

On the other hand, it was noted that the other judges (17 judges) demonstrated consistency in scoring among themselves, as their values fell within the expected quality control range, and thus, they can be considered as appropriate.

Upon examining the results of the data analysis presented in Figure 9, it was observed that the separation index is 3.04, and the reliability coefficient is 0.90. This reliability coefficient indicates that the items used in the study to determine teachers’ levels of using AI tools are reliable. Additionally, there were statistically significant differences among the item difficulty levels used to evaluate teachers’ opinions regarding their use of AI tools (X²=181.7; df=17; p=0.00).

The most difficult items related to the use of AI applications were identified as:

“I can guide students to prepare a presentation describing their environment using AI tools.”

“I can enable students to create simple patterns (e.g., drawing, storytelling) using AI tools.”

“I can integrate AI applications into pedagogical methods and techniques to contribute to students’ learning processes.”

On the other hand, the easiest items were:

“I know how to use AI applications effectively in classroom activities.”

“I know how AI applications can support teaching activities in a classroom setting.”

“I can encourage students to collaborate and improve their skills using AI tools.”

The standard error (RMSE) for the analysis of criteria prepared by the researchers to determine the level of teachers’ use of AI applications was found to be 0.15, indicating that the standard error related to identifying the levels of AI application use is quite low.

Furthermore, when the infit (internal consistency) and outfit (outlier-sensitive consistency) statistical values for the research surfaces in Figure 9 were examined, it was noted that only the item coded as I16 exceeded the outfit threshold. In this context, it can be stated that the mentioned item demonstrates inconsistency in evaluating the levels of AI use.

All other items, however, were within the specified consistency value limits, indicating their reliability in assessing AI application usage levels.

4. Discussion and Conclusions

In this study, a mixed-meta method, integrated with a quantitative research process, was employed to examine the effects of studies conducted at the primary school level on various variables and to reveal the current state of the literature in comparison with other studies.

In line with the purpose of the research, the following methodological steps were followed:

Meta-analysis was conducted first.

This was followed by meta-thematic analysis.

Finally, the participants’ opinions were collected in the quantitative dimension and evaluated using the Rasch measurement model.

4.1. Results of the Meta-Analysis Process

The study includes 37 quantitative studies for the meta-analysis, while 4 qualitative studies were included in the meta-thematic analysis process. In the quantitative scope, the teachers’ views on the relevant variables were collected using the prepared evaluation form. The findings based on the data obtained from the analysis are presented in this section. According to the meta-analysis, the results of the included studies based on the REM showed that the effect of various variables on the use of artificial intelligence was moderate (g = .51 [.28; .74]), based on Thalheimer and Cook’s (2002) classification. The results of other studies in the literature, which were not included in this study, support the research results (Gilad & Levin, 2021; Kajiwara et al., 2023). Based on these results, it can be stated that different variables have an impact on the use of artificial intelligence applications.

During the analysis, a moderator analysis was also conducted regarding the effects of various variables on the use of artificial intelligence. The first moderator analysis, related to the duration of implementation, indicated that a duration of 1-4 weeks had an effect size (g = 0.59). Secondly, when subjects were used as moderators, the highest effect size (g = 0.81) was found for other subjects. Thirdly, regarding sample size as a moderator, large sample groups showed the highest effect size (g = 0.63). When analyzing the effect sizes of moderators, a significant difference was observed for the duration of implementation (QB = 7.69; p <.05), while no significant difference was found for subjects (QB = 1.7; p >.05) and sample size (QB = 1.29; p >.05). The findings of studies in the literature, such as those by Kablan et al. (2013) and Camnalbur et al. (2018), support the findings of this study in terms of subjects and sample size. In contrast to the studies in the literature, no effect on academic achievement was observed with time durations, but this research found a significant difference in the moderator related to implementation time. When considering the results of the moderator analyses as a whole, it can be stated that artificial intelligence applications have a moderate impact across all groups with similar effect sizes. During the meta-analysis process, it was observed that the majority of studies regarding artificial intelligence applications were conducted in mathematics classes. In this context, further research on the use of artificial intelligence applications in other subjects is recommended.

4.2. Results of the Meta-Thematic Analysis Process

In the second phase, a document analysis-based meta-thematic analysis was conducted, integrating the findings from the meta-analysis to validate them and expand the scope of the results. The themes and codes regarding the effects of artificial intelligence applications on learning environments and the problems and solutions encountered during their use were formed. The analysis revealed that students were in an interactive learning environment while using artificial intelligence applications. It can be stated that being highly interactive in the constructivist learning process helps make artificial intelligence applications more comprehensible (Shamir & Levin, 2021). One of the codes reached in the research was that artificial intelligence applications support individuals who learn at different paces and paths. Similar findings were found in Pilco (2020). It can be said that an individual’s self-confidence in any task contributes to their success in that task. Our research concluded that artificial intelligence applications increase individuals’ self-confidence and enhance their creativity. This result is supported by findings from other studies using digital technology to support student learning (Bers et al., 2014; Fitton et al., 2013).

In our research, it was also found that artificial intelligence applications improve problem-solving skills, creative thinking, and higher-order thinking abilities, aligning with findings in the literature indicating that technology-supported lesson content enhances students’ cognitive capacities (Garrison, 2017; Ke, 2010).

Another theme created in our study is the problems and solutions encountered during the use of artificial intelligence applications. Studies indicating that individuals with low self-efficacy experience high computer anxiety (Ellis & Allaire, 1999) align with the findings in this research. The research findings show that students’ anxiety while using artificial intelligence applications may be related to their low self-efficacy in this area. One of the codes found was that students were disappointed by the responses provided by artificial intelligence during the process. Similar to this finding, the research of Toivonen et al. (2020) suggests that poor performance of the applications does not meet students’ expectations. Another issue identified in the study is that artificial intelligence applications do not provide emotional support. Literature review shows that lack of emotional support in artificial intelligence applications can negatively affect the targeted lesson outcomes (Randall, 2019; Wang, 2024). Another identified issue is that students’ feedback on artificial intelligence usage is not understood by the application. Findings from studies in the literature (Chen et al., 2020; Wang et al., 2018) support this finding in our research.

It is suggested that artificial intelligence applications should have an attractive design, as this will encourage students to develop a positive attitude towards using them (Reeves et al., 2020). Based on this, it can be concluded that artificial intelligence applications should be produced with engaging designs. Another problem encountered in the process is that educators do not have the necessary training, which prevents them from obtaining the desired results from the application. Our research found that teachers need sufficient knowledge and skills regarding artificial intelligence applications to overcome this issue. Pilco’s (2020) study also aligns with our findings, indicating that teachers should receive technical training on artificial intelligence.

4.3. Results Related to the Rasch Measurement Model Process

The quantitative section of our research presents the findings related to teachers’ views on their level of use of artificial intelligence applications, derived through analyses using the Rasch measurement model. In this way, the findings obtained through the mixed-meta processes were supported quantitatively to ensure the alignment of the study’s results. The Rasch measurement model adopts a fundamental approach that relates the probability of answering a question correctly to an individual’s ability (Baştürk, 2010). The multi-faceted Rasch measurement model, developed by John M. Linacre, not only examines the relationship between individuals’ ability levels and the difficulty levels of the items on the measurement tool but also allows the evaluation of other variable sources that might affect test results, such as scorers, scoring keys, conditions, tasks, and scoring criteria (Lynch & McNamara, 1998). In this context, simultaneous surfaces (the level of use of artificial intelligence applications, the rigidity/generosity of the jury, and the characteristics of the evaluation questions) that were prioritized and analyzed in the multi-faceted Rasch measurement model have been ranked among themselves. Awareness of artificial intelligence applications, a concept related to Industry 4.0, is crucial as these applications play a significant role in shaping the future of technology (Doğan & Baloğlu, 2020). In the current study, teachers’ views were examined in the context of curricula by associating dimensions such as curriculum and assessment, pedagogy, and the application of digital skills with sub-dimensions like knowledge acquisition, knowledge creation, and knowledge deepening.

The analysis revealed that the science and mathematics curricula had the highest level of use in the teaching programs. Artificial intelligence, which has made significant advancements in the past 50 years, has become an important research area (Talan, 2021). AI encompasses many cognitive areas of human intelligence, including learning, reasoning, planning, problem-solving, perception, natural language processing, deep learning, expert systems, image processing, sentiment analysis, speech recognition, and more (Goodfellow et al., 2016). Therefore, it can be said that the primary use of AI in science and mathematics is due to the functionalities of artificial intelligence itself.

When the scorers’ rigidity/generosity information was evaluated in relation to the assessment of AI applications, it was found that the jury defined by the J2 code was the most generous, while the jury defined by the J11 code was the strictest. Furthermore, it was determined that the scorers were reliably ranked in terms of rigidity/generosity and differed from each other. Studies using the multi-faceted Rasch measurement model in the literature also indicate that scorers (juries) can be both objective and biased at times (Baştürk, 2010; Batdı, 2014a; Köse et al., 2016; Semerci, 2012). In addition, the analysis concluded that the jury separation index was 6.44, which is above the desired level. This value indicates that there are differences in scoring among the juries, that juries vary according to their generosity/rigidity levels, and that scoring errors related to generosity/rigidity exist in the scores given by the juries (Uluman & Tavşancıl, 2017).

The items in the teacher evaluation form regarding the level of use of AI applications were found to serve the purpose of measuring teachers’ competence levels. When the items prepared to determine teachers’ level of use of AI were analyzed, it was found that the most difficult items were “I can make students prepare a presentation describing their environment using AI tools,” “I can make students prepare simple designs (like drawing pictures, creating stories) using AI tools,” and “I can integrate AI applications into pedagogical methods and techniques to support students’ learning processes.” Therefore, it can be said that teachers may face difficulties in using AI. Based on these results, it can be expressed that it is an area that needs further research on how AI can be used effectively in educational environments to achieve the desired outcomes and create designs based on the characteristics of the gains. On the other hand, the easiest items were “I know how to use AI applications effectively in classroom activities,” “I know how AI applications can support teaching activities in the classroom,” and “I can encourage students to collaborate and develop using AI tools.” Although the participants possess basic knowledge about AI, it can be said that teachers need to receive necessary training to stay up-to-date with developing technologies (Benvenuti et al., 2023).

When considering the results of the entire research process as a whole, it is evident that the obtained data support each other. In the meta-analysis process, it was found that AI studies in elementary schools were most frequently conducted in mathematics lessons (Zhang & Aslan, 2021). In the Rasch measurement model section of the study, it was observed that classroom teachers most commonly used AI applications in science and mathematics lessons. In this regard, it can be said that the results of these two processes overlap. Furthermore, during the meta-thematic analysis process, the theme of training practitioners in AI applications emerged as an issue and solution. In the Rasch measurement model in the jury opinions section, it was found that the most difficult items were related to teachers preparing presentations with AI, integrating AI applications into the curriculum, and preparing simple designs related to teaching practices, indicating that educators need training in AI applications. Thus, the results of the study’s processes are compatible. Moreover, during the meta-thematic analysis, the theme related to the effect of AI on educational environments was “providing interactive teaching opportunities,” which aligns with the item in the Rasch measurement model “I can encourage students to collaborate and develop using AI tools” being among the easiest items. In this way, it can be said that the results from the two processes are integrated, as both contribute to the creation of an interactive classroom environment where teachers facilitate collaboration among students using AI tools (Lin et al., 2023). In today’s technological age, the findings of this study, which involves both qualitative and quantitative research within the methodological pluralism framework, reflect how artificial intelligence, as the most prominent tool of the connectivity paradigm in modern educational environments, encapsulates and supports many of the findings.

4.4. Limitations

Although this study presents a mixed-methods approach supported by quantitative analysis within the framework of methodological pluralism, there are several limitations to consider. The data collected for the meta-analysis and meta-thematic analysis were limited to certain databases. The meta-analysis focused solely on the impact of AI applications on academic achievement. Moderator variables in the study included the duration of the intervention, the subject areas, and the sample size. In the Rasch measurement model, data was collected regarding the Turkish, mathematics, science, and life sciences curricula. Additionally, survey questions were developed based on teachers’ digital competencies (UNESCO, 2018), covering aspects such as curriculum and assessment, pedagogy, and digital skills. Finally, the study focused on gathering perspectives from classroom teachers.

4.5. Recommendations

• The application duration, subject areas, and sample sizes in AI-related research have significant effects on academic success and the impact of AI on educational environments. The use of the mixed-meta method, supported by the Rasch measurement model, has provided a more holistic perspective, allowing for a deeper exploration of the topic. Based on the limitations and findings of the study, the following recommendations are made:
• Research on AI applications in primary school subject areas such as art, music, and physical education can be conducted. In addition to quantitative methods, qualitative methods could be employed to explore the effectiveness and applicability of survey questions.
• The meta-analysis phase of the study could include investigations into the impact of AI applications on attitudes and long-term retention.
• Studies could explore teachers’ information and technology competencies (UNESCO, 2018) within other professional practice areas.
• The study focused on perspectives from classroom teachers. Including evaluators from different expertise levels could broaden the scope of the study.
• Despite teachers’ positive expectations regarding AI, it is essential that they first familiarize themselves with the technology and learn how to integrate it into their classrooms. Many teachers may regard AI as an advanced technological product without prior experience. In this regard, in-service training could increase teachers’ knowledge about AI and improve their integration of this technology, significantly enhancing student success and the learning experience (Kim NJ and Kim MK, 2022).
• Given the methodological diversity, the use of a mixed-meta method combined with quantitative analysis has allowed for a comprehensive examination of the findings, with detailed insights into how various variables affect the use of AI applications. Therefore, it is recommended to apply the mixed-meta method integrated with either qualitative or quantitative analyses in other areas to achieve comprehensive research findings.
• Policymakers should take necessary measures to address concerns related to ethics, data security, and human rights as AI becomes more integrated into education.