Scaling Socratic Statistics Instruction Through Large Language Models

1. Introduction

Statistics educators have long struggled with the challenge of helping students perceive statistics as a meaningful way of understanding uncertainty rather than as a collection of formulas and procedures. Traditional approaches frequently place instructors at the centre of classroom activity and students in the role of passive recipients of information. While such approaches may efficiently transmit content, they often fail to develop deeper forms of statistical reasoning and may contribute to statistics anxiety and negative attitudes toward the subject. In response to these concerns, Amin [1] proposed a Socratic approach to teaching introductory statistics that replaces direct instruction with disciplined inquiry, dialogue, and questioning. In this model, students engage with authentic real-world problems and are guided towards statistical concepts through carefully sequenced questions rather than extended lectures. The instructor functions as a facilitator of learning rather than as the primary source of knowledge. By encouraging learners to justify claims, examine assumptions, evaluate evidence, and construct understanding through discussion, Socratic teaching seeks to transform statistics from a subject of memorisation into a process of discovery.

The theoretical appeal of this approach is substantial. Statistics requires learners to evaluate evidence, reason under uncertainty, scrutinise assumptions, and defend conclusions. Consequently, a pedagogy grounded in questioning, inquiry, and critical reflection appears particularly well suited to the development of statistical reasoning. Amin [1] classroom-based implementation demonstrated how Socratic dialogue can be used to introduce statistical concepts through guided discovery rather than procedural exposition.

Despite its promise, a substantial limitation remains. Amin [1] explicitly acknowledges that effective Socratic instruction is difficult to scale because it requires small class sizes, highly skilled facilitators, extensive preparation, and sustained interaction with learners. Consequently, one of the most compelling pedagogical approaches in statistics education may also be one of the most difficult to implement broadly. This limitation raises an important question: can the benefits of Socratic pedagogy be preserved while reducing its dependence on intensive human facilitation?

Recent advances in large language models (LLMs) suggest a possible solution. Unlike traditional educational technologies, LLMs are capable of sustaining extended conversations, adapting to learner responses, probing reasoning, challenging assumptions, and providing immediate feedback. These conversational capabilities position LLMs as potential facilitators of inquiry-oriented learning. However, demonstrating that an LLM can engage in conversation is not sufficient to establish its value as a Socratic tutor. A more fundamental question is whether the dialogue generated by an LLM exhibits the characteristics associated with Socratic learning.

To address this question, the present study adapts the Cambridge Dialogue Analysis Scheme (CDAS) developed by Hennessy et al. [2] into a Socratic Learning Analysis Scheme (SLAS). The adapted framework is then applied to AI-mediated statistics tutoring dialogues to evaluate the extent to which they demonstrate discourse features associated with Socratic pedagogy. Rather than focusing on learning outcomes, the study examines the quality of educational dialogue itself. In doing so, it contributes to statistics education in three ways. First, it extends Amin [1] proposal by exploring a potential solution to the scalability problem associated with Socratic teaching. Second, it extends the CDAS framework by adapting it specifically for the analysis of Socratic learning interactions. Third, it contributes to the growing literature on artificial intelligence in education by providing a framework for evaluating whether LLM-generated dialogue exhibits characteristics consistent with Socratic learning.

2. Theoretical Background

2.1. Socratic Pedagogy

Socratic pedagogy traces its origins to the teaching practices of Socrates, who viewed knowledge as something to be discovered through disciplined inquiry rather than transmitted directly from teacher to learner. Through dialogue, learners are encouraged to examine their beliefs, articulate their reasoning, and refine their understanding in response to carefully constructed questions. Rather than serving as a source of answers, the teacher functions as a facilitator who guides learners toward deeper understanding through questioning and reflection [3,4].

At the heart of the Socratic method lies the belief that questioning is a route to knowledge rather than merely a tool for assessment. Questions are used to clarify ideas, probe assumptions, challenge reasoning, and stimulate intellectual exploration. The teacher intentionally withholds immediate answers, encouraging learners to participate actively in the development of understanding. Through this iterative process of questioning and response, learners move beyond passive reception of information and become active participants in knowledge construction [3,5]. Research suggests that such questioning-based approaches foster reflection, confidence in discussion, intellectual autonomy, and a more inquisitive disposition toward learning [6].

The theoretical foundations of Socratic pedagogy align closely with constructivist learning theory. Constructivism assumes that learners actively construct knowledge rather than simply receive it from an instructor. Learning occurs as individuals integrate new information with existing knowledge through reflection, exploration, and interaction with meaningful experiences [7,8]. From a constructivist perspective, the role of the teacher is not to provide knowledge directly but to create conditions that support the learner’s construction of understanding [7,9]. Socratic questioning operationalises this principle by encouraging students to generate explanations, evaluate competing ideas, and revise their thinking as they engage with new evidence and perspectives.

Constructivist perspectives also emphasize the social nature of learning. Vygotskian and related sociocultural theories argue that knowledge develops through interaction with others, language, and culturally situated forms of reasoning before becoming internalised by the learner [10]. Socratic dialogue provides a mechanism through which such social construction of knowledge can occur. As learners articulate ideas, respond to questions, and engage with alternative viewpoints, understanding emerges through interaction rather than individual memorisation. In this sense, Socratic pedagogy creates opportunities for learners to co-construct knowledge through guided intellectual exchange [7].

Socratic pedagogy is also closely connected to inquiry-based learning. Inquiry-based approaches position questions as the starting point of learning and encourage learners to investigate problems, gather evidence, formulate explanations, and revise conclusions through ongoing inquiry [11]. This view of learning mirrors the Socratic tradition, which similarly begins with questions rather than answers and places learners at the centre of the learning process. Both approaches emphasise curiosity, investigation, and the active construction of meaning. A growing body of evidence suggests that inquiry-based learning can enhance critical thinking, motivation, engagement, and academic achievement while promoting deeper and more transferable learning experiences [12,13].

A further justification for Socratic pedagogy lies in its relationship to critical thinking. Critical thinking is widely regarded as a core objective of higher education and involves both cognitive skills and dispositions that enable learners to evaluate claims, assess evidence, and make reasoned judgments [14,15]. Central to critical thinking is the ability to identify assumptions, evaluate their validity, consider alternative viewpoints, and weigh available evidence before reaching a conclusion [16]. Socratic questioning directly supports these processes by prompting learners to justify their claims, scrutinise underlying assumptions, and reconsider positions when confronted with competing explanations. Consequently, Socratic pedagogy may be understood not simply as a questioning technique but as a structured pedagogical approach for helping learners examine and reconstruct their understanding of the world [5,6].

2.2. Dialogic Learning Theory

While Socratic pedagogy provides the pedagogical foundation for the present study, dialogic learning theory provides the theoretical lens through which educational dialogue can be understood and evaluated. Dialogic theories share the assumption that learning is not merely an individual cognitive process but is fundamentally shaped through interaction, communication, and the exchange of ideas. Across this tradition, Vygotsky explains the role of social interaction in cognitive development, Bakhtin explains how meaning emerges through encounters between different perspectives, and Alexander and Mercer translate these theoretical insights into principles of classroom practice and educational dialogue [17,18,19,20].

The sociocultural foundations of dialogic learning are most commonly associated with Vygotsky. According to Vygotskian theory, higher mental functions originate in social interaction before becoming internalised by the individual. Language and other symbolic tools mediate thinking, making communication a central mechanism of learning rather than a simple means of transmitting information [19]. From this perspective, dialogue serves as a bridge between collective activity and individual cognition. As learners participate in discussions, explain ideas, justify reasoning, and respond to others, they engage in forms of thinking that are initially shared and later become internal mental processes [18,21]. Educational practices such as scaffolding, guided participation, and collaborative learning derive directly from this view of learning as socially mediated activity [22].

Although influenced by similar concerns about language and learning, Bakhtin [20] offers a distinct perspective on dialogue. Rather than focusing primarily on the internalisation of socially mediated knowledge, Bakhtin emphasises the role of multiple voices and perspectives in the creation of meaning. From a Bakhtinian viewpoint, understanding emerges through engagement with difference rather than through its elimination. Meaning is generated in the interaction between responses, perspectives, and interpretations, with dialogue remaining inherently open and unfinished [23]. Consequently, productive educational dialogue is not simply a process of leading learners toward predetermined conclusions but also involves creating opportunities for alternative viewpoints to be expressed, questioned, and explored [24,25]. This perspective highlights the value of challenge, uncertainty, and intellectual diversity within learning conversations.

Building on these theoretical foundations, contemporary researchers have translated dialogic learning theory into observable classroom practices. Alexander’s conception of dialogic teaching is particularly influential in educational research. He argues that productive classroom dialogue is collective, reciprocal, supportive, cumulative, and purposeful. Learners are encouraged to share ideas, respond thoughtfully to one another, build collectively on previous contributions, and pursue educational goals through sustained discussion [17,24]. Dialogic teaching therefore represents more than a classroom discussion technique; it is a principled pedagogical approach that treats classroom talk as a central mechanism for learning and knowledge construction. Research suggests that such forms of dialogue can improve student engagement, reasoning, and academic attainment across multiple subject areas [17,26].

Mercer’s work extends these ideas through the concept of thinking together. Drawing on sociocultural theory, Mercer argues that language functions as a cultural tool for collective reasoning and knowledge construction. Of particular importance is exploratory talk, a form of dialogue in which participants critically but constructively engage with one another’s ideas, provide reasons for claims, challenge assumptions, and work collaboratively toward shared understanding [21,27]. Research has demonstrated that learners who participate in such dialogues often show improvements in reasoning, understanding, and educational attainment, supporting the view that the quality of classroom talk influences the quality of learning [21].

The contributions of Vygotsky, Bakhtin, Alexander, and Mercer suggest that educational dialogue can be studied through observable discourse practices. Learning opportunities are reflected in how participants question, explain, justify, challenge, coordinate perspectives, and build collectively upon ideas. Productive dialogue is therefore characterised not merely by participation but by reasoning, responsiveness, and the joint construction of meaning. This insight has motivated the development of analytical frameworks designed to examine dialogue systematically and evaluate its contribution to learning. Among the most influential of these frameworks is the Cambridge Dialogue Analysis Scheme (CDAS), which provides a means of identifying discourse features associated with productive educational dialogue. The present study builds upon this tradition by adapting CDAS to examine the extent to which AI-mediated tutoring dialogues exhibit characteristics associated with Socratic learning.

2.3. Linking Dialogic Learning to Socratic Pedagogy

Although the CDAS was developed to analyse dialogic learning rather than Socratic learning specifically, there is substantial conceptual overlap between the two traditions. Both perspectives view learning as a process of inquiry supported through dialogue, questioning, reasoning, and the collaborative exploration of ideas. While Socratic pedagogy emerged from philosophical traditions concerned with disciplined questioning and the examination of assumptions, dialogic learning theory emerged from sociocultural perspectives that emphasize the role of language and interaction in knowledge construction. Despite these different origins, both traditions share a commitment to learning through intellectual engagement rather than passive reception of information.

One of the strongest points of convergence lies in the emphasis on questioning. Socratic pedagogy relies on carefully sequenced questions that encourage learners to explain, justify, and refine their thinking. Similarly, the dialogic literature identifies open questions, elaboration invitations, and reasoning invitations as productive forms of educational discourse because they stimulate extended thinking and shared inquiry [2]. The CDAS categories of Elaboration Invitations and Reasoning Invitations therefore capture discourse practices that closely resemble the questioning strategies used in Socratic teaching.

A second area of overlap concerns reasoning. Socratic questioning seeks not only answers but the justification behind them. Learners are encouraged to explain their claims, evaluate evidence, and make their reasoning visible. Likewise, dialogic theories identify explicit reasoning as a key feature of productive educational dialogue, particularly within Mercer’s concept of exploratory talk and Alexander’s model of dialogic teaching [17,21]. Correspondingly, CDAS includes categories that identify both invitations for reasoning and the actual expression of reasoning within dialogue. These categories provide observable indicators of intellectual processes that are central to Socratic inquiry.

Both traditions also emphasise the critical examination of ideas. A defining feature of Socratic teaching is the probing of assumptions and the challenge of claims that are insufficiently justified. Learning advances when participants encounter alternative perspectives, identify contradictions, and reconsider their positions. Dialogic researchers similarly argue that productive dialogue involves questioning, critique, and engagement with competing viewpoints rather than simple agreement [2,23]. This emphasis is reflected in CDAS categories such as Querying and Reasoned Coordination, which capture moments when participants challenge, compare, evaluate, and reconcile different ideas. These forms of discourse closely mirror the dialectical processes associated with Socratic inquiry.

Another shared characteristic is the cumulative development of understanding. In Socratic dialogue, learning unfolds through a sequence of questions and responses in which each contribution builds upon what has been said previously. Similarly, dialogic teaching emphasises cumulative talk, where ideas are connected, elaborated, and developed over time [17]. CDAS operationalises this process through categories such as Elaboration, Reference Back, and Coordination, which capture how participants build on prior contributions and connect ideas across a discussion. These discourse moves provide evidence that knowledge is being progressively constructed rather than merely exchanged.

Finally, both Socratic and dialogic perspectives encourage learners to connect ideas beyond the immediate discussion. Socratic teaching seeks transfer by encouraging learners to apply principles in new situations and to examine broader implications of their conclusions. Dialogic approaches similarly highlight the importance of linking classroom discussions to prior experiences, wider contexts, and other domains of knowledge. Accordingly, CDAS includes categories such as Reference Back and Reference to Wider Context, which capture these broader connections and support assessment of whether dialogue extends beyond local exchanges.

2.4. Large Language Models in Education

Recent advances in LLMs have generated considerable interest in their educational applications. Built on transformer architectures and trained on vast amounts of textual data, LLMs are capable of generating human-like responses, sustaining extended conversations, providing explanations, answering questions, and adapting their outputs to user needs. These capabilities have led researchers to explore their use across a wide range of educational activities, including student support, teacher assistance, adaptive learning, assessment, feedback generation, content creation, and recommendation systems [28,29]. Reviews of the literature consistently highlight the potential of LLMs to support personalised learning experiences while reducing the workload associated with routine educational tasks [30,31].

One of the most prominent educational applications of LLMs is their use as virtual tutors. LLM-based tutoring systems have been employed for automatic question generation, interactive practice activities, personalised feedback, and learner support in domains such as medicine and computer programming [32,33]. In medical education, for example, LLMs have been used to facilitate differential diagnosis brainstorming, generate interactive clinical cases, and support examination preparation [33]. Similarly, programming education research has shown that LLM-based tutors can provide timely feedback and scalable learner support, although students sometimes report concerns about overreliance on AI assistance and the tendency of some responses to remain overly generic [34].

Despite their promise, educational researchers have repeatedly cautioned against treating LLMs as fully autonomous tutors. Studies indicate that unrestricted conversational systems may drift away from intended pedagogical goals, provide excessive assistance, or reveal solutions too quickly, thereby reducing opportunities for productive learning [34,35]. Consequently, effective educational applications often combine the flexibility of LLMs with pedagogical structures, guardrails, and carefully designed instructional frameworks [35]. More broadly, concerns regarding accuracy, bias, privacy, plagiarism, equity, and overreliance continue to be identified as major challenges for the responsible implementation of LLMs in educational settings [28,29,36,37].

An important factor influencing the educational effectiveness of LLMs is prompt design. Prompt engineering has emerged as a critical practice for aligning model outputs with instructional goals and learning outcomes. Educational research increasingly views prompt engineering not only as a technical skill but also as a pedagogical strategy capable of shaping learner interaction with AI systems [38,39]. Effective prompts define roles, goals, constraints, audience, and response structures, thereby improving the relevance, quality, and consistency of generated outputs [40,41]. Studies further suggest that carefully designed prompting strategies can improve feedback quality, learner engagement, personalisation, critical thinking, and the alignment of AI-generated responses with educational objectives [39,42,43].

Despite growing interest in AI tutoring, relatively little research has examined whether LLM-generated dialogue exhibits discourse characteristics associated with Socratic pedagogy. Most studies focus on accuracy, feedback quality, or learner performance rather than the nature of educational dialogue itself.

3. Methodology

3.1. Simulation Design

This study employed a methodological simulation design to investigate whether interactions generated by LLMs exhibit discourse characteristics consistent with Socratic learning. Rather than evaluating student learning outcomes, the study focused on the analysis of educational dialogue itself. Specifically, AI-generated tutoring conversations were examined using an adapted version of the CDAS developed by Hennessy et al. [2].

The study proceeded in two stages. First, CDAS was adapted to create a SLAS grounded in principles of Socratic pedagogy. Second, a series of simulated tutoring dialogues were generated and analysed using the proposed framework. The objective was not to validate SLAS as a psychometric instrument but to explore its usefulness as an analytical framework for examining whether AI-mediated dialogue demonstrates characteristics associated with Socratic learning.

3.2. Development of the SLAS

The CDAS served as the foundation for the analytical framework used in this study. CDAS was originally developed to identify features of productive educational dialogue, including questioning, reasoning, coordination of ideas, and contextual connections. These features overlap substantially with the characteristics of Socratic teaching, which emphasizes disciplined questioning, justification of claims, examination of assumptions, and progressive refinement of understanding.

To adapt CDAS for the present context, categories judged to contribute minimally to inquiry-based learning were removed. In particular, the category Other Invitations was excluded because it largely captures procedural and closed forms of questioning. The remaining categories were re-organised into broader domains representing core dimensions of Socratic learning.

The resulting framework comprised five domains:

1.

Socratic Inquiry

• Elaboration Invitations
• Reasoning Invitations
• Coordination Invitations

2.

Knowledge Construction

• Elaboration
• Reasoning

3.

Dialectical Examination

• Querying
• Simple Coordination
• Reasoned Coordination

4.

Reflective Transfer

• Reference Back
• Reference to Wider Context

5.

Socratic Progression

• Conceptual Advancement
• Independent Reasoning
• Learner-Generated Inquiry

The fifth domain was introduced to capture developmental aspects of Socratic learning that were not explicitly represented within CDAS. Together, these domains formed the SLAS, which was subsequently used to analyse the generated dialogues.

3.3. Computational Environment and Dialogue Generation

3.3.1. Computational Environment

The tutoring dialogues were generated using the ellmer package in R, which provides a unified interface for interacting with multiple large language model providers through application programming interfaces (APIs). Paid API access was obtained for Anthropic Claude Sonnet 5 and OpenAI GPT-5.6 Terra.

Two independent chat sessions were initialized for each simulation. Claude Sonnet 5 was configured as the Socratic statistics instructor and GPT-5.6 Terra was configured as the novice undergraduate student. Fresh chat sessions were created at the beginning of each simulation to ensure that conversations were independent and that contextual information from previous simulations was not retained.

Simulations were conducted in R using the packages ellmer, glue, officer, and tidyverse. The officer package was used to automatically save complete dialogue transcripts as Microsoft Word documents for subsequent evaluation.

3.3.2. Generation of Tutoring Dialogues

The instructional scenario was adapted from Amin [1] example of introducing the chi-square test of independence through guided inquiry. All simulations employed the same contingency table describing the association between tattoo status and Hepatitis C infection.

For each simulation, Claude Sonnet 5 was assigned the role of a statistics professor employing the Socratic method, while GPT-5.6 Terra was assigned the role of a first-year undergraduate student with no formal knowledge of statistical inference.

The instructor model received a system prompt specifying a six-phase pedagogical sequence:

1.

Connect statistical thinking to everyday experiences.

2.

Introduce the Hepatitis C problem.

3.

Establish the need for data.

4.

Explore the contingency table.

5.

Compare observed and expected patterns.

6.

Introduce inferential thinking.

The instructor was required to ask one question at a time, encourage justification of responses, probe student reasoning, and minimise direct explanation. The student model was instructed to behave as a novice learner capable of basic reasoning but prone to uncertainty and occasional misconceptions.

Each simulation consisted of 25 teacher-student interaction cycles following an initial opening question from the instructor. Consequently, each dialogue contained approximately 51 conversational contributions.

Forty independent simulations were generated. At the beginning of each simulation, new teacher and student sessions were initialized to ensure state isolation between conversations. Because responses produced by large language models are probabilistic, the use of identical prompts across independent simulations generated naturally varying dialogue trajectories while maintaining a consistent instructional scenario.

To facilitate reproducibility, the principal system prompts used to configure the instructor and student agents are provided in Appendix C together with the rest of the code. The prompts were held constant across all simulations. An example of a generated dialogue is shown in Appendix B

3.4. Dialogue Evaluation and Scoring

Each dialogue was evaluated using the SLAS. The framework comprises five domains: Socratic Inquiry, Knowledge Construction, Dialectical Examination, Reflective Transfer, and Socratic Progression. Each indicator within a domain was scored using a four-point rubric:

• 0 = absent
• 1 = limited evidence
• 2 = moderate evidence
• 3 = strong evidence

Domain scores were obtained by summing scores across indicators. The maximum possible total score was 39. To enhance transparency and reproducibility, detailed scoring rubrics for all SLAS domains and indicators are provided in Appendix A.

To facilitate interpretation, an overall Socratic Learning Index (SLI) was calculated:

$$SLI={\displaystyle \frac{\mathrm{Total}\mathrm{Score}}{39}}\times 100$$

. Higher values indicate stronger evidence of Socratic learning characteristics within a dialogue.

The resulting scores were interpreted as follows:

• 0–20: Minimal Socratic Learning;
• 21–40: Emerging Socratic Learning;
• 41–60: Moderate Socratic Learning;
• 61–80: Strong Socratic Learning;
• 81–100: Highly Socratic Learning.

All dialogues were evaluated by a single expert evaluator with specialization in educational pedagogy and teaching methods. The evaluator independently reviewed each dialogue and assigned scores according to the SLAS rubric.

3.5. Data Analysis

For each of the fifty simulated tutoring dialogues, scores were recorded for the five SLAS domains (Socratic Inquiry, Knowledge Construction, Dialectical Examination, Reflective Transfer, and Socratic Progression) as well as for the overall Socratic Learning Index (SLI). The unit of analysis was the complete dialogue. Each dialogue was treated as an independent observation and analysed using the SLAS. Descriptive statistics were calculated to summarise the extent to which the dialogues exhibited characteristics associated with Socratic learning. These statistics included the mean, standard deviation, median, minimum score, maximum score, and range for each domain and for the overall SLI.

For each domain, the mean score was calculated as

$$\overline{x}={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i,$$

where $x_i$ denotes the domain score for the $i^{th}$ dialogue and $n=40$ represents the total number of simulated dialogues.

The standard deviation was calculated as

$$s=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^n{(x_i-\overline{x})}^2}{n-1}}},$$

to quantify variability in scores across simulations.

An overall SLI was computed for each dialogue using

$$SLI={\displaystyle \frac{\mathrm{Total}\mathrm{Score}}{39}}\times 100,$$

where 39 represents the maximum attainable SLAS score.

The SLI was subsequently summarised using the same descriptive statistics. For each of the forty simulated tutoring dialogues, scores were recorded for the five SLAS domains and the overall SLI. Descriptive statistics were calculated to summarize the extent to which the dialogues exhibited characteristics associated with Socratic learning. These statistics included the mean, standard deviation, median, minimum score, maximum score, and range for each domain and for the overall SLI. Because the study was exploratory and methodological in nature, the analysis focused on describing the distribution and consistency of SLAS scores across repeated simulations rather than on formal hypothesis testing.

4. Results

4.1. Overall Socratic Learning Performance

A total of 40 independently generated tutoring dialogues were evaluated using the SLAS. The scores were assigned by an expert evaluator with specialization in educational pedagogy and teaching methods.

Descriptive statistics for the overall SLAS score and the SLI are presented in Table 1. The mean total SLAS score across the 40 dialogues was 31.63 out of a maximum possible score of 39 (SD = 5.04), corresponding to a mean SLI of 81.09% (SD = 12.91). The median SLI was 84.62%, indicating that at least half of the generated dialogues achieved a level classified as Highly Socratic Learning. The maximum observed SLI was 97.44%, while the minimum was 35.90%.

Overall, these results indicate that the generated dialogues consistently exhibited discourse characteristics associated with Socratic learning. The average SLI exceeded the threshold for the Highly Socratic Learning category, suggesting that Socratic dialogue patterns emerged reliably across independent simulations.

4.2. Distribution of Socratic Learning Levels

Each dialogue was classified according to the interpretation framework proposed for the SLI. The distribution of dialogues across the five levels is shown in Table 2.

Most dialogues (65.0%) were classified as Highly Socratic Learning, while a further 25.0% were classified as Strong Socratic Learning.

Only four dialogues fell below the Strong category, and no dialogue was classified as exhibiting Minimal Socratic Learning. This distribution suggests that Socratic characteristics were present in the majority of interactions generated under identical prompting conditions.

4.3. Performance Across SLAS Domains

Descriptive statistics for the five SLAS domains are presented in Table 3.

The highest average scores were observed for the domains of Knowledge Construction ($\overline{x}=5.68/6$), Socratic Inquiry ($\overline{x}=7.90/9$), and Dialectical Examination ($\overline{x}=7.73/9$). These findings indicate that the tutoring dialogues consistently incorporated questioning strategies, elicited explanations and reasoning, and engaged learners in examining claims and interpretations.

The comparatively lower scores for Reflective Transfer and Socratic Progression suggest that while the dialogues were effective at sustaining inquiry and reasoning within the immediate task, they were somewhat less successful at encouraging broader contextual transfer, independent learner questioning, and progressive conceptual extension.

5. Discussion

The purpose of this study was to explore whether educational dialogues generated by large language models exhibit characteristics associated with Socratic learning. Using the proposed SLAS, forty independently generated tutoring dialogues were evaluated across five dimensions of Socratic learning: Socratic Inquiry, Knowledge Construction, Dialectical Examination, Reflective Transfer, and Socratic Progression. The findings suggest that large language models can generate dialogues that closely resemble many of the interactional features associated with Socratic pedagogy.

One of the most notable findings is the strong performance observed in the domains of Socratic Inquiry, Knowledge Construction, and Dialectical Examination. These domains represent core elements of both Socratic pedagogy and dialogic learning theory. High scores in Socratic Inquiry indicate that the tutoring dialogues consistently employed questions designed to stimulate elaboration, justification, and coordination of ideas rather than merely eliciting factual responses. This finding aligns with the central principle of Socratic teaching that learning is facilitated through disciplined questioning rather than direct transmission of knowledge.

Similarly, the strong performance in Knowledge Construction suggests that the dialogues frequently encouraged learners to explain concepts, articulate reasoning, and actively participate in the construction of understanding. These characteristics are broadly consistent with constructivist theories of learning, which emphasize the active role of learners in generating meaning through interaction and reflection.

The consistently high scores observed in the Dialectical Examination domain are also noteworthy. Socratic learning depends not only on questioning but also on the examination and refinement of ideas through challenge, critique, and comparison of alternative explanations. The generated dialogues frequently contained evidence of querying, coordination of perspectives, and reasoned evaluation of claims.

From the perspective of dialogic learning theory, these discourse features resemble Mercer’s notion of exploratory talk and Alexander’s conception of dialogic teaching, both of which emphasize reasoning, challenge, and the collaborative development of understanding. The results therefore suggest that large language models may be capable of facilitating forms of dialogue that extend beyond simple question-and-answer interactions and instead support deeper engagement with ideas.

Although the overall findings were positive, not all dimensions of Socratic learning were equally represented. Reflective Transfer and Socratic Progression obtained comparatively lower scores than the other domains. Reflective Transfer concerns the extent to which ideas developed during a dialogue are connected to wider contexts, prior experiences, or novel situations. Socratic Progression focuses on conceptual advancement, learner-generated inquiry, and the development of increasingly independent reasoning. The comparatively weaker performance in these domains suggests that while the models were successful in sustaining inquiry within the immediate instructional context, they were less effective at extending learning beyond the problem under discussion.

This finding may reflect an inherent tendency of current large language models to remain focused on locally relevant conversational objectives. Prompting strategies used in the simulations emphasized guided discovery of the chi-square test of independence and encouraged the instructor model to avoid excessive direct explanation. While these design choices successfully promoted questioning and reasoning, they may have constrained opportunities for broader transfer and learner-led exploration.

Future prompt designs could explicitly require students to formulate new questions, critique their own reasoning, or apply emerging concepts to unfamiliar situations. Such modifications may strengthen dimensions of Socratic learning that were less evident in the present study.

The findings also have implications for statistics education. Amin [1] argued that Socratic approaches to teaching statistics have considerable pedagogical value but are difficult to implement at scale because they require extensive instructor expertise, substantial preparation, and intensive learner interaction. The present findings suggest that large language models may offer a potential mechanism for addressing some of these scalability challenges.

Unlike traditional educational technologies that primarily deliver content or provide automated feedback, conversational AI systems are capable of sustaining extended dialogue, adapting responses, and engaging learners in questioning-based interactions. While such systems cannot replace experienced educators, they may provide opportunities for individualized inquiry-oriented learning experiences that would otherwise be difficult to achieve in large classes.

More broadly, the study contributes to the emerging literature on artificial intelligence in education by shifting attention from learning outcomes alone to the quality of educational dialogue itself. Much of the current research on educational applications of large language models focuses on answer accuracy, feedback quality, or student performance. However, from the perspective of dialogic and constructivist theories, the educational value of a tutoring interaction also depends on how learning conversations unfold. The SLAS Scheme proposed in this study provides a framework for examining these conversational processes and may offer researchers a tool for evaluating educational dialogue generated by both human and artificial tutors.

6. Conclusion

Amin [1] identifies a central challenge facing Socratic statistics instruction. Although Socratic teaching provides a powerful means of promoting inquiry, reasoning, and conceptual understanding, its implementation is difficult to scale because it requires highly skilled instructors, extensive preparation, and sustained interaction with learners. The present study explored whether large language models may offer a potential solution to this challenge.

To investigate this question, the CDAS was adapted into the SLAS, a framework designed to evaluate characteristics of Socratic learning within educational dialogue. The framework was subsequently applied to forty independently generated AI-mediated tutoring dialogues focused on the introduction of the chi-square test of independence.

Across the forty simulations, the generated dialogues exhibited consistently high levels of Socratic Inquiry, Knowledge Construction, and Dialectical Examination. The average SLI exceeded the threshold for highly Socratic learning, with nearly two-thirds of the dialogues classified within this category. Although Reflective Transfer and Socratic Progression were comparatively weaker, evidence of Socratic learning characteristics was observed across the majority of simulations.

Importantly, these characteristics emerged repeatedly despite the probabilistic nature of large language model outputs. The relatively narrow distribution of scores suggests that Socratic dialogue patterns can be generated consistently through appropriate prompt design and instructional structuring. From the perspective of scalability, this finding is particularly significant. If large language models can reliably reproduce many of the dialogic features associated with Socratic teaching, they may offer a practical mechanism for extending inquiry-based statistics instruction beyond contexts where intensive human facilitation is feasible.

The findings should be interpreted in light of several limitations. First, the study used simulated learners rather than real students, and therefore the dialogues may not fully capture the complexity of authentic classroom interactions. Second, the simulations focused on a single statistical topic, namely the chi-square test of independence. However, the prompts were designed to operationalize general principles of Socratic pedagogy rather than topic-specific instructional procedures. For this reason, it is plausible that similar patterns of dialogue would emerge in other areas of statistics when comparable prompting strategies are employed. Nevertheless, future studies should examine additional statistical topics to evaluate the extent to which the present findings generalize across the statistics curriculum Third, the study examined dialogue quality rather than learning outcomes. Consequently, the findings demonstrate the presence of Socratic characteristics within the dialogues but do not establish that students would necessarily achieve superior learning outcomes. Finally, the current evaluation relied on a single expert scorer, and further work is required to establish the reliability and validity of the proposed framework.

Future research should investigate whether the Socratic characteristics identified through SLAS translate into measurable improvements in student learning. Studies involving real learners, alternative statistical topics, different language models, and varied prompting strategies would provide a stronger basis for evaluating the educational potential of AI-mediated Socratic instruction. Further validation of SLAS is also needed to establish its usefulness as a general framework for analysing educational dialogue.

Overall, the findings provide preliminary evidence that large language models may help address one of the principal limitations identified by Amin [1]: the difficulty of scaling Socratic teaching. While AI cannot replace skilled educators, the results suggest that LLMs may be capable of delivering dialogue that preserves many of the inquiry-oriented features of Socratic pedagogy, thereby creating new opportunities for scalable, dialogue-based statistics education.