EduSkillBench: Measuring the Impact of Agent Skills on Single-Turn Educational Tasks

1. Introduction

1.1. Research Questions

EduSkillBench  is designed around three research questions:

RQ1, Overall Skill Effectiveness. Do educational Agent Skills improve LLM agent performance compared with the same agent without Skill access?

RQ2, Cross-model Generalization. Are Skill-induced improvements consistent across different LLM backbones?

RQ3, Skill-level Transferability. Which educational Skills provide reliable positive transfer, and when can Skills introduce limited or negative transfer, where negative transfer means a Skill that lowers observed reward relative to its matched No-Skill baseline?

Figure 1. Agent stack and single-turn results in EduSkillBench. Each horizontal bar stacks the No-Skill baseline and the observed lift from the matched Skill, where lift is the With-Skill mean rubric reward minus the No-Skill mean rubric reward. Both configurations use the OpenCode harness and the BenchFlow evaluation pipeline.

Figure 1. Agent stack and single-turn results in EduSkillBench. Each horizontal bar stacks the No-Skill baseline and the observed lift from the matched Skill, where lift is the With-Skill mean rubric reward minus the No-Skill mean rubric reward. Both configurations use the OpenCode harness and the BenchFlow evaluation pipeline.

LLM agents are increasingly used for education-facing work, including lesson planning, formative assessment, tutoring, differentiation, project design, retrieval practice, and learning-support workflows. A fundamental tension follows. Foundation models contain broad educational knowledge, but many high-quality teaching behaviors are procedural: a tutor must decide when to withhold an answer, how to escalate a hint, how to diagnose a misconception, or how to translate a learning objective into an aligned assessment. These procedures are difficult to capture with factual retrieval alone.

Agent Skills provide an inference-time mechanism for supplying reusable procedures. A Skill typically contains a SKILL.md file with instructions and may include templates, references, scripts, or worked examples. Unlike fine-tuning, Skill augmentation does not modify model weights. Unlike ordinary prompting, Skills are intended to be modular and reusable across task instances. EduSkillBench  operationalizes Skill access as adding the matched Skill package to the agent context, so its experiment measures the effect of adding a matched Skill relative to no added instructions. It does not, by itself, separate the Skill as a reusable package from the general effect of adding task-relevant instruction text; that separation is a required validation step (Section 6.1).

The benchmark problem is therefore different from ordinary education evaluation. EduBench asks whether a model performs well across diverse educational scenarios and introduces multidimensional educational criteria [2]. SkillsBench asks whether curated Agent Skills improve agents on matched tasks and establishes paired No-Skill versus With-Skill evaluation as a central design [1]. EduSkillBench  lies at the intersection: it asks how to construct and score education-specific tasks when the evaluated system is an LLM agent augmented with reusable Skills.

We make four contributions:

1.

Education-specific Skill benchmark. We curate 18 public educational Skills from two open repositories and map them to six of the nine EduBench-inspired scenarios. The curation is a data and engineering contribution, not a new evaluation method.

2.

Skill-aligned tasks and rubrics. We construct 54 benchmark tasks, each with explicit context, user request, expected output, and rubric, and we state explicitly that the tasks and rubrics are written around the behavior each Skill claims to support. The alignment is deliberate and disclosed, and its effect on the measured lift is a first-class quantity of the benchmark rather than a hidden confound.

3.

Reproducible cross-model single-turn evidence. We report 168 clean single-turn runs across two model configurations. Skills are associated with an average reward increase of +18.1 pp for qwen3.7-plus and +24.3 pp for deepseek-v4-flash, while per-Skill effects remain highly heterogeneous. All values are single-run point estimates reported descriptively.

4.

An explicit validation agenda. We specify the experiments required to move from descriptive observation to causal and uncertainty-bounded claims, including repeated runs, a common judge, and instruction-level controls (Section 6.1).

We adopt, rather than introduce, the paired No-Skill versus With-Skill design established by SkillsBench [1]. The goal of EduSkillBench  is not to claim that every Skill is beneficial. Rather, it provides a structured way to ask whether a Skill helps, where it helps, and when it can hurt.

2. Background

Skill definition. Following the emerging Agent Skills convention, we define a Skill as a reusable, file-based package of procedural guidance for a class of tasks. A Skill can specify sequencing rules, constraints, heuristics, templates, references, or execution aids. It is distinct from model weights and becomes actionable only through a compatible agent harness.

Skill augmentation. We use Skill augmentation to denote the inference-time condition in which a matched Skill is made available to the agent. For EduSkillBench, the experimental contrast is deliberately narrow: the same task is run with and without that Skill, and the matched Skill is present only in the With-Skill arm.

Educational evaluation. Open-ended educational outputs are rarely reducible to exact-match scoring. A lesson plan can be factually correct but poorly aligned with its objective; a hint can be correct but pedagogically premature. We therefore use task-specific rubrics with explicit expected behaviors, following the broader practice of rubric-based LLM evaluation [3].

Table 1 situates Skill augmentation among runtime augmentation paradigms. A check mark (✓) means the paradigm fully provides the capability, a partial mark (◐) means it provides it partially or indirectly, and a cross (×) means it does not provide it. The column ordering runs from Prompting, to RAG, to Tools, to Skills. Skills are the only paradigm that simultaneously packages reusable procedural guidance and remains portable across model configurations, which is the property EduSkillBench  is designed to measure.

3. EduSkillBench

A benchmark for Skill-augmented education is only useful if the Skills, tasks, and rubrics are separable enough to avoid reducing evaluation to prompt imitation. EduSkillBench  therefore treats benchmark construction as part of the contribution.

3.1. Design Principles

Public Skills, not benchmark-authored interventions. All 18 retained Skills originate from existing public repositories. Missing scenario coverage is left missing rather than filled with newly authored benchmark-specific Skills.

Scenario-grounded tasks. Tasks instantiate recognizable teacher- or learner-facing educational work, using the EduBench scenario taxonomy as an organizing scaffold.

Observable procedures. A retained Skill must encode behavior that can be observed in an output or interaction and judged through explicit criteria.

Paired conditions. No-Skill and With-Skill runs use the same task and model configuration. The only intended experimental difference is access to the matched Skill.

3.2. Construction Pipeline

Figure 2. EduSkillBench pipeline. The benchmark starts from public educational Skills, constructs Skill-aligned tasks and rubrics, and evaluates matched conditions under a fixed agent harness.

Figure 2. EduSkillBench pipeline. The benchmark starts from public educational Skills, constructs Skill-aligned tasks and rubrics, and evaluates matched conditions under a fixed agent harness.

3.3. Skill Sources and Filtering

We source candidate Skills from the Education Agent Skills Library and the Teaching Skills repository [7,8]. At collection time they contain approximately 165 and 15 Skills, respectively. We retain a Skill when it (i) encodes a recognizable educational procedure, (ii) can be instantiated as a self-contained task, (iii) admits explicit rubric-based evaluation, and (iv) maps meaningfully to an education scenario.

The final pool contains 18 Skills: 14 single-turn and 4 multi-turn. They cover six EduBench-inspired scenarios. We retain no suitable public Skill for Idea Provision, Emotional Support, or Automatic Grading; these gaps are preserved rather than filled artificially.

3.4. Task Specification

Each retained Skill is paired with three cases, intentionally varying subject, educational level, and difficulty. The released task schema includes:

task_id, skill_id, scenario, subject, education_level, difficulty, context, user_prompt, expected_output, rubric.

For the 14 single-turn Skills, each Skill contributes one easy, one medium, and one hard case, yielding 14 tasks at each difficulty level. The multi-turn subset targets interaction-dependent procedures such as retrieval-first gating, error diagnosis, progressive hinting, and confidence calibration. Its 12 task specifications are released, but a dedicated stateful evaluation protocol is not yet implemented, so the reported experiment covers the single-turn subset only.

Table 2. Summary of EduSkillBench  v1.

Statistic	Count
Public Skill source repositories	2
Candidate Skills	180
Selected Skills	18
Single-turn Skills	14
Multi-turn Skills	4
Single-turn tasks	42
Multi-turn tasks	12
Total tasks	54
EduBench-inspired scenarios covered	6 / 9
Single-turn tasks evaluated	42

Table 2. Summary of EduSkillBench  v1.

Statistic	Count
Public Skill source repositories	2
Candidate Skills	180
Selected Skills	18
Single-turn Skills	14
Multi-turn Skills	4
Single-turn tasks	42
Multi-turn tasks	12
Total tasks	54
EduBench-inspired scenarios covered	6 / 9
Single-turn tasks evaluated	42

3.5. Rubric System

The central evaluation artifact in EduSkillBench  is not only the prompt, but the task-specific rubric. Single-turn cases operationalize the target educational procedure as five explicit expected behaviors. Typical criteria include task fulfillment, pedagogical appropriateness, adherence to the intended procedure, actionability, diagnostic quality, alignment with learning objectives, or accessibility constraints. The final reward is normalized to the $[0,1]$ interval.

The rubrics are intentionally sample-specific rather than a single universal checklist. A hinge-question task must reward diagnostic distractors and actionable teaching decisions, while a lesson-building task must reward objective alignment, timings, materials, facilitation, and contingency planning. Multi-turn rubrics additionally require state tracking and response adaptation.

Conceptually, these criteria fall into four evaluation families:

• Task fulfillment: whether the requested artifact or response is complete and correct;
• Pedagogical quality: whether the output is instructionally appropriate for the learner and objective;
• Skill execution quality: whether the procedure targeted by the matched Skill is actually expressed in the output;
• Interaction quality: for multi-turn tasks, whether the agent adapts across learner responses rather than applying a static script.

The current single-turn experiment scores the first three through the released expected-behavior rubrics; multi-turn interaction quality is released as task specification but not yet included in the reported experiment.

Because the single-turn reward includes the Skill execution family, the measured lift is a Skill-aligned utility: it can reward output that expresses the matched procedure even when independent pedagogical quality is unchanged. This is the intended scope of v1, and it means the headline lift can overstate a Skill’s contribution relative to a procedure-free rubric. Quantifying the contribution of the Skill execution family, and validating the rubrics against independent human judgment, are required validation steps (Section 6.1).

3.6. Anatomy of an EduSkillBench  Task

Figure 3. Anatomy of a EduSkillBench single-turn evaluation. The task and scoring rubric are fixed; the two arms differ only in access to the matched Skill.

Figure 3. Anatomy of a EduSkillBench single-turn evaluation. The task and scoring rubric are fixed; the two arms differ only in access to the matched Skill.

3.7. Educational Skill Taxonomy

To analyze the coverage of educational procedures, we categorize collected Skills into six functional groups: instruction design (e.g., lesson planning), learning strategy (e.g., retrieval practice), assessment support (e.g., hinge questions), student support (e.g., self-efficacy and motivation), questioning and reasoning (e.g., Socratic questioning), and project learning (e.g., project scaffolding).

This taxonomy is not intended as a new educational ontology. Instead, it provides a structured view of the procedural knowledge represented in EduSkillBench  and enables analysis of whether Skill effects differ across educational functions.

3.8. Benchmark Composition

The six covered EduBench-inspired scenarios are distributed unevenly because coverage follows the available public Skill ecosystem rather than a quota. The six scenarios are Question Generation (writing questions and diagnostic items), Personalized Content Creation (adapting content to a learner), Teaching Material Generation (building lessons, plans, and teaching resources), Q&A (answer-first gated dialogue), Error Correction (diagnosing and correcting learner mistakes), and Personalized Learning Support (adaptive and calibrated tutoring sequences). The evaluated single-turn subset covers the first three: Question Generation, Personalized Content Creation, and Teaching Material Generation. The released multi-turn subset adds Q&A, Error Correction, and Personalized Learning Support.

Figure 4. Task composition across the six covered EduBench-inspired scenarios. Counts include both single-turn and multi-turn task specifications. Teaching Material Generation contributes 24 tasks, Question Generation 9, Personalized Content Creation 9, Personalized Learning Support 6, Q&A 3, and Error Correction 3, for 54 total.

Figure 4. Task composition across the six covered EduBench-inspired scenarios. Counts include both single-turn and multi-turn task specifications. Teaching Material Generation contributes 24 tasks, Question Generation 9, Personalized Content Creation 9, Personalized Learning Support 6, Q&A 3, and Error Correction 3, for 54 total.

4. Experiment

4.1. Models, Harness, and Conditions

We evaluate two model configurations: qwen3.7-plus (Qwen, a general-purpose LLM family) and deepseek-v4-flash (DeepSeek, another general-purpose LLM family). Both use OpenCode, an open agent harness, as the agent runtime; BenchFlow, an orchestration framework, to create task environments and Docker sandboxes for execution; and Docker for sandboxing. Each of the 42 single-turn tasks is evaluated under two matched conditions: No-Skill and With-Skill. This produces $42\times 2=84$ runs per model and 168 clean runs in the final aggregate.

The judge model follows the model configuration used for each run: Qwen results use the Qwen judge identifier, while DeepSeek results use the DeepSeek judge identifier. This design makes each model’s paired lift internally interpretable, but it also means the model variable and the judge variable are confounded across rows. We therefore treat cross-model comparisons as descriptive and do not claim a causal reading of cross-model differences; a shared independent judge is a required validation step (Section 6.1).

The No-Skill arm adds no matched instruction text. The design therefore measures the effect of making a matched Skill available relative to no added instructions. It does not separate the Skill as a reusable package from the effect of adding task-relevant instruction text of comparable length. A matched-length, non-matched instruction control is a required validation step (Section 6.1).

4.2. Evaluation Protocol

BenchFlow creates the task environment, passes the prompt to the OpenCode agent, and makes the matched Skill available only in the With-Skill arm. The evaluator scores the final response against the case-specific expected behaviors. For DeepSeek, four initial With-Skill runs hit a 300-second wall-clock timeout; those four cells were rerun with a 600-second timeout, and the successful replacements are used in the final 84/84 clean aggregate. Because only the With-Skill cells received the extension, those four lifts could be biased in the With-Skill direction; a sensitivity check under a uniform timeout is a required validation step (Section 6.1).

4.3. Metrics

Our primary metric is mean normalized rubric reward. For task x, model M, agent harness A, and matched Skill s, the No-Skill reward is

$$R^{-}\left(x\right)=R\left(A(M,x)\right),$$

(1)

where R is the rubric reward normalized to $[0,1]$, and $A(M,x)$ denotes the final output produced by running harness A with model M on task x. The With-Skill reward, in which the matched Skill s is also made available to the agent, is

$$R^{+}(x,s)=R\left(A(M,x,s)\right).$$

(2)

The task-level Skill lift is

$$\Delta (x,s)=R^{+}(x,s)-R^{-}\left(x\right).$$

(3)

For each Skill, we average $\Delta$ over its three cases. Because each evaluated Skill contributes exactly three tasks, macro-averaging over Skills and equal-weight averaging over tasks are equivalent up to rounding.

To mirror the headroom-aware analysis used in SkillsBench [1], we also report normalized gain [4]:

$$g={\displaystyle \frac{{\overline{R}}^{+}-{\overline{R}}^{-}}{1-{\overline{R}}^{-}}},$$

(4)

where ${\overline{R}}^{+}$ and ${\overline{R}}^{-}$ are the Skill-level macro-averaged With-Skill and No-Skill rewards, and g measures the fraction of remaining score headroom closed by Skill augmentation. We treat g as descriptive, because each model uses its own judge configuration.

All reported values are single-run point estimates. Following the v1 release scope, we report them descriptively and do not attach confidence intervals or significance tests to them. Bounding these estimates with repeated runs is a required validation step (Section 6.1).

5. Results

We first report overall efficacy across the two model configurations, then analyze scenario-level and Skill-level heterogeneity. Every number below is a single-run point estimate; we use the word “observed” where an interpretation would otherwise sound causal.

5.1. Overall Skill Effectiveness and Cross-model Transfer

5.1.1. Main Results

Table 3. Mean normalized rubric reward (0 to 1 scale), absolute Skill lift in percentage points (pp), normalized gain g in percent, and per-Skill outcome counts on the 42-task single-turn subset. Each model has 84 clean runs: 42 No-Skill and 42 With-Skill. All values are single-run point estimates.

Model	No-Skill	With-Skill	Δ (pp)	g (%)	Pos./Tie/Neg. Skills
Qwen3.7-Plus	0.767	0.948	+18.1	77.7	10 / 3 / 1
DeepSeek-V4-Flash	0.314	0.557	+24.3	35.4	9 / 4 / 1

Table 3. Mean normalized rubric reward (0 to 1 scale), absolute Skill lift in percentage points (pp), normalized gain g in percent, and per-Skill outcome counts on the 42-task single-turn subset. Each model has 84 clean runs: 42 No-Skill and 42 With-Skill. All values are single-run point estimates.

Model	No-Skill	With-Skill	Δ (pp)	g (%)	Pos./Tie/Neg. Skills
Qwen3.7-Plus	0.767	0.948	+18.1	77.7	10 / 3 / 1
DeepSeek-V4-Flash	0.314	0.557	+24.3	35.4	9 / 4 / 1

Finding 1 (observed): Skills provide broad but non-uniform gains. Both model configurations improve substantially in observed mean reward, but the gains differ in magnitude and distribution. Qwen reaches the stronger absolute With-Skill score (0.948), while DeepSeek shows the larger absolute lift (+0.243). Thus, high final performance and high marginal benefit from Skills are not the same property.

Finding 2 (observed): Skill benefit depends on the model. The same Skill can behave very differently across models. lesson-builder rises from 0.200 to 0.933 for Qwen ($+0.733$) but falls from 0.267 to 0.000 for DeepSeek ($-0.267$). Conversely, retrieval-practice-generator improves by $+0.267$ for Qwen and by $+0.867$ for DeepSeek. These observed reversals are consistent with treating Skill quality as model-dependent, but they rest on single-run cells scored by model-specific judges, so they are descriptive until the common-judge and repeated-run validations in Section 6.1 are complete.

Finding 3 (observed): Ceiling effects can hide useful Skills. Three Qwen Skills tie at 1.000 in both conditions. When the baseline already saturates the rubric, the benchmark cannot measure additional benefit even if the Skill improves reasoning or consistency. This motivates harder cases and trajectory-level diagnostics in future releases.

Figure 5. Per-Skill lift for Qwen3.7-Plus and DeepSeek-V4-Flash. Positive values favor the matched Skill condition; negative values indicate observed negative transfer. See Table A1 for per-Skill values.

Figure 5. Per-Skill lift for Qwen3.7-Plus and DeepSeek-V4-Flash. Positive values favor the matched Skill condition; negative values indicate observed negative transfer. See Table A1 for per-Skill values.

5.2. Scenario-Level Analysis

Table 4. Single-turn efficacy by EduBench-inspired scenario. n is the number of tasks. Values are mean normalized rubric rewards (0 to 1 scale), single-run point estimates.

		Qwen3.7-Plus				DeepSeek-V4-Flash
Scenario	n	No	With	Δ		No	With	Δ
Question Generation	9	0.755	0.867	+0.111		0.156	0.556	+0.400
Personalized Content Creation	9	0.919	1.000	+0.081		0.489	0.667	+0.178
Teaching Material Generation	24	0.715	0.958	+0.244		0.308	0.517	+0.208

Table 4. Single-turn efficacy by EduBench-inspired scenario. n is the number of tasks. Values are mean normalized rubric rewards (0 to 1 scale), single-run point estimates.

		Qwen3.7-Plus				DeepSeek-V4-Flash
Scenario	n	No	With	Δ		No	With	Δ
Question Generation	9	0.755	0.867	+0.111		0.156	0.556	+0.400
Personalized Content Creation	9	0.919	1.000	+0.081		0.489	0.667	+0.178
Teaching Material Generation	24	0.715	0.958	+0.244		0.308	0.517	+0.208

Finding 4 (observed): The strongest scenario depends on the model. Teaching Material Generation produces the largest Qwen lift (+0.244), whereas Question Generation produces the largest DeepSeek lift (+0.400). This suggests that the magnitude of the observed lift differs across scenarios and models; attributing the difference to model-specific procedural deficits would require the mechanism analysis described in Section 6.1.

Figure 6. Scenario-level Skill lift on the evaluated single-turn subset. All three scenarios show positive aggregate lift for both models, but the relative ordering differs.

Figure 6. Scenario-level Skill lift on the evaluated single-turn subset. All three scenarios show positive aggregate lift for both models, but the relative ordering differs.

5.3. Task- and Skill-Level Failure Modes

By negative transfer we mean a Skill whose observed With-Skill reward is lower than its matched No-Skill baseline. This usage follows the Agent-Skills literature [1] and is distinct from the learning-science sense of prior learning interfering with new learning.

The strongest Qwen beneficiary is lesson-builder (+0.733), followed by project-brief-designer (+0.600) and socratic-questioning-sequence-generator (+0.467). The only Qwen negative case is hinge-question-designer ($-0.400$), whose No-Skill baseline already scores 1.000.

For DeepSeek, the strongest beneficiary is retrieval-practice-generator (+0.867), followed by ruler-emotional-literacy-sequence (+0.667) and adaptive-hint-sequence-designer (+0.600). The only negative case is lesson-builder ($-0.267$).

These results expose two recurring benchmark phenomena. First, a Skill may over-constrain or displace a strategy that already works well for a model. Second, Skill instructions that are helpful to one model may be too complex, too rigid, or poorly internalized by another. Distinguishing content quality from adherence requires trajectory-level analysis, which is not yet part of the released v1 score, and distinguishing these explanations from single-run noise requires the repeated-run validation in Section 6.1.

6. Discussion

The largest observed gains occur on tasks that require structured educational procedures. Retrieval practice design, lesson planning, adaptive hinting, project briefing, and Socratic questioning are all tasks with explicit procedural structure. This pattern is consistent with the intended role of Skills as reusable procedures rather than factual context. The current experiment does not, however, directly separate procedural benefit from instruction-following or added context. That separation requires the instruction-level control and adherence diagnostics described in Section 6.1, so we state the pattern as consistent with, not evidence for, the mechanism.

Skill evaluation should report more than an average. The observed cross-model reversals show why a single benchmark mean is insufficient. A useful Skill benchmark should report per-Skill lift, negative transfer, ceiling effects, and scenario variation. In future versions, we also plan to report explicit Skill adherence and interaction-quality diagnostics.

Rubric design interacts with the measured lift. Because tasks and rubrics are aligned to the matched Skill, the measured lift is bounded below by the benefit of expressing the Skill’s own procedure. This makes EduSkillBench’s headline numbers interpretable as Skill-aligned utility, and it makes the interaction between rubric design and observed lift a first-class measurement target. Reporting the lift without the Skill execution family, and validating rubrics against independent human judgment, will tell us how much of the observed benefit is pedagogical rather than structural.

Task-specific rubrics are essential in education. Unlike terminal tasks with deterministic test suites, educational artifacts often admit multiple valid outputs. A rubric must therefore encode pedagogical requirements, not only correctness. EduSkillBench  treats those rubrics as part of the benchmark specification rather than as post-hoc evaluation prompts.

Public-only Skill sourcing improves ecological validity. Because the benchmark does not invent Skills to fill missing scenarios, coverage reflects the public Skill ecosystem rather than an idealized taxonomy. The trade-off is uneven coverage: Teaching Material Generation is overrepresented, while three EduBench-inspired scenarios remain uncovered.

Cross-model Skill behavior is itself an evaluation target. Qwen and DeepSeek both benefit overall, yet the per-Skill patterns are not interchangeable. Because each model row was scored by a same-family judge, we treat the cross-model pattern as descriptive pending common-judge validation. If it survives that validation, future Skill benchmarks should treat the model–Skill pair, not the Skill alone, as the unit of practical deployment evaluation.

6.1. Limitations and Required Validation Experiments

We separate the limitations of the current release from the experiments required to move the claims beyond descriptive observation. The limitations are known and disclosed; the validation experiments are the specific actions that would close them.

Known limitations of the v1 release. 

• Limited scale. The current inventory contains 54 tasks and only three cases per Skill. The observed deltas are therefore sensitive to individual task instances.
• Single-run reporting. Each task-condition cell has one final clean result. We report point estimates and do not claim precision, significance, or confidence intervals for any headline value.
• Judge dependence. The current model rows use model-specific LLM judges. Paired lifts within a row are informative, but absolute scores across rows can be affected by judge calibration, and LLM-as-a-judge position, verbosity, and self-enhancement biases [3] are directly relevant because With-Skill outputs are likely longer and more structured.
• Skill-aligned task construction. Tasks are deliberately written around the behavior each Skill claims to support, and the single-turn reward includes a Skill execution family. This measures Skill-aligned utility, not average value over arbitrary educational work, and it may reward structure that resembles the Skill.
• Skill versus added instructions. The No-Skill arm adds no instruction text, so the experiment does not separate the Skill as a reusable package from the effect of adding task-relevant instructions of comparable length.
• Mechanism not directly measured. The current score cannot separate content quality from adherence to the Skill procedure; the manuscript states this itself in Section 5.3.
• Multi-turn evaluation is not yet reported. Twelve multi-turn task specifications are released, but a dedicated stateful learner–agent protocol is still required before they can be scored comparably.
• Incomplete scenario coverage. Six of nine EduBench-inspired scenarios are represented. We intentionally avoid inventing new Skills solely to improve coverage.
• Source drift and model aliases. Public Skill repositories and API-served model aliases can change. Reproducible releases should pin upstream commits and model snapshots whenever possible.

Required validation experiments. Each item below is a concrete, runnable specification. None of the observed numbers in this paper depend on these experiments; they are the work required to upgrade the claims from descriptive to causal and uncertainty-bounded.

• E1. Repeated runs and uncertainty bounds. Run each of the 42 single-turn task-condition cells at least three times under controlled seeds. Report per-cell and per-Skill means with standard deviations and a bootstrap confidence interval over the three cases per Skill, and a paired test (for example a Wilcoxon signed-rank test over the 42 No-Skill versus With-Skill pairs per model) for the aggregate lift. Success criterion: the headline +18.1 pp and +24.3 pp lifts, the 10-of-14 and 9-of-14 classifications, and the one-negative-per-model pattern survive with quantified uncertainty.
• E2. Common independent judge and human validation. Score a shared stratified sample of both models’ No-Skill and With-Skill outputs with a single third-party judge and with human raters. Report inter-judge agreement (for example Cohen’s kappa or intraclass correlation) and a judge-bias analysis. Success criterion: cross-model lift patterns and reversals replicate under the common judge, or the judge-confound explanation is quantified and controlled.
• E3. Matched-length instruction control. Add two control arms over the same 42 tasks: matched-length generic instructions with no Skill, and a non-matched public Skill of comparable length. Compare the With-Skill lift against both. Success criterion: the With-Skill lift exceeds the instruction-only baseline by a margin beyond the E1 intervals; otherwise the claim narrows to the effect of adding Skill-sourced instructions.
• E4. Rubric de-coupling. Re-score outputs with the Skill execution family removed or blinded, and validate rubrics against independent human judgments of pedagogical quality on a sample (for example three tasks per Skill). Optionally add tasks in the same scenarios that are not aligned to any Skill. Success criterion: a material lift persists without the Skill execution family, and human-judged pedagogical quality moves with the With-Skill advantage.
• E5. Trajectory-level adherence analysis. Score agent traces for adherence to the Skill procedure separately from output quality, and report the adherence by outcome interaction. This turns the Section 5.3 caveat into a measurement.
• E6. Timeout sensitivity. Re-run the four affected DeepSeek With-Skill cells and their No-Skill counterparts under a uniform 600-second timeout, and report both conditions.

Beyond these, future versions should expand the task inventory, run the multi-turn subset, and test retrieval from larger noisy Skill libraries rather than assuming a perfectly matched Skill.

7. Related Work

Education-facing benchmarks. EduBench defines nine educational scenarios and multidimensional criteria for education-oriented LLM evaluation [2]. LearnLM studies pedagogical instruction following in education-focused model development [5], while Tutor CoPilot shows that structured AI guidance can change tutoring behavior in real instructional settings [6]. EduSkillBench  differs by evaluating reusable public Agent Skills as the intervention.

Agent Skills and augmentation evaluation. SkillsBench establishes broad paired evaluation for curated Skills and reports substantial but heterogeneous gains across model–harness configurations [1]. EduSkillBench  adopts the paired-condition principle but specializes task and rubric construction to educational procedures and open-ended outputs, and it reports the Skill-aligned nature of that construction as an explicit design feature.

Rubric-based LLM evaluation. LLM-as-a-judge methods provide a scalable approach for evaluating open-ended model outputs but can exhibit position, verbosity, and self-enhancement biases [3]. These limitations motivate our explicit expected-behavior rubrics and the planned addition of a common judge and independent human validation (Section 6.1, E2).

8. Conclusions

We introduced EduSkillBench, a benchmark for evaluating reusable public Agent Skills on educational tasks. The v1 release contains 18 Skills and 54 tasks, with explicit task-specific rubrics and six EduBench-inspired scenarios. On the 42-task single-turn subset, matched Skills are associated with an observed increase in mean rubric reward from 0.767 to 0.948 for qwen3.7-plus and from 0.314 to 0.557 for deepseek-v4-flash, reported as single-run point estimates. The gains are substantial but model-specific, with one negative-transfer Skill in each configuration and several ceiling-limited cases.

The main lesson is conditional: educational Skills can provide meaningful support in the tested conditions, but their value cannot be inferred from the Skill text alone. It must be measured against a specific model, task, rubric, and harness, and it must be bounded by repeated runs and a common judge before the cross-model conclusions can be read causally. By releasing Skills, task specifications, rubrics, evaluation scripts, and results, EduSkillBench  provides a foundation for broader evaluation of Skill-augmented educational agents. The multi-turn subset is released as specifications and is not yet evaluated.