CoT-Core: Accelerating LLM Evaluation via CoT-Aware Coreset Selection

1. Introduction

The rapid advancement of Large Language Models (LLMs) [2,20] has necessitated the development of comprehensive evaluation frameworks and benchmarks, such as MMLU [8], GSM8K [6], and HELM [10]. However, evaluating modern LLMs across tens of thousands of test cases introduces prohibitive computational and financial overhead [17]. Furthermore, evaluation is rarely a one-time event; during the continuous pre-training, alignment, and fine-tuning phases, frequent and repeated evaluations across numerous model checkpoints are strictly required to monitor progress and steer model development [4,7], severely exacerbating the consumption of computational resources.

Recent literature highlights that this exhaustive evaluation paradigm is highly inefficient due to inherent dataset redundancy [21,26]. This creates an urgent need for coresets—minimal subsets of data that accurately reconstruct full-dataset evaluation scores. While various sampling heuristics exist, the most prominent and systematic approaches for constructing evaluation coresets generally align with two main paradigms. Psychometric methods, which score questions based on historical model correctness using Item Response Theory (IRT) [12], are fundamentally bottlenecked by their heavy reliance on massive, pre-existing evaluation logs. Consequently, they face a severe “cold start” problem: for newly constructed benchmarks or private evaluation datasets, the extensive historical response data required to calibrate item parameters is simply unavailable. This strict prerequisite renders history-dependent methods completely inapplicable to emerging evaluation scenarios where ground-truth correctness records across a wide range of models have not yet been accumulated. On the other hand, geometric methods, such as the widely adopted k-center greedy algorithm [14], typically operate on shallow textual embeddings of the questions. While computationally efficient, these dense encoders exhibit a strong bias toward surface lexical overlap [16,19], fundamentally failing to capture the latent reasoning structures underlying the tasks.

Figure 1. The motivation and overview of CoT-Core.

Figure 1. The motivation and overview of CoT-Core.

To overcome the limitations of history-dependent metrics and shallow lexical features, we propose CoT-Core, a novel training-free core question selection framework. Our approach is motivated by a critical observation: lexically disparate questions—which appear entirely unrelated in standard text embedding spaces—can share equivalent underlying logic or reasoning structures. To capture this underlying reasoning manifold, CoT-Core first prompts LLMs to unroll explicit Chain-of-Thought (CoT) reasoning trajectories. By directly projecting these unrolled reasoning paths into a dense embedding space, CoT-Core effectively overcomes lexical divergence, achieving highly precise and cognitive-aligned question clustering. Building upon this logically structured latent space, we apply standard clustering algorithms to extract a minimal, highly representative core subset. This selected coreset serves as a high-fidelity proxy for the full benchmark, drastically mitigating the computational and financial overhead of continuous model evaluation while maintaining accurate performance estimation.

Our main contributions are summarized as follows: 1) We propose CoT-Core, a novel training-free coreset selection framework that circumvents surface-level lexical traps. By projecting unrolled reasoning trajectories into a latent space, it clusters questions based on their underlying reasoning manifold rather than superficial text features. 2) Methodologically, we decouple spatial coreset selection from downstream proxy estimation. CoT-Core employs centroid-based geometric sampling to extract representative instances, naturally supporting Standard Aggregation (SA) for a strictly training-free pipeline, while remaining seamlessly compatible with advanced probabilistic estimators (e.g., GP-IRT). 3) Extensive experiments demonstrate that CoT-Core drastically reduces evaluation overhead while maintaining high-fidelity score estimation. Furthermore, we delineate the boundary conditions of reasoning-aware pruning, revealing that its efficacy is intrinsically tied to task complexity.

2. Related Work

LLM Benchmark Evaluation and Redundancy 

As the demand for rigorous assessment grows, evaluating models across massive suites has become computationally bottlenecked. Studies such as EssenceBench [21] have systematically quantified sample redundancy within prominent evaluation suites, revealing significant overlap at both the textual level and the ranking level. Similarly, Zhang et al. [26] demonstrated that repeated testing of similar capabilities yields diminishing returns. Identifying a representative coreset is therefore an urgent necessity for sustainable AI development.

Geometric and Training-free Methods. 

As a foundational approach to eliminate redundancy without historical reliance, geometric algorithms like the k-center greedy method [14] prioritize maximizing spatial manifold coverage. However, these methods traditionally construct feature spaces using shallow embeddings of raw problem text. Consequently, they are susceptible to surface-level lexical traps: while two problems might share identical vocabulary, they often necessitate completely different reasoning trajectories, leading to suboptimal pruning that misses structural diversity.

Performance-Based and Psychometric Methods. 

Alternatively, another research branch attempts to capture latent item difficulty through historical model performance. Classic paradigms filter instances based on training dynamics [18], while recent advancements like TinyBenchmarks [12] and ATLAS [9] leverage Item Response Theory (IRT) to estimate parameters from large-scale response matrices. While effective at capturing cognitive depth, these history-dependent methods suffer from a severe “cold start” problem. Their reliance on extensive historical records renders them inapplicable to emerging or newly constructed evaluation suites where such data is entirely absent.

CoT-Aware Representations for Coreset Selection 

Chain-of-Thought (CoT) prompting [23] transforms a black-box mapping into a transparent sequence of cognitive operations. While previous works (e.g., Auto-CoT [25]) combine embeddings and clustering, they predominantly operate on the embeddings of the raw questions, conflating superficial lexical overlap with true structural similarity. Recent studies demonstrate that LLM reasoning follows globally coherent abstract paths, which are far better captured by embedding the reasoning process itself [24].

In this work, we propose a paradigm shift by utilizing CoT trajectories and embeddings as a unified feature space. Instead of surface text, CoT-Core extracts the latent logical structure of the reasoning path, abstracting away specific numerical values and concrete entities. By clustering these isomorphic reasoning structures, we group questions necessitating identical cognitive meta-operations. Crucially, because this representation intrinsically captures cognitive difficulty and latent logic, our clustering approach remains entirely training-free.

3. Methodology

In this section, we present the technical details of CoT-Core. We first formally define the evaluation coreset problem, and then detail the three primary stages of our pipeline: CoT Trajectory Generation, Contextualized Trajectory Embedding, and Isomorphic Clustering and Selection.

3.1. Problem Formulation

Let $\mathcal{D}={\left\{(q_i,a_i)\right\}}_{i=1}^N$ denote a comprehensive LLM evaluation benchmark comprising N test instances, where $q_i$ is the question prompt and $a_i$ is the reference answer. For a given large language model $\mathcal{M}$, the standard evaluation process computes its true overall performance by averaging the exact match scores across the entire dataset:

$$S_{\mathcal{D}}\left(\mathcal{M}\right)={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}m(\mathcal{M}\left(q_i\right),a_i)$$

(1)

where $\mathcal{M}\left(q_i\right)$ represents the model’s generated output and $m(·,·)$ is a specific evaluation metric function (e.g., exact match accuracy).

The overarching objective of coreset evaluation intrinsically consists of two orthogonal stages: Subset Selection and Performance Estimation.

First, the selection stage identifies a minimal representative subset $\mathcal{S}\subset \mathcal{D}$, strictly bounded by a budget constraint $\left|\mathcal{S}\right|=B\ll N$. Second, the estimation stage employs a proxy scoring function, denoted as $f_{est}$, to predict the model’s full-dataset capability based solely on its responses to the constrained subset $\mathcal{S}$.

The ultimate goal is to formulate a subset $\mathcal{S}$ and an estimator $f_{est}$ such that the predicted proxy score closely approximates the true full-dataset performance for any unseen model $\mathcal{M}$:

$${\widehat{S}}_{\mathcal{D}}\left(\mathcal{M}\right)=f_{est}(\mathcal{M},\mathcal{S})\approx S_{\mathcal{D}}\left(\mathcal{M}\right)$$

(2)

Unlike history-dependent baselines that tightly couple their selection algorithms with specific parameterized estimators, CoT-Core operates fundamentally as an estimator-agnostic framework. Our primary objective is to deliver a highly informative subset $\mathcal{S}$ in a strictly training-free setting. The resulting coreset is inherently flexible: it naturally supports Standard Aggregation (SA) as $f_{est}$ for a deterministic baseline, while remaining fully compatible with advanced probabilistic estimators if historical data is accessible.

3.2. CoT-Core Framework Overview

The severe absence of historical data in real-world scenarios forces coreset construction into a strict training-free setting. Under this severe constraint, conventional text embeddings fundamentally fail to capture the underlying logic. Compelled by this limitation, CoT-Core leverages unrolled reasoning trajectories as a necessary proxy to measure true semantic distance.

As illustrated in Figure 2, our framework systematically maps the reasoning manifold of evaluation datasets through a straightforward, three-stage pipeline: (1) eliciting zero-shot reasoning trajectories via a generator LLM (Section 3.3), (2) projecting these contextualized paths into a dense latent space (Section 3.4), and (3) applying training-free geometric clustering to extract the final representative coreset (Section 3.5).

3.3. Stage 1: CoT Trajectory Generation

As highlighted in our motivation, embedding the raw question text $q_i$ captures only superficial lexical features (e.g., vocabulary overlap), fundamentally failing to capture its underlying logic. To address this, our first stage shifts the representation space from the problem definition to the problem resolution path.

For each instance in the dataset $\mathcal{D}$, we utilize an instruction-tuned LLM as the generator, denoted as ${\mathcal{M}}_{gen}$. We append a standard Chain-of-Thought trigger (e.g., “Let’s think step by step”) to the question $q_i$, compelling the model to articulate its complete reasoning process. The generated trajectory is defined as:

$$t_i={\mathcal{M}}_{gen}(q_i\oplus p_{cot})$$

(3)

where ⊕ denotes string concatenation and $p_{cot}$ is the CoT prompt. Crucially, we retain this trajectory $t_i$ regardless of whether the final derived answer is correct. The unrolled path itself—even if ultimately flawed—explicitly exposes the underlying logic and step-by-step reasoning structure required to navigate the problem space.

3.4. Stage 2: Contextualized Trajectory Embedding

Instead of relying on history-dependent performance metrics, we directly leverage the structural density of the generated reasoning paths. Crucially, to ensure that the unrolled logic remains grounded in its original premise, we do not embed the trajectory in isolation. For each instance, we concatenate the raw question $q_i$ and the generated trajectory $t_i$ into a structured input sequence:

$$s_i=“\mathrm{Question}\text{:}”\oplus q_i\oplus “\mathrm{Reason}\text{:}”\oplus t_i$$

(4)

where ⊕ denotes string concatenation. This contextualized sequence $s_i$ is then mapped into a continuous, high-dimensional space using a pre-trained dense retrieval model $\mathcal{E}$, yielding the reasoning feature matrix $\mathbf{E}\in {\mathbb{R}}^{N\times d}$:

$${\mathbf{e}}_i=\mathcal{E}\left(s_i\right)$$

(5)

By embedding the logically unrolled CoT explicitly grounded in its problem context, ${\mathbf{e}}_i$ naturally captures the underlying reasoning manifold of the task. This effectively overcomes lexical divergence between lexically disparate but logically isomorphic problems.

3.5. Stage 3: Isomorphic Clustering and Selection

Because the embedding space now inherently encodes the underlying logic and structural equivalence, we apply standard k-means clustering to partition the N trajectory embeddings into B clusters, aligning with the coreset budget constraint. For each cluster, we select the problem instance whose reasoning embedding is closest to the geometric centroid. This training-free sampling strategy maximizes coverage of the reasoning manifold and systematically eliminates dataset redundancy without requiring prior evaluation data or trainable parameters.

Crucially, CoT-Core decouples the coreset selection $\mathcal{S}$ from the downstream proxy scoring function $f_{est}$. By default, it naturally accommodates Standard Aggregation (SA)—a deterministic baseline that applies selection-derived instance weights or reduces to an unweighted empirical mean. Furthermore, this formulation remains seamlessly compatible with advanced probabilistic models (e.g., GP-IRT) for parameterized estimation when historical response data is available.

4. Experiments

4.1. Experimental Setup

Datasets & Implementation Details. We evaluate CoT-Core on four widely adopted reasoning benchmarks: GSM8K [6], GPQA (Diamond split) [13], MMLU [8], and MMLU-Pro [22]. For trajectory generation (${\mathcal{M}}_{gen}$), we employ the open-weight Phi-4 model [1] via the OpenCompass framework [11] to ensure standardized reproducibility. The contextualized trajectories are then mapped into a continuous feature space using the BGE-M3 encoder [5] ($\mathcal{E}$).

Baselines & Data Splits. We compare CoT-Core against two paradigms of coreset selection. Training-free baselines include Random sampling, Question-Emb K-Means (clustering raw problem text embeddings), and the k-Center Greedy algorithm [14]. History-dependent baselines include Correctness K-Means (clustering historical binary response matrices) and IRT[12]. Evaluation logs are sourced from TinyBenchmarks [12] and the Open LLM Leaderboard [3]. For a rigorous evaluation of generalization, we hold out 100 disjoint models as the test set. For history-dependent methods, we fit latent parameters using a strictly disjoint training set of $N=25$ models to simulate practical low-resource settings (results for larger N are deferred to Appendix C).

Evaluation Protocol. Coresets are extracted at strict budgets of $1\%,5\%,10\%,15\%$, and $20\%$ of the full dataset size. To isolate selection quality from downstream aggregation, all subsets are evaluated using two decoupled estimators: Standard Aggregation (SA)—a deterministic baseline that applies selection-derived instance weights or reduces to an unweighted empirical mean—and GP-IRT[12], a parameterized historical estimator. The discrepancy between proxy estimations and true full-dataset scores is quantified via Mean Absolute Error (MAE) (↓) and Ranking Similarity (Sim) (↑). To rigorously mitigate initialization variance, we report the mean performance across 5 independent random seeds for all algorithmic methods, and 20 independent trials for the Random baseline.

4.2. Main Results

Delineating Boundary Conditions via Task Complexity. A cross-sectional analysis of Table 1 and Table 2 delineates our approach’s boundary conditions: CoT-Core’s efficacy is intrinsically gated by benchmark difficulty. On straightforward tasks like GSM8K, the method hits a lower boundary (simplicity floor), yielding noticeable but bounded improvements. However, on expert-level benchmarks (MMLU-Pro, GPQA) where raw text acts as an information bottleneck, entering this operative domain via CoT-Core provides massive informational lift. Remarkably, our training-free approach bridges the gap with history-dependent methods. Under the advanced GP-IRT estimator, CoT-Core frequently matches or surpasses IRT and Correctness K-Means at larger budgets (e.g., $\ge 10\%$ on MMLU/MMLU-Pro), proving that capturing latent logical structure can serve as a highly effective alternative to compiling expensive historical evaluation logs.

Robustness Across Estimators. Under the strictly training-free SA estimator (Table 1), CoT-Core’s CoT-aware coresets demonstrate robust representativeness compared to Question-Emb K-Means and Random sampling across the majority of selection budgets. When transitioning to the advanced GP-IRT estimator (Table 2), the absolute predictive errors (MAE) naturally compress across all methods due to the estimator’s parameterized advantage. Despite this tighter global margin, CoT-Core maintains highly competitive—and often leading—performance among training-free methods. Most notably, our approach showcases exceptional resilience in extreme high-compression regimes (e.g., $5\%$ to $10\%$), where it effectively preserves the relative performance ranking of evaluated models regardless of the scoring function. For instance, with a mere $5\%$ budget under GP-IRT, CoT-Core achieves a remarkable Ranking Similarity (Sim) of $0.5123$ on GPQA, nearly doubling the correlation preserved by traditional text clustering ($0.2759$), underscoring the structural stability of our reasoning-aware selection.

4.3. Analysis

The Simplicity Floor and Informational Boundaries.

CoT-Core’s performance gains are bounded by task complexity, directly reflecting the variance in discriminative information gain introduced by the CoT trajectories. In straightforward benchmarks like GSM8K (our lower boundary), reasoning paths predominantly involve homogeneous basic arithmetic. Consequently, unrolled trajectories offer limited structural variance, and CoT verbosity occasionally introduces linguistic noise rather than useful signals. Conversely, reasoning-aware pruning finds its robust operative domain in complex benchmarks demanding diverse underlying logical structures. Invoking advanced theorems for STEM questions, or explicitly evaluating distractors for non-STEM questions, provides powerful spatial discriminators. Crucially, this informational advantage is highly resilient: even when the generator (${\mathcal{M}}_{gen}$) hallucinates on excessively difficult questions, the specific nature of its attempted retrieval or logical confusion provides a far more informative clustering signal than compressed raw question text.

Exposing Cognitive Isomorphism via Logical Trajectories

To understand how CoT-Core externalizes implicit reasoning paths into explicit structural features within the representation space, we qualitatively analyze the reasoning manifold of the embeddings. We focus on GPQA, a highly complex benchmark spanning Physics, Chemistry, and Biology, as it provides an ideal testbed for observing cross-disciplinary semantic shifts.

Macro-Level Manifold: Breaking Lexical Traps.Figure 3 visualizes the embedding spaces before and after introducing Chain-of-Thought (CoT) trajectories. In the Question-Only space (left), the dense retriever naturally separates the three sciences into distinct clusters. However, this apparent separability is largely driven by lexical traps. Because domain-specific vocabularies (e.g., "cell" in Biology vs. "velocity" in Physics) are highly disjoint, the encoder creates artificial boundaries based on surface-level entities rather than intrinsic logical structures.

Conversely, the Question+CoT space (right) exhibits a fascinating phenomenon: while maintaining global structure, we observe deliberate inter-cluster merging. Specifically, the originally monolithic domain clusters (e.g., Biology, in green) begin to fragment, with individual instances clearly breaking away from their main group. Rather than being noise, this structural dispersion indicates that CoT-Core forces the embedding space to align with the underlying reasoning structures. Questions within the same domain that require fundamentally different logical trajectories (e.g., a biology question requiring mathematical calculation versus one requiring factual recall) are pulled apart, disrupting the artificial, lexicon-driven boundaries.

Micro-Level Case Study: Correcting the Logical Manifold. To validate that spatial reorganization captures underlying reasoning rather than surface semantics, we examine two contrasting GPQA scenarios (detailed in Appendix B).

Case 1: Uncovering Logical Isomorphism (False-Negative Correction). Two physics problems—one regarding a CERN Bubble Chamber and another on a spherical detector—exhibit a high Question-Only cosine distance of 0.5461 due to disjoint vocabularies. However, CoT unrolling reveals an identical mathematical skeleton (relativistic kinematics and time dilation). By encoding this shared logic, CoT-Core collapses their distance by 61% to 0.2128, ensuring clustering targets reasoning skills rather than experimental dressing.

Case 2: Escaping Lexical Traps (False-Positive Correction). Conversely, traditional encoders frequently conflate distinct problems sharing deceptive surface vocabulary. Two astrophysics questions about a $6000\mathrm{K}$ spotted star (quantum state population vs. macroscopic photometric transit) are dangerously entangled with a distance of 0.1957. CoT-Core exposes their divergent physical laws (microscopic Boltzmann distribution vs. macroscopic Stefan-Boltzmann law), shattering the lexical illusion and nearly doubling their distance to 0.3890 to prevent pruning non-redundant knowledge.

4.4. Ablation Studies

Impact of the CoT Generator: The Verbosity Penalty. We investigate the impact of the LLM used to externalize reasoning trajectories on MMLU-Pro, varying the generator from lightweight (1B) to frontier models (72B) while fixing the encoder to BGE-M3.

Table 3. Ablation on generator scale (MMLU-Pro). Lightweight models perform remarkably competitively, particularly at extreme compression ratios.

Generator	1% Budget	10% Budget	20% Budget
LLaMA-3-1B	$0.0280\pm 0.0204$	$0.0109\pm 0.0082$	$0.0057\pm 0.0043$
Mistral-8B	$0.0316\pm 0.0240$	$0.0081\pm 0.0058$	$0.0066\pm 0.0051$
Phi-4	$0.0295\pm 0.0241$	$0.0082\pm 0.0061$	$\mathbf{0}.\mathbf{0054}\pm \mathbf{0}.\mathbf{0043}$
Qwen-72B	$0.0305\pm 0.0236$	$\mathbf{0}.\mathbf{0077}\pm \mathbf{0}.\mathbf{0060}$	$0.0065\pm 0.0048$

Table 3. Ablation on generator scale (MMLU-Pro). Lightweight models perform remarkably competitively, particularly at extreme compression ratios.

Generator	1% Budget	10% Budget	20% Budget
LLaMA-3-1B	$0.0280\pm 0.0204$	$0.0109\pm 0.0082$	$0.0057\pm 0.0043$
Mistral-8B	$0.0316\pm 0.0240$	$0.0081\pm 0.0058$	$0.0066\pm 0.0051$
Phi-4	$0.0295\pm 0.0241$	$0.0082\pm 0.0061$	$\mathbf{0}.\mathbf{0054}\pm \mathbf{0}.\mathbf{0043}$
Qwen-72B	$0.0305\pm 0.0236$	$\mathbf{0}.\mathbf{0077}\pm \mathbf{0}.\mathbf{0060}$	$0.0065\pm 0.0048$

A counter-intuitive observation is that scaling to a massive 72B generator yields marginal, often statistically insignificant, clustering improvements over lightweight alternatives. This highlights a core premise of our framework: trajectory correctness is secondary to logical structure. Even when 1B models derive incorrect final answers, they successfully externalize the requisite reasoning steps, fully exposing the problem’s underlying reasoning manifold. Furthermore, frontier models suffer from a Verbosity Penalty. Extensive instruction tuning causes 72B models to generate excessive conversational boilerplate. Because dense encoders are sensitive to lexical frequency [15], this universal linguistic noise artificially inflates similarities between fundamentally disparate problems. Consequently, the compact, raw logical trajectories of smaller models prove highly robust for geometric clustering, confirming CoT-Core’s cost-efficiency.

Impact of Hierarchical Cleaning: The Optimal Abstraction Horizon. To isolate CoT-Core’s mechanisms, we evaluate three abstraction levels on GPQA using the Phi-4 generator (prompts in Appendix A.2): Raw CoT (unaltered), Cleaned CoT (conversational boilerplate removed, entities retained), and Abstract CoT (extreme compression retaining only pure logical structures).

As Table 4 shows, the intermediate Cleaned CoT consistently minimizes predictive error. Stripping universally shared boilerplate drastically reduces linguistic noise, sharpening manifold boundaries. However, extreme compression (Abstract CoT) severely degrades performance, exposing a critical failure mode: Semantic Starvation. Unlike elementary math, expert domains require domain-specific semantic anchors to contextualize the underlying logic. Stripping these disciplinary groundings deprives the encoder of essential contextual dimensionality. Thus, CoT-Core operates optimally by pruning generic noise while preserving the semantic anchors required to contextualize the latent logical structure.

5. Conclusions

We presented CoT-Core, a training-free coreset framework that circumvents surface lexical traps by clustering the underlying reasoning manifold of unrolled Chain-of-Thought trajectories, successfully aligning lexically disparate problems via their intrinsic logical isomorphism. Extensive evaluations confirm CoT-Core consistently yields high-fidelity proxy estimations for both standard and advanced probabilistic estimators. Crucially, we delineate the boundary conditions of reasoning-aware pruning: its efficacy intrinsically scales with task complexity, bounded by a simplicity floor on basic arithmetic while delivering massive informational gains on expert-level benchmarks. Furthermore, achieving optimal representation requires balancing model-induced conversational noise (Verbosity Penalty) against the severe loss of disciplinary context (Semantic Starvation). Ultimately, CoT-Core provides a scalable, data-free alternative to compiling expensive historical evaluation logs, proving that capturing latent logical structures is sufficient for robust benchmark compression and accelerating continuous LLM development.

Appendix A.1. Model Configurations

We employ the pre-trained dense encoder (e.g., BGE-M3) for contextualized trajectory embedding. For generating the Unrolled Reasoning Trajectories, we utilize the generator ${\mathcal{M}}_{gen}$ with the following generation hyperparameters: temperature $T=0.7$, and maximum token length set to 1024.

Appendix A.2. Prompt Templates for Hierarchical Cleaning

To elicit implicit reasoning from the generator and subsequently perform our hierarchical cleaning ablations (as discussed in Section 4.4), we utilize specific system prompts. To strictly control for confounding variables, both the initial reasoning generation and the subsequent hierarchical cleaning steps employ the identical Phi-4 model.

For the initial generation, we append a standard CoT trigger $p_{cot}$ (e.g., “Let’s think step by step”) to the original question $q_i$. The subsequent abstraction processes utilize the exact prompt templates presented below.

Appendix B.1. Case 1: Uncovering Logical Isomorphism (False-Negative Correction)

These two problems are lexically disjoint but share an identical underlying mathematical skeleton rooted in relativistic kinematics and time dilation.

Question A (Experimental Resolution):

In the CERN Bubble Chamber a decay occurs, $X^0\to Y^{+}{Z}^{-}$ in ${\tau }_0=8\times {10}^{-16}$ s, i.e. the proper lifetime of $X^0$. What minimum resolution is needed to observe at least 30% of the decays? Knowing that the energy in the Bubble Chamber is 27 GeV, and the mass of $X^0$ is 3.41 GeV.

Question B (Kinematic State):

Particles are collided at the center of a spherical detector producing new type of particles that travel uninterrupted at ultra-relativistic velocities highly centered around Lorentz factor of $\sim 20$. On average, one third of these fast-decaying particles reaches the detector inner walls. The radius of the detector is 30 meters. What Lorentz factor is needed in order to have about two thirds of these particles reaching the detector inner walls?

Analysis of the Isomorphism:

This case illustrates the False-Negative problem inherent in standard text-embedding approaches and how unrolled Chain-of-Thought (CoT) trajectories mitigate it.

If we rely solely on the problem formulations, Question A and Question B appear semantically disjoint. Question A is framed as an experimental high-energy physics problem (featuring entities like “CERN Bubble Chamber,” “27 GeV,” and “resolution”), whereas Question B is framed as a macroscopic kinematic problem (featuring “spherical detector,” “30 meters,” and “inner walls”). A naive embedding model would map these two questions to distant regions in the latent space due to their lexical divergence.

However, the unrolled CoTs reveal a shared, identical mathematical skeleton. Both reasoning trajectories sequentially invoke the same underlying physical principles:

1.

Relativistic Kinematics: Calculating or utilizing the Lorentz factor ($\gamma$).

2.

Time Dilation: Linking proper lifetime to dilated lifetime ($\tau =\gamma {\tau }_0$).

3.

Mean Decay Length: Formulating the distance traveled before decay ($d\approx c\gamma {\tau }_0$).

4.

Threshold Probability: Correlating the traveled distance with the survival/decay ratio.

By exposing this implicit reasoning structure, the CoT acts as a semantic equalizer. The generation of shared specialized tokens (e.g., $\gamma$, ${\tau }_0$, $v\approx c$) and structurally identical mathematical equations forces the embedding space to recognize their deep logical isomorphism. Consequently, these two instances, which would be falsely separated (False-Negative) in a purely lexical feature space, are successfully clustered together based on their true cognitive and reasoning demands.1

Appendix B.2. Case 2: Escaping Lexical Traps (False-Positive Correction)

These two problems share extensive surface-level vocabulary regarding spotted stars and effective temperatures, but require fundamentally orthogonal physical laws (quantum statistical mechanics vs. macroscopic photometric geometry).

Question C (Quantum States):

Astronomers are studying a star with a 1.5 solar radius and 1.1 solar masses. When the star’s surface is not covered by dark spots, its $T_{\mathrm{eff}}$ is 6000K. However, when 40% of its surface is covered by spots, the overall photospheric effective temperature decreases to 5500K. In the stellar photosphere, when examining the ratio of the number of neutral atoms of Ti in two energetic levels (level 1 and level 2), astronomers have observed that this ratio decreases when the star has spots. What is the factor by which this ratio changes when the star does not have spots compared to when it has spots? Note that the transition between the energy levels under consideration corresponds to a wavelength of approximately 1448 Å. Assume that the stellar photosphere is in LTE.

Question D (Photometric Flux):

Astronomers are currently observing a star with a radius equal to that of the Sun. One hemisphere of the star is covered in dark spots with a filling factor of 20%. The star has an effective temperature ($T_{\mathrm{eff}}$) of 6000K, and the spots exhibit a temperature difference of 1000K. As only one hemisphere is spotty, photometric time-series observations will reveal periodic variations in brightness due to rotational modulation. Interestingly, this situation can closely resemble the presence of an exoplanet. To produce the same amplitude signal in the star’s light curve (if the star was not covered by spots!), what should be the radius of a hypothetical exoplanet relative to the radius of the host star (i.e. $R_{\mathrm{pl}}/R_{\mathrm{star}}$)?

Analysis of the Orthogonality:

This case demonstrates the “Lexical Trap” (False-Positive) problem in benchmark evaluation and how unrolled CoT prevents erroneous clustering.

At the surface level, Question C and Question D are lexically entangled. Both problem descriptions share a high density of domain-specific keywords and identical numerical setups: “star,” “dark spots,” “effective temperature,” and “6000K.” A naive text-embedding model, which heavily weights term frequencies and surface-level semantic proximity, would inevitably map these two questions into the same cluster, classifying them as highly similar stellar astrophysics problems.

However, the unrolled CoT trajectories expose fundamentally orthogonal underlying mathematical and physical engines:

• Question C (Quantum Statistical Mechanics): The reasoning trajectory is entirely driven by atomic physics. It invokes the Boltzmann distribution to calculate microscopic population states, generating specialized tokens like energy differences ($\Delta E$), Planck’s constant (h), and exponential decay functions dependent on temperature ($exp(-\Delta E/kT)$).
• Question D (Macroscopic Photometry): The reasoning trajectory operates on geometric and thermodynamic scales. It relies on the Stefan-Boltzmann law (where flux is proportional to $T^4$) and geometrical area ratios (projected disk areas and filling factors) to calculate macroscopic light curve variations.

By expanding the implicit reasoning paths, the CoT introduces dimensionally distinct vocabularies (e.g., $hc/\lambda k$ vs. $T^4$ ratios). This forced articulation of the underlying physical laws acts as a semantic repeller in the latent space, effectively pushing these two lexically similar but logically distinct problems apart, thereby resolving the False-Positive collision.2