Mechanistic Interpretability for Understanding Language Abilities in Large Language Models: A Survey

1. Introduction

Large Language Models (LLMs) achieve strong performance across a wide range of language tasks, including translation and text generation. This success raises a fundamental question about the nature of their underlying linguistic knowledge: do LLMs internalize systematic grammatical and semantic abstractions akin to human-like competence, or do they primarily exploit surface-level statistical regularities in large-scale data to approximate linguistic performance? While their behavior suggests the emergence of nontrivial linguistic structure [19,100], it remains plausible that such results arise from sophisticated pattern matching without the underlying symbolic or rule-based representations traditionally associated with human language [15,28].

Mechanistic interpretability (MI) aims to clarify this distinction by decomposing a model’s blackbox predictions into the internal computational structures, such as specialized attention heads or individual neurons for linguistic competence, that produce them. Recent studies have identified structured “circuits” or feature subspaces that appear to correlate with specific linguistic phenomena, offering partial analogies to human cognitive processes [98,109]. However, a critical challenge in this field is the conceptual gap between mechanistic identification (e.g., locating a circuit) and linguistic attribution (e.g., confirming that the circuit implements a generalizable rule). By scrutinizing these internal states, researchers seek to determine whether a model truly implements abstract generalizations or merely leverages high-dimensional heuristics that are sufficient for its training distribution.

Scope and Audience

In this survey, linguistic competence refers primarily to internal knowledge of morphosyntax (e.g., agreement, binding, word order) and formal semantics (e.g., argument structure and thematic roles). We discuss discourse, pragmatics, reasoning, and neural alignment only where mechanistic evidence directly connects them to linguistic representations. This focus is intentionally narrower than the full space of modern agentic, multimodal, or tool-using systems: our goal is to ask what MI reveals about structured language knowledge in Transformer-based LLMs. The survey is written for linguists seeking a map of what MI can and cannot establish, MI researchers seeking linguistic grounding, and NLP researchers interested in why models succeed or fail on linguistic evaluations.

Figure 1. The overview of this survey.

Figure 1. The overview of this survey.

Prior surveys primarily categorize interpretability methods (Table 1), leaving a critical gap in synthesizing what these techniques reveal about linguistic competence. We argue that mechanistic evidence supports a layered account of linguistic implementation in LLMs: feature-level variables encode local grammatical and semantic distinctions; circuit-level mechanisms reuse computational subroutines across tasks; and multilingual models employ routing-level organization to link surface forms to shared semantic spaces. This thesis suggests that LLM behavior reflects structural organization rather than shallow memorization, without implying the existence of classical symbolic grammars.

This account occupies a middle position between two extremes. While mechanistic findings rule out a pure n-gram or rote-memorization explanation by identifying explicit pathways for linguistic competence (e.g., syntax), they do not justify claims of human-like symbolic competence. The discovered mechanisms remain distributed, frequency-conditioned, and model-specific. Therefore, we treat mechanistic evidence as a window into how specific models implement linguistic behavior, rather than proof of human-like cognitive architecture.

This survey presents a comprehensive overview of MI methods for studying linguistic competence in LLMs. Our contributions are as follows: (1) We review the evolution of interpretability techniques, from early probing approaches to recent advances, and clarify the linguistic claims each method can support. (2) We synthesize empirical evidence on how linguistic competence dynamically emerges in LLMs, uncover the underlying structural mechanisms, and evaluate its neurocognitive alignment with human language processing. (3) We extend the discussion to multilingual models, distinguishing between language-specific mechanisms and shared, language-agnostic representational spaces that support cross-lingual transfer. (4) We identify open challenges in scaling interpretability to increasingly large models, limitations of existing linguistic benchmarks, and tensions between interpretability and performance, and outline directions for future research.

2. Interpretability Methodology

In this paper, we review key methods for analyzing how LLMs represent linguistic knowledge (see Table 3 for a summary and a running example for all methods in Appendix 7.1.4).

2.1. Vocabulary Projection

Vocabulary Projection (or the Logit Lens) [124] interprets intermediate representations by projecting hidden states ${\mathbf{h}}_{\ell }$ directly onto the vocabulary via the unembedding matrix $W_U$, revealing real-time predictions [31]:

$${\mathrm{Logits}}_{\ell }=W_U·\mathrm{LayerNorm}\left({\mathbf{h}}_{\ell }\right)$$

(1)

This method tracks layer-by-layer prediction evolution [50]. Enhancements include the Tuned Lens [14], which calibrates intermediate representations, and the Future Lens [128], which predicts subsequent states.

Linguistic Leverage

Vocabulary projection helps localize when a model favors specific tokens (e.g., number-marked verbs). However, it does not confirm the causal necessity of these representations.

2.2. Activation Patching

To establish causality, Activation Patching systematically replaces a component’s activation in a corrupted input ($x_{\mathrm{corrupt}}$) with the corresponding activation from a clean input ($x_{\mathrm{clean}}$) [110,161,177]. If the intervention restores the model’s performance on a target task, that component is considered causally decisive [45,157]. Attribution Patching [121] further refines this by using gradients to approximate these interventions linearly.

The importance of a component i is quantified by its indirect effect (IE), measuring the change in target token probability $p_r$ after patching:

$$\begin{array}{c}\hfill p_r\left(f(x_{\mathrm{corrupt}};a_i\leftarrow a_i\left(x_{\mathrm{clean}}\right))\right)-p_r\left(f\left(x_{\mathrm{corrupt}}\right)\right)\end{array}$$

(2)

While powerful, these methods require caution [68]. [105] warn of an “interpretability illusion,” where interventions might activate dormant pathways rather than revealing the natural mechanism.

Linguistic Leverage

Patching provides direct causal evidence for linguistic tasks. Its validity depends on demonstrating generalizability across diverse lexical items, templates, and distractors to avoid identifying prompt-specific pathways.

2.3. Sparse Autoencoders

A fundamental barrier to interpreting individual units (e.g., neurons) is superposition, where models compress more features than they have dimensions, resulting in polysemantic neurons [39]. For example, a single neuron might fire for both “plural nouns” and “Python code comments,” making it impossible to assign it a unique linguistic label. Sparse Autoencoders (SAEs) address this by decomposing dense, polysemantic activations into an overcomplete, sparse basis where each direction represents a single concept [16,67,76,143,155]. SAEs reconstruct an activation $\mathbf{x}$ as a linear combination of features:

$$\mathbf{x}\approx \widehat{\mathbf{x}}={\mathbf{b}}_{dec}+\sum _i{f}_i\left(\mathbf{x}\right){\mathbf{d}}_i$$

(3)

where $f_i\left(\mathbf{x}\right)$ is the sparse feature activation (via $L_1$ regularization) and ${\mathbf{d}}_i$ is the decoder direction.

Researchers have used this to identify monosemantic units for specific linguistic phenomena (e.g., "German plural endings") [79]. However, [82] caution that SAEs require rigorous evaluation, as they do not consistently outperform baselines in downstream tasks.

Linguistic Leverage

SAEs improve inspectability by disentangling dense activations. The primary evidential challenge is demonstrating that these extracted features are causally active: does ablating, steering, or tracing a specific feature reliably alter the corresponding linguistic behavior in a model?

2.4. Circuit Discovery

While SAEs disentangle representations into interpretable features, these units can also be composed into higher-level computational mechanisms. Building on activation patching, Circuit Discovery [115,177] aims to identify the complete subgraph of components, such as attention heads and neurons, that are responsible for a specific task. While early work relied on manual hypothesis testing [125], recent approaches employ automated search algorithms. Automated Circuit Discovery [32] automates the process of pruning the computational graph. It iteratively ablates edges in the network, keeping only those whose removal would significantly degrade performance on a specific task. To address the difficulty of analyzing dense MLP layers in circuits, [38] introduce Transcoders, which approximate MLPs with sparse, interpretable components. More recently, researchers have shifted focus from “component circuits” (where nodes are heads/MLPs) to “SAE feature circuits”. [107] and [86] utilize sparse autoencoders to define circuits over fine-grained semantic features rather than coarse model components. This allows for the discovery of circuits that are more interpretable, as edges represent causal dependencies between specific concepts (e.g., a “plural subject” feature causing a “plural verb” prediction) rather than opaque vectors.

Linguistic Leverage

Circuit discovery is most informative when it identifies a reusable algorithmic pathway rather than a single correlated unit. For language, the central test is whether the circuit implements an abstraction across lexical and structural variation, or whether it only solves a narrow benchmark template.

2.5. Activation Steering

Methodology has expanded from observation to control, enabling direct intervention in a model’s linguistic behavior. Activation Steering involves injecting vectors [83,131,157,171] into the residual stream to elicit specific behaviors. [159] introduced Activation Addition (ActAdd), utilizing mean activation differences between positive and negative examples. To improve robustness, Contrastive Activation Addition (CAA) [134] refines this by averaging differences across contrastive pairs (e.g., “Helpful” vs. “Unhelpful”). Building on this, Steering Target Atoms (STA) [163] leverages SAEs to decompose steering vectors into sparse, interpretable features, allowing for more precise control with fewer side effects.

2.5.0.6. Linguistic Leverage

Steering links explanation to intervention: if linguistic properties are predictably shifted, the underlying representation is confirmed as functional rather than merely descriptive. However, these results require extra validation to ensure that steering does not inadvertently degrade unrelated fluency, style, or factual accuracy.

3. MI on Linguistic Competence

This section reviews MI evidence on linguistic competence, focusing on how grammatical knowledge emerges during training, how it is implemented in model internals, and how far these representations align with human language processing. Together, these perspectives shift the analysis from whether models perform well on linguistic tasks to how such competence is formed, organized, and constrained.

3.1. Dynamic View: How It Emerges

Behavioral benchmarks often report strong performance on core grammatical phenomena [72]. However, analyses of internal model states suggest that this competence is not a monolithic body of knowledge acquired instantaneously, but rather the outcome of a complex, frequency-dependent developmental process.

Critical Periods and Structural Evolution Evidence suggests that the timing of language acquisition may be structurally important. [36] identify “critical periods” in training, specific windows where rapid internal syntactic specialization occurs. [127] and [10] trace this evolution using circuit analysis and crosscoders, observing that linguistic features are not merely added but actively consolidated during pre-training. This temporal evolution is complemented by cross-scale findings from [103], who note that syntactic acquisition also saturates at specific model scales.

From Memorization to Abstraction The acquisition of language competence during training appears to follow a hierarchical trajectory shaped by data distribution. In early stages, models preferentially learn high-frequency tokens and local n-grams, while rarer and long-distance dependencies emerge only through gradual refinement [21,165]. This suggests that linguistic abstraction is not purely symbolic but remains anchored to the statistical density of the training data. At the mechanistic level, this temporal progression has been linked to shifts in internal circuitry, including reported transitions from transient in-context strategies to more stable in-weights representations [144]. For a comprehensive review, see [85].

What We Learn

A recurring pattern across these studies is developmental: formal linguistic competence appears to emerge through staged, frequency-dependent trajectories rather than all at once. Mechanistic evidence points to increasingly stable grammatical variables during training, but claims about abstract rules are best supported when they survive lexical, frequency, and structural controls rather than only benchmark-level accuracy.

3.2. Mechanistic View: Specialized vs. Shared Circuits

Following the gradual emergence of linguistic capabilities during training, it remains unclear whether neural networks rely on isolated modules or reconfigure overlapping subsystems to support diverse tasks. While numerous studies have identified language-specific neurons and features, recent circuit-level analyses increasingly challenge the assumption of strict functional independence [89]. [112] report that circuits identified for Indirect Object Identification (IOI) are reused in tasks sharing similar algorithmic structure, with substantial overlap in attention heads. These findings suggest that models rely on reusable circuits rather than strictly task-specific mechanisms [e.g., [32]. Beyond circuit overlaps, researchers have identified functional neurons whose targeted perturbation can degrade or redirect performance. The works [58,77,154,170] report that ablating these neurons yields large drops in downstream accuracy.

However, the degree of localization varies: simple local agreement checks may be handled by a compact set of heads [61,125], whereas complex reasoning requires interactions among numerous polysemantic units [107].

What We Learn

Current circuit evidence favors reusable subroutines over fully isolated linguistic modules. Duplicate-token detection, agreement routing, and mover/inhibition patterns suggest that models can reuse low-level mechanisms while attaching task-specific execution heads.

3.3. Cognitive View: Partial Alignment with Human Language Processing

Recent works [3,70,91] suggest that the relationship between LLMs and human language processing can be characterized as partial alignment: systematic correspondences emerge at specific representational levels, while substantial divergences persist elsewhere.

Alignment is most relevant for this survey at the level of language-selective structure. [2] identify language-selective units across a wide range of LLMs, which exhibit stronger alignment with neural activity in human language regions than randomly sampled areas [91]. At a finer grain, [37] use theory-driven psycholinguistic paradigms to probe neuron-level mechanisms, reporting that when a model exhibits human-like behavior in tasks such as sound–gender association, a small subset of neurons can be causally identified.

At the representational level, hidden states can predict neural responses during naturalistic language comprehension [47,71]. Furthermore, this alignment exhibits a layered correspondence between model depth and the temporal hierarchy of neural responses [52]. However, this alignment is selective. Tracking models across training, [3] report that brain alignment correlates more strongly with formal linguistic competence than with functional abilities such as reasoning.

What We Learn

Cognitive alignment findings are suggestive but bounded. The most relevant evidence concerns formal linguistic competence and language-selective representations; broader pragmatics and human-like cognition remain less directly explained by current MI.

4. Multilingualism

This section reviews MI evidence on multilingualism, focusing on how multilingual LLMs represent language-specific information, enable cross-lingual transfer, and align semantic content across languages. Current evidence points to a hybrid organization of multilingual representations. Language-specific components appear to govern language identity and surface-form processing, whereas shared representations are more closely associated with transfer and semantic alignment.

4.1. Language-Selective Components

A growing body of work reports language-selective components in multilingual decoder-style LLMs at multiple granularities, from coarse region-level localization to fine-grained, steerable features.

At the neuron level, language competence exhibits partial spatial localization. [88] find these language-specific neurons are concentrated in early and late layers, and that perturbing fewer than 1% of neurons can substantially alter output language probabilities. [154] further study this localized sensitivity by introducing Language Activation Probability Entropy to pinpoint these neurons, reporting that deactivating a small subset selectively impairs comprehension or generation in a target language. Beyond individual neurons, region-level analyses show similar patterns. [182] identify distributed monolingual regions associated with the syntactic and lexical properties of particular languages.

Beyond static neuron parameters, dynamic routing mechanisms further isolate language processing. On the attention side, [102] identify both language-specific and language-general attention heads. Circuit-level analyses further support this view: Anthropic’s circuit tracing work on Claude 3.5 Haiku shows that multilingual behavior recruits language-dominant clusters alongside abstract, shared pathways, with the proportion of shared circuitry increasing with model scale [5,98].

Recent SAE-based studies provide additional perspective and fine-grained steering. [33] and [6] identify monosemantic language-specific features whose ablation selectively degrades individual languages. Building on this, [186] propose that language control relies on a sparse, cross-layer set of dimensions, and [30] report zero-shot language control through sparse feature steering.

Overall, these findings suggest that multilingual LLMs, despite extensive parameter sharing, preserve some specialized internal structures that support language-specific processing.

What We Learn

Language-selective components offer a useful entry point to study how multilingual LLMs encode language identity, typological variation, and language-specific form. Current evidence suggests that parameter sharing does not eliminate specialized structure; rather, multilingual models combine localized language-sensitive units with broader shared pathways.

4.2. Cross-Lingual Transfer

We define cross-lingual transfer as a model’s ability to reuse knowledge or task competence learned in one language when processing inputs in another. Mechanistically, recent evidence supports a two-component view: transfer is primarily enabled by language-agnostic computation in intermediate layers, while language-specific units help route information between language-dependent surface forms and that shared computation.

The Multilingual Workflow and Shared Latent Spaces. [184] formalize this as a multilingual workflow, where non-English inputs are progressively mapped into an English-like internal workspace for reasoning, and later projected back to the query language during generation. This remains a model-specific hypothesis rather than a settled account of multilingual computation. The mechanistic explanations for this transfer are becoming more precise: [156] propose the Transfer Neuron Hypothesis, identifying specific MLP neurons associated with transformations between language-specific and shared latent spaces. As evidence for partial functional independence, [184] report that targeted tuning of those neurons can selectively improve a language without broadly affecting others. To map this flow to the entire network, [64] utilize cross-layer transcoders, finding that while internal representations are largely shared, language-specific decoding emerges in the final layers driven by high-frequency features. Furthermore, [17] reports shared feature directions for grammatical concepts even in models trained primarily on English.

Developmental Dynamics of Transfer. [22] posit that during training, an LLM initially allocates its resources to a “primary” or dominant language, only later partitioning its internal representation to accommodate new languages. Their empirical analysis tracks language transferring neurons that mark the point at which the model begins establishing dedicated circuits for additional languages. [133] observe a compression process during training, where models initially form language-specific representations that gradually converge into cross-lingual abstractions.

While the localization of cross-lingual mechanisms is becoming precise, the causal utility of these components remains contested. [114] systematically test neuron-specific interventions and find that manipulating language-specific neurons does not reliably improve downstream cross-lingual performance on benchmarks such as XNLI and XQuAD for low-resource languages.

What We Learn

Cross-lingual transfer appears to depend on an interaction between language-specific routing and shared task-solving computation. Localizing language-specific neurons is therefore not sufficient: the open question is how these components coordinate with shared intermediate representations, and why targeted interventions sometimes change language identity without improving downstream transfer.

4.3. Mechanisms of Semantic Alignment

While multilingual LLMs develop components that exhibit strong language selectivity, a growing body of work shows that these models converge on a shared latent space for semantic content. [185] and [166] find that representations for inputs in different languages often pass through English-like intermediate spaces. Generalizing this observation, [175] propose a hidden-state lingua franca: a shared semantic manifold to which inputs are mapped irrespective of language, with larger models and increased training further strengthening this language-agnostic alignment.

This shared semantic structure is supported by partially overlapping subspaces and token representations. [176] report that multilingual embeddings cluster near-synonymous tokens across languages in overlapping regions, while [101] find that domain knowledge is integrated into these spaces in a way that balances shared semantic representations with language- or domain-specific structure. Together, these results point to a dual organization where localized mechanisms coexist with a common semantic hub.

The shared space itself may be English-centric in some models. Rather than showing that multilingual LLMs literally “think in English,” [139] provide evidence that intermediate states can become English-aligned before generation in the target language. Related evidence extends beyond translation: [17] report that grammatical concepts such as number and gender can be encoded in shared feature directions across typologically diverse languages, and [146] find semantic alignment across models of varying scales.

Overall, these findings suggest that multilingual LLMs implement a hybrid representational strategy: language-specific circuits capture idiosyncratic form, while higher-level semantic representations converge into a shared latent space. Understanding this balance between specialization and convergence has practical implications for model design, enabling targeted control of language-specific behavior while preserving cross-lingual generalization.

What We Learn

The multilingual evidence extends the layered account: local language-specific features and circuits handle surface-form control, while higher-level representations increasingly converge into a shared semantic hub. The English-centric nature of this hub is plausible but not settled; it should be tested across more low-resource, non-Indo-European, and typologically distant languages before being treated as a universal property of multilingual LLMs.

5. Challenges and Open Questions

Despite progress in MI, model scaling complicates the isolation and manipulation of linguistic mechanisms. Simultaneously, existing evaluation paradigms face increasing scrutiny regarding their scope and inclusivity.

MI is also most useful when triangulated with complementary approaches. Behavioral evaluation constrains what a model can do; probing and representational geometry characterize what information is encoded; cognitive modeling tests whether model behavior and representations align with human processing; and MI contributes causal internal evidence about how a behavior is produced. A full account of linguistic competence requires these sources of evidence to be interpreted together.

5.1. Scaling vs. Interpretability Trade-Off

While scaling enhances performance, it challenges fine-grained interpretability. [96] observe that circuit-level analysis becomes increasingly intractable at scale, suggesting emergent behaviors may resist decomposition. Furthermore, [109] argue that extensive polysemanticity in complex models precludes unique interpretations. Conversely, [32] provide evidence that automated circuit discovery can scale beyond fully manual analysis, indicating that methodological advances may offset some complexity. However, the tractability of such approaches, and techniques like SAE [79,107], for massive-scale models remains unproven.

5.2. Evaluation Gaps

Although benchmarks like CausalGym [7], Holmes [160] and BHASA [93] have expanded linguistic coverage, significant gaps persist. Minimal-pair and targeted syntactic tests [7,73] effectively probe local grammar but often neglect higher-level reasoning, discourse, and pragmatics. Moreover, the focus on high-resource languages limits the detection of failures in underrepresented contexts, risking interpretability methods that overfit to narrow linguistic variations. Developing robust, culturally sensitive frameworks for low-resource languages remains a critical challenge [20,175].

5.3. Causality and Model Editing

A core goal of MI is causal intervention. Techniques such as attribution patching [43] and neuron or feature steering [8,98] test whether specific components can be causally linked to linguistic behavior. However, these interventions often lack stability: localized changes can propagate through the network, producing unintended side effects [149].

Superposition further complicates causal control. As shown by [39], polysemantic neurons encode multiple features simultaneously, making it difficult to edit one function without disrupting others. Developing reliable methods to identify modular, independently editable circuits and organize them into a safe, reusable “circuit library” remains an open challenge.

5.4. Cognitive Plausibility and Human-Like Generalization

Alignment between LLMs and human language processing remains partial. [3] observe that while specific units mirror human language networks, divergence increases as models integrate broader reasoning capabilities. Training dynamics also echo human acquisition via staged morphological and syntactic learning [29,36,41], yet generalization remains limited: LLMs transfer argument-structure patterns mainly when the relevant contextual relation was seen in pre-training [167]. Distinguishing shared cognitive implementation from behavioral convergence is critical for interpretability [183]. Despite calls for analyzing critical training transitions [7], the fundamental cognitive plausibility of LLM competence remains an open question.

5.5. Ethical and Societal Dimensions

MI also raises ethical considerations, particularly regarding bias, fairness, and language equity. Neuron-level analyses can expose how models encode stereotypes or toxic associations, enabling targeted mitigation strategies [78,94]. From a global perspective, interpretability highlights persistent linguistic inequities: multilingual models remain strongly English-centric [54], with uneven performance across languages [20]. Developing interpretable and equitable models for underrepresented languages, remains an underexplored but critical challenge.

5.6. Implications of Synthetic Data and Model Collapse

As LLMs are increasingly trained on machine-generated data, interpretability faces new uncertainties. [55] show that iterative training on synthetic text can lead to lexical or syntactic homogenization, potentially simplifying internal circuits. Moreover, large-scale synthetic corpora training may also distort internal representations in ways that obscure the origins of linguistic behavior [20,190]. Whether mechanistic insights derived from current models will generalize to future systems trained predominantly on synthetic data remains an open and pressing question.

6. Conclusion

This survey reviewed MI advances for understanding linguistic abilities in LLMs, from static probing to circuit discovery [32] and causal intervention [110]. Together, these methods provide evidence about candidate mechanisms underlying grammatical and semantic competence and support a hybrid view in which LLMs can exhibit structured abstraction while still relying on surface heuristics.

The reviewed evidence is difficult to reconcile with a pure memorization account: several studies identify reusable circuits for grammatical phenomena such as indirect object identification and subject–verb agreement [112,161]. At the same time, these capabilities are implemented through distributed, polysemantic representations [39,107] and frequency-dependent learning dynamics [21,165], which differ from classical symbolic accounts of grammar. In multilingual settings, this complexity is further reflected in evidence for language-specific components [154,182] and proposed shared semantic representations such as a model-internal lingua franca [171,175].

Looking ahead, progress will hinge on addressing the trade-off between model scale and interpretability [96], developing evaluation benchmarks that extend beyond local syntactic competence [34], expanding multilingual replication beyond high-resource languages, and triangulating MI with behavioral and cognitive evidence.

Appendix A.1. Related Works

Appendix A.1.1. Foundational Interpretability Studies

Early interpretability research on RNNs and LSTMs used behavioral diagnostics and hidden-state analysis to show that individual neurons capture long-distance syntactic dependencies [92,99].

As focus shifted to transformers (e.g., BERT, GPT), studies utilized causal and correlational methods to link attention heads and hidden states to linguistic phenomena [45,119,130,168]. These models distribute linguistic information across layers for increased compositional flexibility [63], with early surveys [137] synthesizing these insights to establish a basis for mechanistic interpretability.

Appendix A.1.2. Benchmarks and Linguistic Probing

Researchers evaluate grammatical competence using controlled benchmarks, primarily employing the minimal pairs paradigm to compare model likelihoods for grammatical vs. ungrammatical sentences [145,147,152,164,172] (See Table 2 for a summary).

To minimize surface heuristic reliance, more rigorous approaches include SyntaxGym [49] for standardized evaluation, Holmes [160] for classifier-based hidden-state probing, and HLB [35] for assessing psycholinguistic humanlikeness. Furthermore, BHASA [93] extends these diagnostics to Southeast Asian languages.

However, because these benchmarks measure behavioral outcomes rather than causal mechanisms, they risk rewarding spurious correlations [7]. This limitation drives the shift toward MI, which targets the internal computational structures that causally generate linguistic behavior.

Appendix A.1.3. Shift from Model-Agnostic to Model-Specific Interpretability

To move beyond model behavior, interpretability has shifted from model-agnostic saliency methods to transformer-specific techniques [75,137]. By explicitly targeting structural components like attention heads, feed-forward layers, and the residual stream, these methods enable precise causal interventions and analysis of information flow [40,113,125].

This transition is bolstered by integrated toolkits such as the LM Transparency Tool [158], Neuronpedia [97], and TransformerLens [122].

Appendix A.1.4. Running Example: Subject–Verb Agreement

Subject–verb agreement illustrates how the methods above form a cumulative evidential chain rather than separate tool descriptions. Behavioral minimal pairs first test whether a model prefers The keys are over The keys is, but such success may still reflect local frequency or lexical memorization. Vocabulary projection can then show when number-consistent verb predictions become available across layers. Activation patching tests whether specific heads, MLP activations, or residual-stream states are causally necessary for transferring number information from the subject to the verb position. SAEs and probes can identify candidate number features, while circuit discovery asks whether those features compose into a reusable pathway that survives attractor nouns and lexical substitutions. A strong mechanistic claim therefore requires more than one positive result: it requires causal necessity, generalization beyond templates, and evidence against simpler frequency-based explanations.

Figure 2. A taxonomy of the MI landscape for LLMs. Click on the nodes to jump to the corresponding section.

Figure 2. A taxonomy of the MI landscape for LLMs. Click on the nodes to jump to the corresponding section.

Table 2. A comparative summary of minimal pair datasets. The list includes monolingual and multilingual benchmarks, detailing their target language, total number of samples, and the count of specific linguistic paradigms evaluated.

Dataset	Language	Size	Paradigms
BLiMP [164]	English	67k	67
CLiMP [172]	Chinese	16k	16
SLING [147]	Chinese	38k	38
ZhoBLiMP [103]	Chinese	35k	118
JBLiMP [145]	Japanese	331	39
RuBLiMP [152]	Russian	45k	45
BLiMP-NL [150]	Dutch	8.4k	84
QFrBLiMP [12]	Quebec-French	1761	20
TurBLiMP [11]	Turkish	16k	16
UrBLiMP [1]	Urdu	5696	19
Irish-BLiMP [108]	Irish	1020	102
Arabic MPs [4]	Arabic	3000	9
BLiMP-IT [9]	Italian	2899	78
CLAMS [118]	English, French, German, Hebrew and Russian	229.9k	7
MultiBLiMP 1.0 [81]	101 languages	128321	2
BHS [90]	Basque, Hindi, Swahili	300	3

Table 2. A comparative summary of minimal pair datasets. The list includes monolingual and multilingual benchmarks, detailing their target language, total number of samples, and the count of specific linguistic paradigms evaluated.

Dataset	Language	Size	Paradigms
BLiMP [164]	English	67k	67
CLiMP [172]	Chinese	16k	16
SLING [147]	Chinese	38k	38
ZhoBLiMP [103]	Chinese	35k	118
JBLiMP [145]	Japanese	331	39
RuBLiMP [152]	Russian	45k	45
BLiMP-NL [150]	Dutch	8.4k	84
QFrBLiMP [12]	Quebec-French	1761	20
TurBLiMP [11]	Turkish	16k	16
UrBLiMP [1]	Urdu	5696	19
Irish-BLiMP [108]	Irish	1020	102
Arabic MPs [4]	Arabic	3000	9
BLiMP-IT [9]	Italian	2899	78
CLAMS [118]	English, French, German, Hebrew and Russian	229.9k	7
MultiBLiMP 1.0 [81]	101 languages	128321	2
BHS [90]	Basque, Hindi, Swahili	300	3

Table 3. Comparison of core interpretability methods and intervention methods in LLMs. Each method is characterized by its representative studies, key idea, strengths and limitations.

Methodology	Representative Works	Key Idea	Strengths and Limitations
Methodology	Representative Works	Key Idea	Strengths and Limitations
Vocabulary Projection	Logit Lens [50,124]; Tuned Lens [14]; Future Lens [128].	Decodes intermediate hidden states into vocabulary tokens to reveal the model’s evolving predictions layer-by-layer.	Strengths: Maps word vectors across languages rapidly; no model retraining needed. Limitations: Ineffective for distant languages; performance depends on initial dictionary quality; struggles with new words or complex tasks.
Patching	Activation Patching/Attribution Patching [7,43,110,121,151,177]	Identifies causally decisive components by swapping activations between inputs or using gradient-based approximations.	Strengths: Pinpoints layers/heads encoding key information; enables transplant experiments; shows causal roles of activations. Limitations: Sensitive to superposition; computationally costly.
Sparse Autoencoders	SAE, Transcoder, Cross-layer Transcoder, Crosscoder and Binary Autoencoder [16,26,39,76]	Decomposes polysemantic neurons into sparse, interpretable feature directions, resolving superposition.	Strengths: Disentangles polysemantic neurons; reveals interpretable subspaces; quantifies feature entanglement. Limitations: Requires training or specialized modeling; imperfect disentanglement.
Automated Circuit Discovery	Circuit Tracing [5,32,107]	Automatically finds minimal subgraphs (component- or feature-level) that implement specific tasks.	Strengths: Traces information flow via minimal subgraphs; reveals emergent “circuit reuse”; mechanistically grounded. Limitations: Hard to automate fully; scalability remains challenging.
Activation Steering	ActAdd, CAA, STA [42,110,111,134,157,159,163]	Manipulates model behavior via inference-time activation injection (Steering) or permanent weight modification (Editing).	Strengths: Directly alters internal behavior; useful for debiasing and control; connects interpretability with intervention. Limitations: Can disrupt unrelated behaviors; hard to find minimal safe edits.

Table 3. Comparison of core interpretability methods and intervention methods in LLMs. Each method is characterized by its representative studies, key idea, strengths and limitations.

Methodology	Representative Works	Key Idea	Strengths and Limitations
Methodology	Representative Works	Key Idea	Strengths and Limitations
Vocabulary Projection	Logit Lens [50,124]; Tuned Lens [14]; Future Lens [128].	Decodes intermediate hidden states into vocabulary tokens to reveal the model’s evolving predictions layer-by-layer.	Strengths: Maps word vectors across languages rapidly; no model retraining needed. Limitations: Ineffective for distant languages; performance depends on initial dictionary quality; struggles with new words or complex tasks.
Patching	Activation Patching/Attribution Patching [7,43,110,121,151,177]	Identifies causally decisive components by swapping activations between inputs or using gradient-based approximations.	Strengths: Pinpoints layers/heads encoding key information; enables transplant experiments; shows causal roles of activations. Limitations: Sensitive to superposition; computationally costly.
Sparse Autoencoders	SAE, Transcoder, Cross-layer Transcoder, Crosscoder and Binary Autoencoder [16,26,39,76]	Decomposes polysemantic neurons into sparse, interpretable feature directions, resolving superposition.	Strengths: Disentangles polysemantic neurons; reveals interpretable subspaces; quantifies feature entanglement. Limitations: Requires training or specialized modeling; imperfect disentanglement.
Automated Circuit Discovery	Circuit Tracing [5,32,107]	Automatically finds minimal subgraphs (component- or feature-level) that implement specific tasks.	Strengths: Traces information flow via minimal subgraphs; reveals emergent “circuit reuse”; mechanistically grounded. Limitations: Hard to automate fully; scalability remains challenging.
Activation Steering	ActAdd, CAA, STA [42,110,111,134,157,159,163]	Manipulates model behavior via inference-time activation injection (Steering) or permanent weight modification (Editing).	Strengths: Directly alters internal behavior; useful for debiasing and control; connects interpretability with intervention. Limitations: Can disrupt unrelated behaviors; hard to find minimal safe edits.

Table 4. Representative papers in MI. We categorize works into Method (foundational techniques, tools, frameworks, or benchmarks) and Application (studies leveraging these methods to investigate specific linguistic capabilities). The table spans five core domains: Probing, Vocabulary Projection (Logit Lens), Activation Patching, Sparse Autoencoders (SAE), and Circuit Discovery.

Paper Title	Type	Venue
Benchmarks
MIB: A Mechanistic Interpretability Benchmark [117]	Method	ICML
AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders [169]	Method	ICML
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders [84]	Method	ICML
InterpBench: Semi-Synthetic Transformers for Evaluating MI Techniques [56]	Method	NIPS
Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Senses [18]	Method	NAACL
CausalGym: Benchmarking causal interpretability methods on linguistic tasks [7]	Method	ACL
Holmes: A Benchmark to Assess the Linguistic Competence of Language Models [160]	Method	TACL
SyntaxGym: An Online Platform for Targeted Evaluation of Language Models [49]	Method	ACL
Probing
Lexical Popularity: Quantifying the Impact of Pre-training for LLM Performance [136]	Method	Arxiv
Steering Embedding Models with Geometric Rotation: Mapping Semantic Relationships Across Languages and Models [46]	Method	Arxiv
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks [126]	Application	Arxiv
ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework [178]	Method	ACL
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models [162]	Application	ACL
Probing Syntax in Large Language Models: Successes and Remaining Challenges [34]	Application	COLM
Probing Internal Representations of Multi-Word Verbs in Large Language Models [87]	Application	MWE
Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? [24]	Method	ACL
Finding Neurons in a Haystack: Case Studies with Sparse Probing [61]	Application	Arxiv
Probing Classifiers: Promises, Shortcomings, and Advances [13]	Method	CL
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT [120]	Application	EACL
Finding Universal Grammatical Relations in Multilingual BERT [23]	Application	ACL
On the Language Neutrality of Pre-trained Multilingual Representations [95]	Application	EMNLP
Emergent Linguistic Structure in Artificial Neural Networks Trained by Self-supervision [106]	Application	PNAS
A Structural Probe for Finding Syntax in Word Representations [69]	Application	NAACL
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information [51]	Application	BlackboxNLP
Vocabulary Projection (Logit Lens)
Eliciting Latent Predictions from Transformers with the Tuned Lens [14]	Method	Arxiv
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State [128]	Method	CoNLL
Interpreting GPT: the logit lens [124]	Method	Blog
Unraveling Syntax: How Language Models Learn Context-Free Grammars [138]	Application	Arxiv
The Semantic Hub Hypothesis: Language Models Share Semantic Representations [171]	Application	ICLR
Do Multilingual LLMs Think in English? [139]	Application	Workshop
Do Llamas Work in English? On the Latent Language of Multilingual Transformers [166]	Application	ACL
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models [31]	Application	ICLR
Sparse AutoEncoders (SAE)
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing [82]	Method	ICML
Binary Autoencoder for Mechanistic Interpretability of Large Language Models [26]	Method	Arxiv
LinguaLens: Towards Interpreting Linguistic Mechanisms via Sparse Auto-Encoder [79]	Method	EMNLP
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models [67]	Method	Arxiv
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima [155]	Method	Arxiv
The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans [25]	Application	Arxiv
From Syntax to Emotion: A Mechanistic Analysis of Emotion Inference in LLMs [142]	Application	Arxiv
Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMs [80]	Application	Arxiv
What’s the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering [104]	Application	ICLR
Crosscoding Through Time: Tracking Emergence of Linguistic Representations [10]	Application	ICML
Large Language Models Share Representations of Latent Grammatical Concepts [17]	Application	NAACL
Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models [62]	Application	NAACL
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages [6]	Application	Arxiv
Unveiling Language-Specific Features via Sparse Autoencoders [33]	Application	ACL
Analyzing Multilingualism in Large Language Models with Sparse Autoencoders [27]	Application	COLM
Sparse Autoencoders Can Capture Language-Specific Concepts [6]	Application	Arxiv
Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders [64]	Application	Arxiv
Causal Language Control in Multilingual Transformers via Sparse Feature Steering [30]	Application	Arxiv
Semantic Convergence: Investigating Shared Representations Across Scaled LLMs [135]	Application	SRW
Extended Abstract for “Linguistic Universals”: Emergent Shared Features in Independent Monolingual Language Models via Sparse Autoencoders [187]	Application	MRL
Sparse Autoencoders Find Highly Interpretable Features in Language Models [76]	Method	ICLR
Transcoders Find Interpretable LLM Feature Circuits [38]	Method	NIPS
Activation Patching
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context [57]	Method	Arxiv
How to use and interpret activation patching [68]	Method	Arxiv
Is This the Subspace You Are Looking for? An Interpretability Illusion [105]	Method	ICLR
Towards Best Practices of Activation Patching in Language Models [177]	Method	ICLR
Function Vectors in Large Language Models [157]	Method	ICLR
Attribution Patching Outperforms Automated Circuit Discovery [151]	Method	Blackbox
Attribution Patching: Activation Patching At Industrial Scale [121]	Method	BLOG
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models [45]	Method	IJCNLP
The Dual-Route Model of Induction [44]	Application	COLM
Neuron Analysis
From Directions to Regions: Decomposing Activations in Language Models via Local Geometry[140]	Method	Arxiv
LANDeRMT: Dectecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation [189]	Method	ACL
Task-Specific Skill Localization in Fine-tuned Language Models [129]	Method	ICML
Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation [58]	Application	AACL
Cross-Lingual Generalization and Compression [133]	Application	ACL
How Syntax Specialization Emerges in Language Models [36]	Application	Arxiv
Language Lives in Sparse Dimensions: Interpretable Multilingual Control [186]	Application	Arxiv
Inducing Dyslexia in Vision Language Models [70]	Application	Arxiv
Different types of syntactic agreement recruit the same units [89]	Application	Arxiv
Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models [59]	Application	Arxiv
Multilingual Knowledge Editing with Language-Agnostic Factual Neurons [181]	Application	COLING
From Language to Cognition: How LLMs Outgrow the Human Language Network [3]	Application	EMNLP
The Transfer Neurons Hypothesis [156]	Application	EMNLP
Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [123]	Application	EMNLP
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model [22]	Application	ICLR
The LLM Language Network: A Neuroscientific Approach [2]	Application	NAACL
Language-Specific Neurons Do Not Facilitate Cross-Lingual Transfer [114]	Application	Workshop
Language-Specific Neurons: The Key to Multilingual Capabilities [154]	Application	ACL
Unveiling Linguistic Regions in Large Language Models [182]	Application	ACL
Converging to a Lingua Franca: Evolution of Linguistic Regions [175]	Application	COLING
Unveiling Language Competence Neurons: A Psycholinguistic Approach [37]	Application	COLING
Linguistic Minimal Pairs Elicit Linguistic Similarity [188]	Application	COLING
Neuron-Level Knowledge Attribution in Large Language Models [174]	Application	EMNLP
Neuron Specialization: Leveraging Intrinsic Task Modularity for Multilingual Machine Translation [153]	Application	EMNLP
Decoding Probing: Revealing Internal Linguistic Structures [65]	Application	LREC
On the Multilingual Ability: Finding and Controlling Language-Specific Neurons [88]	Application	NAACL
How do Large Language Models Handle Multilingualism? [184]	Application	NIPS
Universal Neurons in GPT2 Language Models [60]	Application	TMLR
Same Neurons, Different Languages [148]	Application	NAACL
Importance-based Neuron Allocation for Multilingual Neural Machine Translation [173]	Application	ACL
Circuit Discovery
Circuit Tracing: Revealing Computational Graphs in Language Models [5]	Method	Anthropic
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs [107]	Method	ICLR
Scaling Sparse Feature Circuits For Studying In-Context Learning [86]	Method	ICML
Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT [66]	Method	Arxiv
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small [161]	Method	ICLR
Towards Automated Circuit Discovery for Mechanistic Interpretability [32]	Method	NIPS
Between Circuits and Chomsky: Pre-pretraining on Formal Languages [74]	Application	ACL
The Same But Different: Structural Similarities in Multilingual LM [180]	Application	ICLR
Circuit Component Reuse Across Tasks in Transformer Language Models [112]	Application	ICLR

Table 4. Representative papers in MI. We categorize works into Method (foundational techniques, tools, frameworks, or benchmarks) and Application (studies leveraging these methods to investigate specific linguistic capabilities). The table spans five core domains: Probing, Vocabulary Projection (Logit Lens), Activation Patching, Sparse Autoencoders (SAE), and Circuit Discovery.

Paper Title	Type	Venue
Benchmarks
MIB: A Mechanistic Interpretability Benchmark [117]	Method	ICML
AXBENCH: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders [169]	Method	ICML
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders [84]	Method	ICML
InterpBench: Semi-Synthetic Transformers for Evaluating MI Techniques [56]	Method	NIPS
Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Senses [18]	Method	NAACL
CausalGym: Benchmarking causal interpretability methods on linguistic tasks [7]	Method	ACL
Holmes: A Benchmark to Assess the Linguistic Competence of Language Models [160]	Method	TACL
SyntaxGym: An Online Platform for Targeted Evaluation of Language Models [49]	Method	ACL
Probing
Lexical Popularity: Quantifying the Impact of Pre-training for LLM Performance [136]	Method	Arxiv
Steering Embedding Models with Geometric Rotation: Mapping Semantic Relationships Across Languages and Models [46]	Method	Arxiv
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks [126]	Application	Arxiv
ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework [178]	Method	ACL
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models [162]	Application	ACL
Probing Syntax in Large Language Models: Successes and Remaining Challenges [34]	Application	COLM
Probing Internal Representations of Multi-Word Verbs in Large Language Models [87]	Application	MWE
Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? [24]	Method	ACL
Finding Neurons in a Haystack: Case Studies with Sparse Probing [61]	Application	Arxiv
Probing Classifiers: Promises, Shortcomings, and Advances [13]	Method	CL
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT [120]	Application	EACL
Finding Universal Grammatical Relations in Multilingual BERT [23]	Application	ACL
On the Language Neutrality of Pre-trained Multilingual Representations [95]	Application	EMNLP
Emergent Linguistic Structure in Artificial Neural Networks Trained by Self-supervision [106]	Application	PNAS
A Structural Probe for Finding Syntax in Word Representations [69]	Application	NAACL
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information [51]	Application	BlackboxNLP
Vocabulary Projection (Logit Lens)
Eliciting Latent Predictions from Transformers with the Tuned Lens [14]	Method	Arxiv
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State [128]	Method	CoNLL
Interpreting GPT: the logit lens [124]	Method	Blog
Unraveling Syntax: How Language Models Learn Context-Free Grammars [138]	Application	Arxiv
The Semantic Hub Hypothesis: Language Models Share Semantic Representations [171]	Application	ICLR
Do Multilingual LLMs Think in English? [139]	Application	Workshop
Do Llamas Work in English? On the Latent Language of Multilingual Transformers [166]	Application	ACL
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models [31]	Application	ICLR
Sparse AutoEncoders (SAE)
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing [82]	Method	ICML
Binary Autoencoder for Mechanistic Interpretability of Large Language Models [26]	Method	Arxiv
LinguaLens: Towards Interpreting Linguistic Mechanisms via Sparse Auto-Encoder [79]	Method	EMNLP
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models [67]	Method	Arxiv
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima [155]	Method	Arxiv
The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans [25]	Application	Arxiv
From Syntax to Emotion: A Mechanistic Analysis of Emotion Inference in LLMs [142]	Application	Arxiv
Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMs [80]	Application	Arxiv
What’s the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering [104]	Application	ICLR
Crosscoding Through Time: Tracking Emergence of Linguistic Representations [10]	Application	ICML
Large Language Models Share Representations of Latent Grammatical Concepts [17]	Application	NAACL
Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models [62]	Application	NAACL
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages [6]	Application	Arxiv
Unveiling Language-Specific Features via Sparse Autoencoders [33]	Application	ACL
Analyzing Multilingualism in Large Language Models with Sparse Autoencoders [27]	Application	COLM
Sparse Autoencoders Can Capture Language-Specific Concepts [6]	Application	Arxiv
Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders [64]	Application	Arxiv
Causal Language Control in Multilingual Transformers via Sparse Feature Steering [30]	Application	Arxiv
Semantic Convergence: Investigating Shared Representations Across Scaled LLMs [135]	Application	SRW
Extended Abstract for “Linguistic Universals”: Emergent Shared Features in Independent Monolingual Language Models via Sparse Autoencoders [187]	Application	MRL
Sparse Autoencoders Find Highly Interpretable Features in Language Models [76]	Method	ICLR
Transcoders Find Interpretable LLM Feature Circuits [38]	Method	NIPS
Activation Patching
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context [57]	Method	Arxiv
How to use and interpret activation patching [68]	Method	Arxiv
Is This the Subspace You Are Looking for? An Interpretability Illusion [105]	Method	ICLR
Towards Best Practices of Activation Patching in Language Models [177]	Method	ICLR
Function Vectors in Large Language Models [157]	Method	ICLR
Attribution Patching Outperforms Automated Circuit Discovery [151]	Method	Blackbox
Attribution Patching: Activation Patching At Industrial Scale [121]	Method	BLOG
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models [45]	Method	IJCNLP
The Dual-Route Model of Induction [44]	Application	COLM
Neuron Analysis
From Directions to Regions: Decomposing Activations in Language Models via Local Geometry[140]	Method	Arxiv
LANDeRMT: Dectecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation [189]	Method	ACL
Task-Specific Skill Localization in Fine-tuned Language Models [129]	Method	ICML
Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation [58]	Application	AACL
Cross-Lingual Generalization and Compression [133]	Application	ACL
How Syntax Specialization Emerges in Language Models [36]	Application	Arxiv
Language Lives in Sparse Dimensions: Interpretable Multilingual Control [186]	Application	Arxiv
Inducing Dyslexia in Vision Language Models [70]	Application	Arxiv
Different types of syntactic agreement recruit the same units [89]	Application	Arxiv
Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models [59]	Application	Arxiv
Multilingual Knowledge Editing with Language-Agnostic Factual Neurons [181]	Application	COLING
From Language to Cognition: How LLMs Outgrow the Human Language Network [3]	Application	EMNLP
The Transfer Neurons Hypothesis [156]	Application	EMNLP
Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [123]	Application	EMNLP
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model [22]	Application	ICLR
The LLM Language Network: A Neuroscientific Approach [2]	Application	NAACL
Language-Specific Neurons Do Not Facilitate Cross-Lingual Transfer [114]	Application	Workshop
Language-Specific Neurons: The Key to Multilingual Capabilities [154]	Application	ACL
Unveiling Linguistic Regions in Large Language Models [182]	Application	ACL
Converging to a Lingua Franca: Evolution of Linguistic Regions [175]	Application	COLING
Unveiling Language Competence Neurons: A Psycholinguistic Approach [37]	Application	COLING
Linguistic Minimal Pairs Elicit Linguistic Similarity [188]	Application	COLING
Neuron-Level Knowledge Attribution in Large Language Models [174]	Application	EMNLP
Neuron Specialization: Leveraging Intrinsic Task Modularity for Multilingual Machine Translation [153]	Application	EMNLP
Decoding Probing: Revealing Internal Linguistic Structures [65]	Application	LREC
On the Multilingual Ability: Finding and Controlling Language-Specific Neurons [88]	Application	NAACL
How do Large Language Models Handle Multilingualism? [184]	Application	NIPS
Universal Neurons in GPT2 Language Models [60]	Application	TMLR
Same Neurons, Different Languages [148]	Application	NAACL
Importance-based Neuron Allocation for Multilingual Neural Machine Translation [173]	Application	ACL
Circuit Discovery
Circuit Tracing: Revealing Computational Graphs in Language Models [5]	Method	Anthropic
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs [107]	Method	ICLR
Scaling Sparse Feature Circuits For Studying In-Context Learning [86]	Method	ICML
Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT [66]	Method	Arxiv
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small [161]	Method	ICLR
Towards Automated Circuit Discovery for Mechanistic Interpretability [32]	Method	NIPS
Between Circuits and Chomsky: Pre-pretraining on Formal Languages [74]	Application	ACL
The Same But Different: Structural Similarities in Multilingual LM [180]	Application	ICLR
Circuit Component Reuse Across Tasks in Transformer Language Models [112]	Application	ICLR