Where Does Knowledge Live? Representational, Causal, and Editable Localization in Language Models: A Survey

1. Introduction

Neural networks have a large memory and learn a variety of skills, facts, and task-specific abilities. During the rise of large language models (LLMs), an old scholarly problem has resurfaced: where does that knowledge reside? The solution to this problem affects the understandability of knowledge in a model, the modification of particular associations, the elimination of private or dangerous material, and the understanding of the sources of model behavior.

The literature seems to provide contradictory answers. Some studies provide evidence of inside the model in which there is no clear separation between the factual slots. For example, according to Geva et al. [1], the sublayers of the feed-forward network (FFN) can be interpreted as key–value memories, and Geva et al. [2] decompose the FFN updates into parameter vector contributions that promote specific concepts. There are also studies that deal more directly with factual localization, such as analyzing the causes in the mid-layer of a Multilayer Perceptron (MLP) [3], and performing attribution with context-based interventions on specific ’knowledge neurons’ [4]. However, the localization claims that have been achieved do not necessarily provide grounds for stronger conclusions, as the tracing score might not necessarily reflect the best editing location [5], the neurons attributed might not be specific enough with regard to a certain fact [6], and the localization results [7] can differ depending on how the query is formed.

Evidence for the presence of a distributed architecture depends on the particular hypothesis in question: simple architectures produce feature superposition [8], sparse sampling suggests that there are varying amounts of sparseness in the representation of different features and layers [9], and causal analysis of factual recall shows the presence of redundant and non-contiguous pathways [10]. Learned descriptions can also show compact structure under various notions. Sparse autoencoders produce interpretable features with fine-grained intervention effects [11], while causal abstraction methods find compact causal variables in learned subspaces that may remain distributed in the original neuron coordinates [12].

We suggest that much of the disagreement comes from grouping several questions into one. The question, “Is knowledge localized?”, could be referring to at least three things:

• Representational localization: if in a given readout and representation system, knowledge can be reliably recovered from a restricted set of representational units;
• Causal localization: if a restricted set of units can be intervened on so as to bring about specific changes in the behavioral expression of that knowledge;
• Editable localization: if a restricted region of a model’s native parameters can be updated so as to bound the changes to knowledge.

All three notions rely on different types of evidence. Since there is no clear cross-validation of the three notions, evidence that supports one of the notions cannot be used to support another notion. The evidence is uneven across the three pairwise relationships, and localization conclusions depend on the choice of units and the representation system, knowledge target, and the measuring instrument. Section 3, Section 4 and Section 5 review each notion separately, while Section 6 synthesizes their relationships. Figure 1 summarizes the three notions.

Comparison to existing surveys.

Knowledge localization is typically treated as one component of broader surveys on adjacent topics, including knowledge editing [13,14], mechanistic interpretability [15], sparse representation analysis, machine unlearning [16,17], and training-data attribution. Most directly, Wang et al. [18] organize research on LLM knowledge mechanisms around knowledge utilization and evolution, including memorization, comprehension and application, and knowledge creation. Our survey focuses on a different dimension. Instead of considering how knowledge is utilized or evolves, we seek to identify the empirical statement that is made when a study makes the assertion that knowledge is localized. This includes what is localized, what defines localization, what units and what method of measurement are used, and when the data obtained under a particular notion is used to draw conclusions under a different notion. From this perspective, our taxonomy of representational (R), causal (C), and editable (E) localization classifies localization claims in terms of their logical evidence, rather than their knowledge mechanisms or their applications.

Contributions.

(i) We distinguish three notions of knowledge localization: representational, causal, and editable localization, together with a shared vocabulary for specifying units, representation systems, instruments, and budgets (Section 2). (ii) We classify the literature at the level of individual claims, organizing evidence by localization notion and related dimensions rather than forcing each study into a single localization camp. (Section 3, Section 4 and Section 5) (iii) We synthesize the evidence for agreement and dissociation among the three notions and document how uneven that evidence is across the three pairwise relationships (Section 6). (iv) We propose some open problems to make localization claims more comparable and to test unresolved relationships among the three notions (Section 8).

2. Preliminaries: Three Notions of Localization

2.1. Setup and Units of Localization

Let $f_{\theta }$ refer to a trained network whose parameters are denoted by $\theta$. A knowledge itemK denotes a fact, skill, or task capability defined in terms of the behavior used to evaluate it. Because localization can vary across queries expressing the same knowledge [7], we define K by an evaluation distribution ${\mathcal{D}}_K$ over queries or task instances that express the target behavior. If needed, matched controls or contrast items are included so that knowledge requires more than giving a constant response or superficial input cue. We summarize the model’s expression of K over ${\mathcal{D}}_K$ by a behavioral score $m_K\left(f\right)$.

For the purpose of representational and causal analyses, let $\mathcal{U}=\{u_1,\dots ,u_m\}$ denote the units of analysis. Those units may include native components (neurons, attention heads, MLP sublayers, or layers) or units under analysis (learned features, directions, or activation subspaces). When activations are redescribed, localization is also relative to a representation system, such as native neuron coordinates, a learned feature dictionary [11], or a subspace defined by the analysis. These systems do not need to be bases in the sense of linear algebra. Representation system and granularity are different concepts: changing coordinates or dictionaries changes how a representation is described, whereas shifting from neurons to heads or layers means changing the object that is localized.

We use k to denote the budget of localization, and the counting rule is provided explicitly. The budget could count the number of units, dimensions of a subspace, number of layers or parameter blocks, trainable coordinates, or update complexity. Such budgets based on different units or counting rules are not directly comparable. The question here is how the effect that is measured changes as the budget is changed.

2.2. The Three Notions

Representational.

We define representational localization as the situation in which information relevant to K can be reliably recovered from a restricted subset of representational units under a specified readout. With a constrained budget, evidence for representational localization comes up when a readout limited to a subset $S\subseteq \mathcal{U}$ reliably predicts the target associated with K on held out evaluation instances, compared to relevant controls.

Strong readout performance from a small S establishes sparse decodability. However, this does not imply that K is uniquely or mostly localized in S, because the same information may be available in other places as well. Representational localization is a stronger statement and requires showing that S is significantly more informative than its complement or alternative subsets of similar budget under the same readout method. This means that representational localization claims involve the evaluation distribution, readout, units of analysis, and in some cases, the representation system as well.

Causal.

Causal localization concerns whether interventions on a subset of units produce specified counterfactual changes in the behavioral expression of K. It indicates the causality of the units. When there is a given budget, it is determined by the change in $m_K\left(f\right)$ due to an intervention on a restricted set $S\subseteq \mathcal{U}$, without worsening the model itself.

The claim of causal localization should identify the type of intervention and state if the result is considered evidence for necessity, sufficiency, or both. Therefore, the interpretation of the claim depends on the type of intervention used and the method employed in assessing it, including corrupting, replacing, ablating, or controlling the model to create a counterfactual.

Editable.

Editable localization refers to whether the knowledge K can be changed by an update confined to a subset of the model’s native parameters, with a given budget. Editable localization does not require sparsity for the changes at the level of individual coordinates, as the budget may be defined over different units and counting rules.

To claim editable localization, the evidence needs to show that an allowed update $\Delta \theta$ can change K in accordance with criteria such as target efficacy, generalization, and locality or preservation [13]. Here, locality ensures unrelated facts remain unchanged, while whether logically and semantically entailed facts update consistently is measured by ripple-effect benchmarks [19].

A successful localized update can demonstrate that the behavior can be modified by the specified region, but it does not show that the knowledge is located in this region or that the region is unique or better than any other site for editing. To support a claim of editable localization for a particular region, it should be compared to other parameter regions within the same update class and under the same optimization procedure, budget, and evaluation criteria. Stronger evidence would be to demonstrate better editing performance for the region given the equivalent budget, or equivalently good editing performance by a smaller update. Note that interpreting an editable region as a representational or causal locus requires separate evidence.

We use the term editable localization for cases where updates are made to native parameters. Some editors that operate with external memory, additional modules, or in-context demonstrations may provide behavioral editability without identifying a native editable locus [20,21].

2.3. Localization as a Function of Budget

For all three notions, a localization claim is meaningful only in the context of a fixed budget and counting rule. That is, representational analyses look at readout performance, causal analyses look at intervention results, and editable analyses study editing quality with varying budgets. These profiles are not required to be monotonic, and profiles based on different notions or unit systems are not directly comparable. Therefore, a localization claim should also specify the knowledge target and distribution used for the evaluation, the localization notion itself, the units and representation system/parameter region, the budget and counting rule, and the measurement procedure. Figure 2 shows the main components.

The three notions may identify overlapping loci in practice, but whether they do so is an empirical question rather than something implied by their definitions. Training-data attribution asks a different “where” question: where the behavior comes from in the training data rather than where the relevant information is localized within the model. We discuss this provenance question separately in Section 7.

3. Representational Localization

The problem of representational localization is about how we know if a particular fact or skill can be generalized from just a few units. It depends on the way our model is represented: because of superposition, evidence based on single neurons is complicated, learned representation tends to be sparse, and paraphrasing the same fact could change the set of informative units [7]. This section first examines the evidence at a neural level and then reviews the recent Sparse autoencoder (SAE) and the limitations of probing and attribution.

3.1. Neuron-Level Encoding

Initial studies provided two different perspectives on feed-forward network (FFN) computation. According to the key–value interpretation set out by Geva et al. [1], the keys are related to input patterns that can be recognized, whereas values produce probability distributions over the output vocabulary. Geva et al. [2] build upon this perspective, considering the calculation of FFN updates through contributions of particular parameter vectors, and showing that many of these contributions support understandable concepts in vocabulary space.

In the knowledge-neuron literature, the unit of analysis is refined into a more granular representation. With the help of an integrated-gradients attribution technique, Dai et al. [4] find that neurons in the FFN of BERT [22] are activated in response to the relation. The inhibition and augmentation of the neurons cause changes in predictions, and the case studies are conducted with selective updating and deletion without complete fine-tuning. In the same way, for task skills, Wang et al. [23] rank FFN neurons according to their ability to predict the label of a task under prompt tuning. The perturbation of high-ranked neurons results in a more significant drop in performance in comparison to perturbation of random neurons, and similar tasks share more influential neurons. The importance remains stable for adapter tuning and BitFit. There is some support for the representation of semantically structured and sometimes compact neuron subsets. However, neither readout nor perturbation proves the uniqueness of storage in these neurons.

The concern with aggregate storage capacity is different. In controlled synthetic environments, Allen-Zhu and Li [24] find that about two bits of factual knowledge are associated with each parameter. These constraints do not specify how much of a particular fact is stored, and they do not indicate where each fact is located.

3.2. Superposition and Why Raw Neurons Mislead

When superposition is taken seriously, the picture at the neuron-level becomes more unstable. In fully characterized toy models, networks can encode more features than they have neurons by placing features along an overcomplete set of directions in activation space [8]. As a result, individual neurons become polysemous and a feature doesn’t have to be associated with one specific coordinate axis. It means that there is no way to draw a conclusion about “not sparse in neurons” meaning “not localized”. The distribution of activity in the original coordinates might be much more sparsely represented in another representation. Of course, these toy examples don’t show that there should necessarily be a preferred dictionary in pretrained language models or that this dictionary is unique; however, they indicate the usefulness of searching for a more suitable representation of units. Figure 3 illustrates this description dependence.

Findings within this field are highly variable. Using over one hundred features within seven different language models, k-sparse probing has shown the presence of sparse neuronal compositions and specialized single neurons [9]. These patterns vary by feature, layer, and model scale. Attention is another factor that creates another restriction to the explanation of FFN-based models. For single-layer transformer models trained on synthetic datasets through attention mechanisms only, Wong [25] finds a connection between tensor rank of the database and rank and capacity of the attention layer, noting an additive property for factual recall. This is an example of an existence-and-capacity result achieved in a very artificial architectural setup and does not describe how pretrained multilayer transformers partition factual knowledge between attention and MLP components.

3.3. Relocalization in Learned Dictionaries

When the problem originates from neuron coordinates, an option is coordinate adjustment. This is achieved using SAEs by using sparse feature direction compositions to replace raw activations. Initial studies showed promise in this method. Specifically, in a one-layer transformer, the learned features demonstrated much more monosemanticity compared to raw neurons; there were some features common to independently trained models, and effects from certain interventions matched up with expected interpretations [26]. Further investigation expands these findings to large language models and counterfactual behavior, revealing that SAE features relate to the Indirect Object Identification task [11] and a million-scale dictionary, trained on Claude 3 Sonnet, provides evidence for both interpretation and feature steering [27]. Moreover, sparse feature graphs can be used in intervention-friendly circuits [28]. However, these results do not imply that a learned dictionary is the only possible way to decompose a model. The feature set of the Claude dictionary is described as incomplete [27], and feature circuits refer to representational and causal structure instead of general parameter-level editing [28].

Cross-layer transcoders (CLTs) build on the basic principle beyond reconstruction within each individual layer. By training features which take input from the residual stream at one layer and then contribute to later MLP outputs in subsequent layers, CLTs make it possible to build attribution graphs across layers that are specific to the prompt used [29]. While these graphs may be tested via intervention, the approach itself requires replacing a part of the network with its approximation. Therefore, the resulting graph is a mechanism hypothesis.

Modification of the dictionary leads to modification of what is considered to be localized. Gated SAEs [30] separate the problem of selecting features from the task of determining their magnitudes and thus reduce the shrinkage caused by the $L_1$ norm and reach a better balance between sparsity and reconstruction. Matryoshka SAEs [31] learn nested dictionaries of different sizes, such that smaller dictionaries identify more general features while larger dictionaries identify more specific features, thus reducing feature absorption.

The choice of geometry and readout also changes the object being localized. For instance, as shown by Park et al. [32], directions and similarities have a representation-specific geometry. In contrast, different output orientations or behavioral directions are introduced by the Jacobian lens [33], the tuned lens [34], and representation engineering [35].

3.4. The SAE-Skeptic Wave

When localization is allowed to use learned dictionaries, a more difficult question arises: how can we trust the dictionary? Recent studies addressed this problem from several directions, including validity of metrics, choice of sparsity, generalization, and identifiability.

One of the challenges with this area is that of validation. For synthetic environments with well-known properties and for actual activations of the models, constrained random baselines are capable of performing as well as trained SAEs for a number of interpretability, sparse probing, and causal editing benchmarks [36]. It is possible to achieve robust reconstruction while still insufficiently reconstructing synthetic ground-truth features. A high score on a traditional SAE benchmark does not prove anything.

The second challenge is related to the possibility of an emphasis bias in favor of the training target by the dictionary. Alteration in the sparsity of the target $L_0$ may result in mixing of correlated features at lower sparsity and may lead to degenerate solutions at higher sparsity [37]. The resultant units depend on the choice of the sparsity level. Peng et al. [38] suggest that SAEs might prove useful in discovering unknown concepts as opposed to dealing with known concepts by the analyst.

Generalization is the topic of the third challenge. Even with successful decompositions on the in-distribution data in superposition experiments with compositional distribution shifts, the SAE representations will continue to underperform due to the same problem [39]. The problem identified with regards to SAE representations’ generalization failure in such experiments is that the learned dictionary directions do not match the synthetic ground truth.

This theory explains the difference between recovery that is possible and guaranteed. Superposition models are identifiable only up to certain transformations and require additional structural assumptions [40], but within the additive ridge-function model, the feature directions can be recovered even by using noisy queries in the overcomplete setting [41]. Although this gives the conditions in which the recovery can take place, it does not certify a specific SAE trained on the pretrained language model.

3.5. Probing and Attribution

Prudence must be exercised in conducting dictionary learning. The reliability of claims about representations will depend on the methods used for their analysis. It is especially clear when conducting probing where the results that could be recovered from an activation are not always the information that the model actually uses [42]. If it is sparsely represented, then it is just evidence that the information could be used but is not necessarily used. Perturbation analysis such as the one in the skill-neuron paradigm Wang et al. [23] helps strengthen the case, but it does not solve the problem of redundancy or representation choice.

The examples obtained from knowledge-neuron analyses make the problem more tangible. Reassessing the original hypothesis based on BERT, GPT-2, and LLaMA-2, Niu et al. [6] observe that an identical attribution technique can be used to pinpoint neurons associated with grammatical patterns. The modifications usually affect only the probability of the prediction while keeping the categorical prediction intact. In another test, Rank-One Model Editing (ROME) [6] edits fail to generalize in symmetry and synonymy settings.

4. Causal Localization

Evidence is said to be representational when it indicates the location of information. Causal localization focuses on which part of the model makes a difference when that information is used. It can be more than one neuron or layer. Depending on the behavior as well as the intervention, the causal structure has been described as a high-effect component, a multi-stage circuit, a task-conditioned vector, or a low-dimensional subspace. This section takes up these different loci of causality. It starts with a discussion of factual recall mechanisms (Section 4.1) and circuit analysis (Section 4.2), then continues with an analysis of task vectors and function vectors (Section 4.3) as well as variables identified in learned representations (Section 4.4). Conclusions drawn throughout the section depend on the type of intervention at hand: statements of necessity, sufficiency and mediation are always made conditional upon a certain counterfactual. The last subsection concludes by looking into how patching design, approximations and benchmarks affect the inference of the locus of causality (Section 4.5).

4.1. Factual Recall Mechanisms

The standard starting point is a localized causal effect. Using causal tracing, Meng et al. [3] taint the representation of the subject token and then selectively reconstruct the clean hidden states. Most pronounced are the effects on MLP computations for intermediate layers close to the last subject token. A rank-one update to the MLP’s weight matrix transforms this kind of localization into ROME. Although these two works provide different arguments, one asserts that restoration of causal evidence is achieved and the other that the weight modification achieves editability.

Others consider the retrieval of information through the single-peak process in terms of separate stages. According to one of the interpretations [43], MLPs at the early stage enhance the representation of the subject, followed by propagation of relational information and extraction of the needed attribute by attention heads. On the other hand, Lv et al. [44] argue that it is the task-oriented attention heads that convey topic information, while MLPs increase, reduce, and finally redirect the residual stream to the answer. Both cases consider the process of factual retrieval through the coordination of attention heads and MLPs and not through independent computations of a single component. However, a multi-stage process is not sufficient even in this setting. According to Chughtai et al. [45], separating the contribution of different heads and MLPs, one can see that multiple qualitatively distinct mechanisms are necessary for retrieval of the correct attribute.

4.2. Circuit Analysis and Discovery

A circuit-level claim extends the scope of localization to subgraphs, where the computation is what gives rise to the observed behavior. Wang et al. [46] analyze indirect object identification (IOI) using this method and explain that, of the 26 attention heads, they identified seven types of functionality, and then investigated the circuit for faithfulness, completeness and optimality. However, incomplete localization and the existence of backup name-mover heads reveal that there is still redundancy in the computation.

More general behaviors tend to blur circuit boundaries. Although induction heads offer strong causal evidence for repeated-pattern completion in small attention-only transformers [47], their role in accounting for most in-context learning in larger models is treated with more caution. Mechanistic completeness does not imply architectural sparsity. In grokked modular addition [48], the recovered Fourier algorithm spans embeddings, attention, MLPs, and the unembedding while concentrating computation in a small set of Fourier frequencies. In this case, the mechanism is compact in one description while it may not be concise in the model’s component graph.

With increasing complexity of circuit analysis, the challenge for reverse engineering becomes increasingly difficult. Instead of changing the causality question, the automation techniques solve the search problem. ACDC [49] discovers behaviourally meaningful subgraphs through iterative patching and pruning of edges of the computational graph. The Edge Attribution Patching (EAP) [50] replaces the iterative patching procedure with an approximation technique, thus significantly lowering the cost of scoring edges at the edge level. In a number of benchmark studies, EAP was able to recover the known circuits competitively, yet the degree of correlation with explicit patching varies with the task and metric used. The described techniques improve the scalability of the circuit discovery process, yet the need for definition persists. The question of whether the circuits were recovered depends on the dataset, metric, component specification, clean-corrupted difference, and pruning rule.

4.3. Task and Function Vectors

Causal localization does not require specification as a set of architectural components. In in-context learning, the relevant object could alternatively be a task-conditioned activation. In the case of relatively simple token-based tasks analyzed by Hendel et al. [51], a single hidden state “task vector” suffices to capture most of the effects brought about by a demonstrated set. Incorporation of that vector in a query-only inference process reconstructs a large part of the in-context performance. This experiment reveals the causally effective task representation without revealing how it is created or what elements form it. Function vectors provide a component-level account of this concept. Using the approach of causal mediation analysis, Todd et al. [52] find a small number of attention heads whose task-conditioned activations combine into a function vector. Injection of this vector into intermediate layers makes the demonstrated function possible in zero-shot and text-based settings. In both experiments, conciseness is achieved in terms of intermediate activation and not a sparse set of neurons.

4.4. Causal Structure in Learned Bases

Causal-abstraction approaches displayed by Distributed Alignment Search (DAS) do not require an initially specified high-level variable to be aligned with specific neurons. DAS introduces a rotation that maps the variable. Alignment is then verified by intervention exchanges [12]. For the equality and natural language inference problems studied, the causal variables may be short in the learned subspace. However, these variables may be dispersed in native neuron space. The version of DAS referred to as Boundless allows one to no longer specify the subspace dimension and scale to the Alpaca 7B model [53]. Even in the simple price tagging problem, the system provides a brief causal explanation using two Boolean variables, where both are distributed in the subspace of the residual stream.

Concurrently, there is another development where units shift from being native components to learned features. These SAE features could be integrated into sparse circuits, where selected interactions among them become subjected to intervention testing [28]. Cross-layer transcoders expand this framework to construct prompt-specific attribution graphs on learned features [29]. Using these graphs with Claude 3.5 Haiku, one obtains conjectures about multi-step reasoning, multilingualism, and planning [54]. An important consideration relative to the observed structure is that the transcoder graph represents a substituted network and not the original network, and mechanistic understanding was limited to a few of the prompts investigated. Although the chosen hypotheses could be confirmed through intervention testing, the graphs shall be considered as partial mechanistic models and not fully causal models.

4.5. Methodological Limits and Evaluation

However, all of the results mentioned previously rely on the design of the counterfactual intervention. This is not a trivial element of the procedure, since changing how the corruption is introduced, how it is evaluated, and how the patch is applied may have considerable influence on the produced localization map [55]. In both factual recall and circuit discovery scenarios, such changes may impact the apparent area of causal relevance. This constitutes a methodological dependence issue, but this does not mean that every localization map is an artifact. Instrumental approximation is another source of uncertainty. For instance, EAP makes circuit search more computationally feasible but its first-order scores might be different from those obtained using explicit patching in the case of nonlinear response to the intervention [50]. Hence, approximation does not mitigate the need for using clean-corruption difference, graph dissection, metric selection, and the accuracy of the approximation.

Things get further complicated if the success of the interventions is due to some other reason. According to the findings provided by Makelov et al. [56], a learned subspace direction could influence behavior by triggering a latent route via a feature causally unrelated to the rest, instead of singling out a variable that is used in the usual functioning of the system. This does not invalidate the approach of intervention within a subspace, but suggests that success alone is not enough for a faithful localization, unless there is proof of the generalization outside of contrasts.

Benchmarks used by Chang et al. [57] for evaluating the localization techniques complement each other. The INJ Benchmark measures the ability of a technique to infer parameters when information injection occurs, while the DEL Benchmark checks the effect of selective deactivation of identified neurons on natural memory sequences. While in most cases these two benchmarks result in the same ranking of methods, they show poor sequence specificity. To address the above issues, benchmarking was introduced to compare different causal localization methods. The Mechanistic Interpretability Benchmark [58] defines different tracks and metrics for circuit localization and causal variable localization, and BlackboxNLP 2025 Shared Task on circuit and causal-variable localization provides a common setting for comparing mechanistic interpretability methods [59].

5. Editable Localization

The key challenge with editable localization is the degree to which native parameters can be edited to achieve the desired target behavior without changing the out-of-scope behavior. In order to get a limited kind of edit that allows for practical parameter-level control, the assertion on some region must be made in comparison with other regions. None of these individual findings shows any representational or causal basis for itself. Yao et al. [13] present the concepts of reliability, generalization, and locality, while Zhang et al. [14] develop the editing taxonomy and KnowEdit.

5.1. Locate Then Edit

Methods using a locate-then-edit approach implement the assumption that localization enables the editing task. These methods start with localization based on either causal tracing or other sources of localization signals and then proceed with an edit in the localized area. ROME [3] models the down projection of the MLP in the mid-layer as a linear associative memory and makes an additional association through a closed-form rank-one update of a layer selected causally. MEMIT [60] extends the associative memory approach to thousands of facts by structuring the updates in many mid-layer MLPs. It is shown that the small updates of the MLP regions lead to successful edits.

Further methods involve changing the update function, the preservation condition, or the granularity of the edit process but keeping the basic notion of having a constrained edit point. In PMET [61], optimization is done in terms of the intermediate states of both the attention and feedforward modules where parameter updates are made in terms of FFN weights. In AlphaEdit [62], the new update function is projected into a null space designed to ensure that any prior knowledge is not destroyed in the edit sequence. At a higher granularity level, Pan et al. [63] identify the selected FFN neurons as effective targets for editing, thereby improving the specificity as compared to editing at the layer level.

5.2. Hypernetwork and Meta-Learned Editors

Another way is learning the update directly. KnowledgeEditor [64] learns an edited gradient-based parameter update constrained on that parameter, while MEND [65] uses auxiliary networks to update the gradients produced through low-rank fine-tuning. While both these approaches are still native parameter editors, the locality they provide comes from the update process or the training loss.

5.3. Editing Beyond Native Weight Updates

However, the fact is not true for all successful interventions since some do not modify the initial parameters responsible for generating the output. In one such approach called SERAC [20], the relevant inputs are conveyed to both an explicit edit memory and a counterfactual model without modifying the preexisting network in any way. In GRACE [21], the sequential edits are stored using a separate key-value adapter, and in IKE [66], in-context demonstrations are used without any parameter modification.

Architectural augmentation is a middle-ground example. While T-Patcher [67] incorporates and trains only a small number of new neurons in the last feed-forward layer of the network, WISE [68] routes the edits to a separate memory space. Following the terminology introduced in Section 2, such approaches have behavioral editability while failing to find a native editable locus on their own.

Wang and Veitch [69] provide an alternative viewpoint. Since their methods change the internal representations, but not the initial parameters, such methods do not belong to the previously described category of editable localization. However, methods aimed at attention heads selected randomly yield similar performance to localization-based optimization. This result suggests that the argument based on evidence obtained through localization alone is unconvincing.

5.4. Benchmarks and Failure Modes

Inconsistencies resulting from editing errors have a set of evaluation criteria attached to them. As noted in RippleEdits [19], for instance, a target success may also lead to inconsistency of related facts, while MQuAKE [70] detects situations where edits fail to propagate throughout multi-hop reasoning. Further studies report collateral effects on related information and general capabilities of models [71,72], as well as interference or forgetting as a result of multiple editing iterations [73]. The above results imply that edit success should be evaluated on multiple dimensions. Therefore, efficacy, generalization, relational consistency, preservation, and robustness to multiple editing should be measured independently.

6. The Dissociations

Section 3, Section 4 and Section 5 show the distinctness of representational, causal, and editable localization from one another as separate evidential claims. We use dissociation to describe the difference between the three instead of statistical independence. A study can be relevant to several notions but only supports the particular claim measured. Multi-notion evidence refers to the inclusion of experiments relating to more than one notion within a particular study, while direct cross-notion evidence involves comparisons of their loci, scores or rankings. Comparisons must be made using descriptions that are aligned to each other, either through the use of candidate region indexes or explicit mappings of notion-related units.

Our claim does not imply that representational, causal, and editable localizations are independent. In previous work, neither have all three localization notions been measured concurrently and rigorously, nor have the differences between them been demonstrated. While there are many studies that support at least one notion, the directly comparative evidence is scarce, especially when compared to three possible pair combinations. Most evident is the case of causal and editable localization, where there is considerable direct evidence [3,5,74], while there is less convincing evidence for inequality between representational and causal localizations and even less evidence for the relationship between representational and editable localization. Figure 4 represents this uneven evidence.

6.1. Causal vs. Editable

Hase et al. [5] compare the correlation between localization scores computed through Causal Tracing and the editing efficiency achieved with the same MLP layers. As far as the standard factual overwriting is concerned, the localization score cannot be considered as any hint towards the most suitable layer for editing, whereas the information about the layer itself plays the role of a significantly better predictor of the editing quality. Even in the case of Fact Forcing editing that was designed specifically to match the localization manipulation, the connection with the localization score becomes much less influential than the choice of the layer.

This result calls into question approaches like ROME [3] that rely on locate-and-edit: the causal importance of the activation would be relevant for determining the best place to edit weights, whereas editing the weights is not necessarily informative about the location of the primary causal locus. Editors based on external memory or additional components further elucidate the distinction between the two by succeeding in modifying behavior without an internal locus.

6.2. Representational vs. Causal

The ability to perform a readout shows that there is relevant information about K available to the model, but it does not show that the model uses the information to produce the behavior. This is one of the primary shortcomings of interpretability through probing [42]. Therefore, Sparse decodability should not be treated as evidence of causal necessity or sufficiency without an intervention.

This difference has been made manifest by recent studies in which matching is performed between the two aspects under discussion. In particular, Sharma and Le [75] investigate both the representation of demographic information and its causal influence in the same experimental settings. By using the sparse autoencoder representations of demographic information along with the use of activation patching, feature steering, and ablation techniques for assessing behavioral influence, they find that demographic information is highly represented in the early layers, whereas measurable causal influence occurs mostly in the later layers. This provides converging evidence of both non-equivalence of representation and causation within the tested scenario.

This connection also depends on the representation. Causal variables have been found using DAS and Boundless DAS to be compactly encoded within the learned subspaces but remain distributed over the coordinate space of the original neurons [12,53]. Though this does not preclude the possibility of existence of a compact representation under some other encoding, it shows that localization of causation alone is not sufficient to ensure localization of representation.

6.3. Representational vs. Editable

The relationship between representational and editable localization is the least researched of the three pairs. It is not enough to find a limited number of units that are connected to some knowledge just to prove that these units create a generalizable locus for editing, and, on the other hand, editing the parameters does not mean that there was a concentration of knowledge in this area.

The research on knowledge neurons helps illustrate this gap. Chen et al. [7] provide an example where loci can be different in two expressions with the same fact, showing that loci identified in one expression do not have to be a consistent locus in paraphrases of the fact. In addition, Niu et al. [6] show that attribution allows identifying neurons associated with grammatical phenomena, and interventions often change probabilities but not the predicted categories.

The reverse inference also lacks empirical support. ROME [3] and MEMIT [60] demonstrate that limited parameter regions can enable efficient factual updates given certain editing goals, although the efficiency of editing does not directly measure the extent to which the relevant information has been localized in that way as discussed in Section 2. It follows that the existing findings provide grounds for non-substitution but not the separation of representation from editability. The relation between representation localization and efficient editability remains to be tested.

6.4. Sources of Dissociation

These disagreements did not arise randomly, but rather stemmed from three decisions: model description, knowledge target, and measurement instrument.

Model description.

Localization relies upon the units used and the representation system. Superposition and learned dictionaries can make the information spread out over native neurons much more sparse under a different framework [8,26,27], while DAS does the same by identifying causal variables that are sparse in the learned subspace [12,53]. Because learned sparseness is decomposition-dependent [36,37], any claims of dissociation between notions must be consistent in their units.

Knowledge target.

Localization also depends on the nature of the object that is being localized and how the elicitation was done. Paraphrases of a common fact may imply different units [7]. The acquisition of knowledge in the entity and in the relation may have a different pattern, and relational recall may involve the attention module as well [74]. And there may be different mechanisms contributing to factual recall rather than just one mechanism [45].

Measurement instrument.

The inferred position is also affected by the method of measurement itself. The effects of activation patching will be determined by the type of corruption, criteria, and patches being used [55], while subspace intervention may change behavior through the activation of pathways that are not used in the regular calculation [56]. Editable localization is similarly affected by the criterion used, which includes effectiveness, generalizability, persistence, and relation consistency [19]. These relations, however, do not make localization random, as any inconsistencies between notion, target, description, and procedure are empirical in nature.

7. Broader Implications and Extensions

These distinctions, outlined above, have implications far beyond the scope of localization itself. They affect how knowledge editing takes place, what constitutes removal in relation to unlearning, and the understanding of localization in new modes and structures. They also help clarify the distinction between internal localization and the source of the training data.

7.1. Model editing at scale

On a large scale, the problem becomes one of ensuring the compatibility of multiple updates without interfering with each other. MEMIT distributes structured updates to a small number of MLP layers affecting a large number of connections [60], while AlphaEdit applies new updates to a null space designed to protect existing knowledge [62]. As a result, the real challenge lies not in the search for the correct storage site but in the description of the parameter area and update geometry suitable for multiple updates.

7.2. Machine unlearning and safety

Unlearning aims at removing a particular target behavior while retaining other abilities [17]. The existing techniques often focus on optimizing forgetting rather than figuring out the underlying causality first. This ranges from techniques like forgetting using gradient ascent or fine-tuning [76,77], as well as RMU [78] where forgetting takes place in fine-tuning but no representational locus is found beforehand.

Further evaluation distinguishes unlearning from editing. TOFU [79] shows that the existing baselines cannot meet the more rigorous standard of exhibiting behavior as if the forget set was never learned, while locate-and-edit methods act as unlearning baselines only partly [80]. More directly, according to Lee et al. [81], limiting the update to the ground-truth region itself is not always effective in improving the forgetting-retention tradeoff. Thus, establishing an alternative connection, locating the parameter region, and eliminating the prior knowledge are different tasks.

7.3. Beyond dense text language models

There is recent research on the question of whether localization is maintained under change either in modality or architecture. Fact-sensitive layers have been isolated and used for guiding restricted editing in multi-modal LLMs [82]. Another parallel approach to studying localization in models can be found in audio-language models, where causal-tracing-style analysis finds components involved in facts and sound, and targeted edits can affect their behavior [83]. The SAKE method [84] expands upon editing studies in the direction of evaluating auditory attributes under multiple criteria.

Mixture-of-experts systems manipulate the candidate units themselves. In other words, the restriction of the task to neurons, attention heads, or dense layers is no longer necessary; localization can be defined over routed experts. Cross-lingual routing differences find a small number of experts associated with the facts that become less accurate at recalling when these experts are deactivated [85]. It is not clear whether the three types of localization always relate to each other similarly.

7.4. Provenance boundary

The problem of training data provenance refers to a different spatial query, which is concerned with the determination of the data points that constitute the basis of a particular behavior, rather than localization of the relevant information inside the neural network. There exist multiple techniques for following predictions back to their data sources, such as influence functions [86] and their extensions to bigger networks through Datamodels [87] and TRAK [88]. There exist memorization studies, which examine how each point is stored [89], and capacity studies, which evaluate the total capacity of the model to memorize information [24]. In any case, provenance is a topic, which falls next to, but not into the R/C/E taxonomy. However, such boundaries should not be too strict, since if attribution in data space reliably predicts localization, a solid empirical connection between “where from” and “where in” will be established.

8. Open Problems

We highlight seven open research problems in joint measurement, definition of localization, larger knowledge scales, provenance, safety, training dynamics, and validation of instruments.

Joint measurement of the three notions.

Evidence can often be pairwise when notions are compared in the context of cross-notions. For any fair comparison to be made, one needs to evaluate these three notions with respect to the same knowledge structures and models. The comparison should be made either using the same architectural coordinates or by finding mappings between representations related to the notion. The cost budget must match the notions or have the same cost measure in a meaningful way. In this way, direct assessment of similarity is possible at the item level.

A representation-robust theory of localization.

The discussion of localization currently depends on the chosen units and representation system. A deeper theory should describe when localization is stable across allowable descriptions instead of defining it with respect to a particular coordinate system or a learned dictionary. One possible path is to formulate it as an optimization problem over a description length, complexity, or rate-distortion criterion in the context of fidelity constraints for representing systems. Such an approach may relate to the geometry of superposition [8] and feature extraction theory [41] without assuming that the truth is contained in sparse representation.

Localization beyond item-level facts.

Factual knowledge provides a clear setting for comparing the three notions. However, we should also notice that many task- and skill-level studies suggest analyzing localization at broader behavioral scales, including task vectors, specialized skill parameters, modular components, and mixture-of-experts routing [85,90,91,92]. What still missing is a matched cross-notion test. For example, does a compactly representable task also have a compact causal locus, and is that locus particularly effective for modifying the task? Extending joint measurements to studies of skills and tasks would test whether the observed dissociations generalize beyond factual knowledge.

Does “where from” predict “where in”?

The questions of provenance and internal localization differ from each other (Section 7), and the empirical connection between them is still not known. An empirical study will be done by testing if there is a concentration of behaviors associated with a small number of training samples [87,88,93] using representation, causation, or editable localization. There would be a clear relation between data space attribution and localization if such a strong correlation exists; otherwise, there will be a distinction between them.

Localization-aware unlearning and safety.

Given the causal–editable dissociation, one can say that a causal region is not required to be optimal in order to unlearn the target task. In this way, any localization-aware unlearning algorithm must perform a direct comparison between candidate regions or representations of parameters within the relevant objective, which in turn includes the minimization of unintended interference. Previous studies noted that current unlearning techniques do not have a separate step for localization [77]. The benchmarks WMDP [78] and TOFU [79] still highlight the inherent forget-preserve dilemma.

When does localization emerge?

This study focuses on localization in trained models, but the relevant structure was created during training. Therefore, although the results of the capacity and memorization experiments [24,89] provide useful insights, they cannot be used to determine when localization occurs. Therefore, a separate analysis should ask whether localization becomes stronger, weaker, or merely changes description over the training process, and whether the three notions evolve at the same or different stages.

Calibrating localization instruments.

So far, there are no common calibration settings for localization tasks in cases where the mechanisms are well-known or partially known. Some benchmarks, such as the INJ benchmark by Chang et al. [57], can illustrate the use of partial ground truth, whereas synthetic and semi-synthetic tasks will allow us to find out whether the tools of localization can restore the correct structure before proving strong hypotheses concerning the pre-trained LLMs. Common evaluation methods, which are discussed by Arad et al. [59], can also be used for this purpose.

9. Conclusion

So where does knowledge live? The literature does not support a single answer, partly due to the fact that this question brings together three different attributes of a model. Representational, causal, and editable localization are three evidentially independent concepts. The asymmetry of the evidence between these three notions means that evidence for the claim of one notion cannot generally be considered as evidence for the other. More generally, what appears localized depends on how the model is described, what knowledge and queries are being studied, and how localization is measured. Acknowledging these distinctions helps resolve some of the apparent inconsistencies between localism, skepticism, and distribution. Additionally, it provides a clearer basis for cumulative work on knowledge localization.

Scope.

We compile literature that examines knowledge localization in neural networks, with particular emphasis on language models and closely related settings. Studies that provide evidence for representational, causal, or editable localization are labeled with an R/C/E suffix. We also review literature that is relevant to localization in cases of provenance, memorization, capacity, behavioral editing outside of original parameters, unlearning benchmarking, and model merging.

Claim-level coding.

Claims are coded rather than entire papers, so a single paper can have multiple codes depending on several different claims made in different experiments. Evidence from various notions of localization is included only in analyses when it is part of a comparison between two notions.

Corpus summary.

The corpus consists of 96 papers. Of those, 79 include at least one localization label, whereas the other 17 papers are classified as adjacent or boundary literature. For each paper, Table A1 includes the localization notion(s), the target or context that is analyzed, the parameter region or the analysis unit, the measuring tool, and the findings of the study. Findings are reported without labeling them as localized or distributed, since many findings depend on the particular notion of representation, intervention, criteria, or knowledge. The coding follows the distinctions in Section 2.

Table A1. Evidence map of the surveyed corpus. R, C, and E denote representational, causal, and editable localization; a dash denotes adjacent or boundary work. Multiple labels indicate evidence bearing on multiple notions, not necessarily a direct comparison between them. Target / unit reports the study target and analyzed model region.

Reference	Notion	Target / unit	Method / instrument	Key finding
Multi-notion evidence
Dai et al. [4]	R,C	factual knowledge / FFN neuron	attribution + activation intervention	Fact-associated FFN neurons affect factual predictions.
Meng et al. [3]	C,E	factual knowledge / MLP layer + weight matrix	causal tracing + parameter editing	Causal mid-layer MLPs also support successful rank-one edits.
Wang et al. [23]	R,C	task / skill / FFN neuron	sparse probing + perturbation	Sparse task neurons are functionally important.
Bricken et al. [26]	R,C	language-model features / learned feature	dictionary learning + feature intervention	Learned features are more monosemantic and interventionally active.
Hase et al. [5]	C,E	factual knowledge / MLP layer	causal tracing + parameter editing	Causal-Tracing scores weakly predict the best edit layer.
Hendel et al. [51]	R,C	in-context tasks / activation vector	task-vector extraction + activation patching	Compact task vectors restore corresponding task behavior.
Zou et al. [35]	R,C	high-level behavior / activation subspace	representation readout + activation steering	Activation directions support both readout and steering.
Huben et al. [11]	R,C	language-model features / IOI / SAE feature	SAE + activation intervention	SAE features support interpretable causal interventions.
Niu et al. [6]	R,C	factual knowledge / neuron	attribution + activation intervention	Knowledge-neuron attribution lacks fact specificity.
Tan et al. [94]	R,E	multilingual translation / FFN neuron	activation readout + sparse parameter update	Language-specific neurons enable sparse multilingual updates.
Templeton et al. [27]	R,C	language-model features / SAE feature	SAE + feature steering	Large SAE dictionaries yield interpretable steerable features.
Wei et al. [74]	C,E	factual knowledge / MLP + attention	causal analysis + parameter editing	Entity and relation knowledge show different causal/editing patterns.
Marks et al. [28]	R,C	language-model tasks / SAE feature + circuit	SAE + circuit intervention	SAE feature graphs support validated causal circuits.
Ameisen et al. [29]	R,C	language-model behavior / CLT feature + circuit	cross-layer transcoder + attribution graph	CLT features support testable attribution graphs.
Chen et al. [7]	R,E	factual knowledge / neuron	attribution + parameter editing	Query-consistent localization improves edit robustness.
Lim et al. [95]	R,C	multilingual reasoning / layer + activation space	representation readout + steering	Shared multilingual semantics are readable and steerable.
Lindsey et al. [54]	R,C	language-model behavior / learned feature + circuit	attribution graph + intervention	Attribution graphs yield partially validated mechanistic hypotheses.
Tezuka and Inoue [96]	R,C	multilingual reasoning / MLP neuron	neuron readout + intervention	Transfer neurons affect multilingual reasoning transitions.
Bandarkar et al. [85]	R,C	multilingual factual recall / MoE expert	expert readout + deactivation	Few fact-associated experts can be causally important.
Chung et al. [83]	C,E	audio factual knowledge / layer / module	causal tracing + parameter editing	Causal audio modules also support targeted edits.
Gu et al. [82]	R,E	multimodal factual knowledge / FFN layer + weights	readout + layer-specific parameter editing	Fact-critical multimodal layers guide restricted edits.
Korznikov et al. [36]	R,C	language-model features / SAE feature	baseline evaluation	Random baselines approach trained SAE localization metrics.
Michaud et al. [97]	C,E	task / skill / prunable components	ablation + pruning	Skills are causally nonlocal yet selectively removable.
Sharma and Le [75]	R,C	demographic information / SAE feature + layer	SAE feature analysis + causal intervention	Strong encoding and causal influence peak at different depths.
Representational localization
Geva et al. [1]	R	language-model computation / FFN vectors	key–value analysis	FFN keys/values align with patterns, not unique facts.
Belinkov [42]	R	general methodology / —	probing review	Decodability alone does not establish causal use.
Elhage et al. [8]	R	synthetic features / neuron + feature direction	toy-model analysis	Superposition makes neuron-level sparsity representation-dependent.
Geva et al. [2]	R	language-model computation / FFN vector	vocabulary projection	FFN sub-updates promote concepts without unique fact storage.
Belrose et al. [34]	R	language-model predictions / layer	tuned lens	Tuned lenses trace latent predictions across depth.
Gurnee et al. [9]	R	language-model features / neuron	sparse probing	Representational sparsity varies across features and layers.
Park et al. [32]	R	representation geometry / activation direction / subspace	theory	Representation geometry depends on the chosen metric.
Rajamanoharan et al. [30]	R	language-model features / SAE feature	SAE	Gated SAEs improve sparsity–reconstruction trade-offs.
Wendler et al. [98]	R	multilingual reasoning / layer + activation space	representation tracking	Shared multilingual concept space emerges in intermediate layers.
Bussmann et al. [31]	R	language-model features / SAE feature	SAE	Nested SAEs recover features at multiple granularities.
Klindt et al. [40]	R	representation theory / sparse code / subspace	theory	Sparse-feature recoverability is transformation-dependent.
Liu [41]	R	feature recovery theory / feature direction / subspace	theory	Overcomplete features are recoverable under noisy queries.
Wong [25]	R	synthetic factual knowledge / attention layer	rank / capacity analysis	Attention-layer rank bounds factual capacity.
Peng et al. [38]	R	language-model features / SAE feature	position paper	SAEs may suit concept discovery better than intervention.
Barin-Pacela et al. [39]	R	representation generalization / learned feature	generalization evaluation	Probe/SAE features can fail under compositional shift.
Chanin and Garriga-Alonso [37]	R	language-model features / SAE feature	SAE sensitivity analysis	Target sparsity strongly affects SAE feature quality.
Gurnee et al. [33]	R	language-model representations / J-space subframe	Jacobian lens	J-space contains a selectively active workspace-like subframe.
Causal localization
Olsson et al. [47]	C	in-context learning / attention head	ablation + circuit analysis	Induction heads causally support repeated-pattern completion.
Conmy et al. [49]	C	language-model circuits / circuit edge	causal circuit discovery	ACDC recovers circuits by pruning unnecessary edges.
Geva et al. [43]	C	factual recall / attention head + MLP	activation patching	Factual recall spans multiple attention/MLP stages.
Nanda et al. [48]	C	modular arithmetic / circuit	mechanistic reverse engineering	Grokking yields a compact multi-component algorithmic circuit.
Wang et al. [46]	C	indirect-object identification / attention-head circuit	activation patching	A sparse attention-head circuit supports IOI behavior.
Wu et al. [53]	C	simple reasoning / residual-stream subspace	interchange intervention	Compact causal variables occupy distributed residual subspaces.
Chughtai et al. [45]	C	factual recall / multiple components	activation patching + decomposition	Multiple mechanisms contribute additively to factual recall.
Geiger et al. [12]	C	causal abstractions / activation subspace	interchange intervention	Compact causal variables can occupy distributed subspaces.
Lv et al. [44]	C	factual recall / attention head + MLP	activation patching	Factual recall follows a multi-component causal pipeline.
Makelov et al. [56]	C	subspace interventions / activation subspace	subspace intervention	Subspace interventions may exploit dormant pathways.
Syed et al. [50]	C	language-model circuits / circuit edge	edge attribution patching	Attribution-patching scores can diverge from explicit patching.
Todd et al. [52]	C	task functions / attention head + activation vector	activation patching + addition	Function vectors can causally induce task behavior.
Zhang and Nanda [55]	C	language-model behavior / layer / component	patching-protocol evaluation	Patching localization is sensitive to protocol choices.
Chang et al. [57]	C	memorized sequences / FFN neurons	injection benchmark + neuron dropout	Direct benchmarks support localization ability but reveal limited sequence specificity.
Arad et al. [59]	C	localization methods / circuit / subspace	benchmark	Shared task benchmarks causal-localization methods.
Mueller et al. [58]	C	localization methods / circuit / subspace	benchmark	Standardizes circuit and causal-variable localization metrics.
Hochman et al. [10]	C	factual recall / layers / pathways	iterative activation patching	Factual retrieval can use redundant causal paths.
Editable localization
De Cao et al. [64]	E	factual knowledge / parameter update	hypernetwork editing	Hypernetworks produce constrained edits without pre-localized loci.
Mitchell et al. [65]	E	factual knowledge / low-rank parameter update	meta-learned editing	Meta-learned low-rank updates enable scalable editing.
Hoelscher-Obermaier et al. [71]	E	factual knowledge / parameter subset	specificity benchmark	Strict specificity metrics reveal collateral edit effects.
Ilharco et al. [90]	E	task behavior / weight-space direction	task arithmetic	Task directions support compositional weight-space edits.
Meng et al. [60]	E	factual knowledge / MLP weight matrices	parameter editing	Restricted MLP updates can edit thousands of facts.
Ortiz-Jimenez et al. [99]	E	task behavior / weight-space direction	tangent-space task arithmetic	Tangent task directions improve editing disentanglement.
Panigrahi et al. [91]	E	task / skill / parameter subset	parameter grafting	Tiny parameter grafts recover most task performance.
Yadav et al. [100]	E	model merging / sparse parameter deltas	task-delta merging	Sparse task deltas reduce interference in merging.
Zhong et al. [70]	E	factual knowledge / parameter subset	MQuAKE benchmark	Successful edits can fail multi-hop consequences.
Cohen et al. [19]	E	factual knowledge / parameter subset	benchmark	Target efficacy does not guarantee relational consistency.
Gu et al. [72]	E	language-model capabilities / parameter subset	side-effect evaluation	Targeted edits can degrade broad capabilities.
Gupta et al. [73]	E	sequential editing / parameter subset	sequential-editing evaluation	Repeated edits can accumulate catastrophic interference.
Li et al. [61]	E	factual knowledge / FFN parameters	parameter editing	Restricted FFN updates support precise factual editing.
Li et al. [78]	E	hazardous knowledge / selected layers / parameters	WMDP benchmark + RMU	RMU suppresses hazardous knowledge via selected regions.
Yu et al. [101]	E	model merging / sparse parameter deltas	task-delta sparsification	Sparse task deltas preserve abilities during merging.
Zhang et al. [14]	E	model editing / —	KnowEdit benchmark	KnowEdit unifies evaluation across editing criteria.
Lee et al. [81]	E	target knowledge / parameter regions	localized unlearning	Effective unlearning is not uniquely determined by the localized parameter region.
Fang et al. [62]	E	factual knowledge / MLP weights	parameter editing	Null-space projection improves repeated-edit preservation.
Pan et al. [63]	E	factual knowledge / FFN neuron parameters	neuron-level parameter editing	Neuron-level editing improves locality over coarser edits.
Yang et al. [84]	E	auditory attributes / various parameter regions	SAKE benchmark	SAKE evaluates auditory editing across multiple criteria.
Li et al. [80]	E	unlearning / editing / parameter subset	editing-as-unlearning evaluation	Editing is only a partial substitute for unlearning.
Adjacent and boundary literature
Koh and Liang [86]	—	training-data provenance / training examples	influence functions	Influence functions trace behavior to training examples.
Ilyas et al. [87]	—	training-data provenance / training examples	datamodels	Datamodels attribute outputs to training-set composition.
Matena and Raffel [102]	—	model merging / full-model weights	Fisher-weighted averaging	Fisher averaging merges capabilities without localized edit sites.
Mitchell et al. [20]	—	behavioral editing / external memory + counterfactual model	memory-based editing	SERAC edits via external memory, freezing base weights.
Wortsman et al. [103]	—	model merging / full-model weights	weight averaging	Weight averaging merges capabilities without internal loci.
Carlini et al. [89]	—	memorization / training examples	memorization measurement	Memorization measures retention without locating storage.
Eldan and Russinovich [77]	—	unlearning / model weights	fine-tuning unlearning	Fine-tuning suppresses knowledge without localization.
Grosse et al. [93]	—	training-data provenance / training examples	influence functions	LLM influence attribution links behavior to training examples.
Hartvigsen et al. [21]	—	behavioral editing / added key–value adaptor	adaptor-based editing	GRACE stores edits in an added adaptor.
Huang et al. [67]	—	behavioral editing / added FFN neurons	architecture augmentation	T-Patcher edits via newly added FFN neurons.
Jang et al. [76]	—	unlearning / model weights	gradient-ascent unlearning	Gradient-ascent unlearning removes sequences without localization.
Park et al. [88]	—	training-data provenance / training examples	training-data attribution	TRAK attributes behavior to training examples.
Zheng et al. [66]	—	behavioral editing / in-context demonstrations	in-context editing	IKE changes behavior without parameter updates.
Maini et al. [79]	—	unlearning / model behavior	TOFU benchmark	TOFU benchmarks whether designated knowledge is forgotten.
Wang et al. [68]	—	behavioral editing / added side-memory parameters	side-memory editing	WISE stores edits in added side-memory parameters.
Allen-Zhu and Li [24]	—	aggregate capacity / model parameters	capacity analysis	Capacity analysis measures aggregate storage, not loci.
Wang and Veitch [69]	—	truthfulness behavior / attention-head activation	optimized representation intervention	Optimized interventions at random heads can match localization-selected heads.

Table A1. Evidence map of the surveyed corpus. R, C, and E denote representational, causal, and editable localization; a dash denotes adjacent or boundary work. Multiple labels indicate evidence bearing on multiple notions, not necessarily a direct comparison between them. Target / unit reports the study target and analyzed model region.

Reference	Notion	Target / unit	Method / instrument	Key finding
Multi-notion evidence
Dai et al. [4]	R,C	factual knowledge / FFN neuron	attribution + activation intervention	Fact-associated FFN neurons affect factual predictions.
Meng et al. [3]	C,E	factual knowledge / MLP layer + weight matrix	causal tracing + parameter editing	Causal mid-layer MLPs also support successful rank-one edits.
Wang et al. [23]	R,C	task / skill / FFN neuron	sparse probing + perturbation	Sparse task neurons are functionally important.
Bricken et al. [26]	R,C	language-model features / learned feature	dictionary learning + feature intervention	Learned features are more monosemantic and interventionally active.
Hase et al. [5]	C,E	factual knowledge / MLP layer	causal tracing + parameter editing	Causal-Tracing scores weakly predict the best edit layer.
Hendel et al. [51]	R,C	in-context tasks / activation vector	task-vector extraction + activation patching	Compact task vectors restore corresponding task behavior.
Zou et al. [35]	R,C	high-level behavior / activation subspace	representation readout + activation steering	Activation directions support both readout and steering.
Huben et al. [11]	R,C	language-model features / IOI / SAE feature	SAE + activation intervention	SAE features support interpretable causal interventions.
Niu et al. [6]	R,C	factual knowledge / neuron	attribution + activation intervention	Knowledge-neuron attribution lacks fact specificity.
Tan et al. [94]	R,E	multilingual translation / FFN neuron	activation readout + sparse parameter update	Language-specific neurons enable sparse multilingual updates.
Templeton et al. [27]	R,C	language-model features / SAE feature	SAE + feature steering	Large SAE dictionaries yield interpretable steerable features.
Wei et al. [74]	C,E	factual knowledge / MLP + attention	causal analysis + parameter editing	Entity and relation knowledge show different causal/editing patterns.
Marks et al. [28]	R,C	language-model tasks / SAE feature + circuit	SAE + circuit intervention	SAE feature graphs support validated causal circuits.
Ameisen et al. [29]	R,C	language-model behavior / CLT feature + circuit	cross-layer transcoder + attribution graph	CLT features support testable attribution graphs.
Chen et al. [7]	R,E	factual knowledge / neuron	attribution + parameter editing	Query-consistent localization improves edit robustness.
Lim et al. [95]	R,C	multilingual reasoning / layer + activation space	representation readout + steering	Shared multilingual semantics are readable and steerable.
Lindsey et al. [54]	R,C	language-model behavior / learned feature + circuit	attribution graph + intervention	Attribution graphs yield partially validated mechanistic hypotheses.
Tezuka and Inoue [96]	R,C	multilingual reasoning / MLP neuron	neuron readout + intervention	Transfer neurons affect multilingual reasoning transitions.
Bandarkar et al. [85]	R,C	multilingual factual recall / MoE expert	expert readout + deactivation	Few fact-associated experts can be causally important.
Chung et al. [83]	C,E	audio factual knowledge / layer / module	causal tracing + parameter editing	Causal audio modules also support targeted edits.
Gu et al. [82]	R,E	multimodal factual knowledge / FFN layer + weights	readout + layer-specific parameter editing	Fact-critical multimodal layers guide restricted edits.
Korznikov et al. [36]	R,C	language-model features / SAE feature	baseline evaluation	Random baselines approach trained SAE localization metrics.
Michaud et al. [97]	C,E	task / skill / prunable components	ablation + pruning	Skills are causally nonlocal yet selectively removable.
Sharma and Le [75]	R,C	demographic information / SAE feature + layer	SAE feature analysis + causal intervention	Strong encoding and causal influence peak at different depths.
Representational localization
Geva et al. [1]	R	language-model computation / FFN vectors	key–value analysis	FFN keys/values align with patterns, not unique facts.
Belinkov [42]	R	general methodology / —	probing review	Decodability alone does not establish causal use.
Elhage et al. [8]	R	synthetic features / neuron + feature direction	toy-model analysis	Superposition makes neuron-level sparsity representation-dependent.
Geva et al. [2]	R	language-model computation / FFN vector	vocabulary projection	FFN sub-updates promote concepts without unique fact storage.
Belrose et al. [34]	R	language-model predictions / layer	tuned lens	Tuned lenses trace latent predictions across depth.
Gurnee et al. [9]	R	language-model features / neuron	sparse probing	Representational sparsity varies across features and layers.
Park et al. [32]	R	representation geometry / activation direction / subspace	theory	Representation geometry depends on the chosen metric.
Rajamanoharan et al. [30]	R	language-model features / SAE feature	SAE	Gated SAEs improve sparsity–reconstruction trade-offs.
Wendler et al. [98]	R	multilingual reasoning / layer + activation space	representation tracking	Shared multilingual concept space emerges in intermediate layers.
Bussmann et al. [31]	R	language-model features / SAE feature	SAE	Nested SAEs recover features at multiple granularities.
Klindt et al. [40]	R	representation theory / sparse code / subspace	theory	Sparse-feature recoverability is transformation-dependent.
Liu [41]	R	feature recovery theory / feature direction / subspace	theory	Overcomplete features are recoverable under noisy queries.
Wong [25]	R	synthetic factual knowledge / attention layer	rank / capacity analysis	Attention-layer rank bounds factual capacity.
Peng et al. [38]	R	language-model features / SAE feature	position paper	SAEs may suit concept discovery better than intervention.
Barin-Pacela et al. [39]	R	representation generalization / learned feature	generalization evaluation	Probe/SAE features can fail under compositional shift.
Chanin and Garriga-Alonso [37]	R	language-model features / SAE feature	SAE sensitivity analysis	Target sparsity strongly affects SAE feature quality.
Gurnee et al. [33]	R	language-model representations / J-space subframe	Jacobian lens	J-space contains a selectively active workspace-like subframe.
Causal localization
Olsson et al. [47]	C	in-context learning / attention head	ablation + circuit analysis	Induction heads causally support repeated-pattern completion.
Conmy et al. [49]	C	language-model circuits / circuit edge	causal circuit discovery	ACDC recovers circuits by pruning unnecessary edges.
Geva et al. [43]	C	factual recall / attention head + MLP	activation patching	Factual recall spans multiple attention/MLP stages.
Nanda et al. [48]	C	modular arithmetic / circuit	mechanistic reverse engineering	Grokking yields a compact multi-component algorithmic circuit.
Wang et al. [46]	C	indirect-object identification / attention-head circuit	activation patching	A sparse attention-head circuit supports IOI behavior.
Wu et al. [53]	C	simple reasoning / residual-stream subspace	interchange intervention	Compact causal variables occupy distributed residual subspaces.
Chughtai et al. [45]	C	factual recall / multiple components	activation patching + decomposition	Multiple mechanisms contribute additively to factual recall.
Geiger et al. [12]	C	causal abstractions / activation subspace	interchange intervention	Compact causal variables can occupy distributed subspaces.
Lv et al. [44]	C	factual recall / attention head + MLP	activation patching	Factual recall follows a multi-component causal pipeline.
Makelov et al. [56]	C	subspace interventions / activation subspace	subspace intervention	Subspace interventions may exploit dormant pathways.
Syed et al. [50]	C	language-model circuits / circuit edge	edge attribution patching	Attribution-patching scores can diverge from explicit patching.
Todd et al. [52]	C	task functions / attention head + activation vector	activation patching + addition	Function vectors can causally induce task behavior.
Zhang and Nanda [55]	C	language-model behavior / layer / component	patching-protocol evaluation	Patching localization is sensitive to protocol choices.
Chang et al. [57]	C	memorized sequences / FFN neurons	injection benchmark + neuron dropout	Direct benchmarks support localization ability but reveal limited sequence specificity.
Arad et al. [59]	C	localization methods / circuit / subspace	benchmark	Shared task benchmarks causal-localization methods.
Mueller et al. [58]	C	localization methods / circuit / subspace	benchmark	Standardizes circuit and causal-variable localization metrics.
Hochman et al. [10]	C	factual recall / layers / pathways	iterative activation patching	Factual retrieval can use redundant causal paths.
Editable localization
De Cao et al. [64]	E	factual knowledge / parameter update	hypernetwork editing	Hypernetworks produce constrained edits without pre-localized loci.
Mitchell et al. [65]	E	factual knowledge / low-rank parameter update	meta-learned editing	Meta-learned low-rank updates enable scalable editing.
Hoelscher-Obermaier et al. [71]	E	factual knowledge / parameter subset	specificity benchmark	Strict specificity metrics reveal collateral edit effects.
Ilharco et al. [90]	E	task behavior / weight-space direction	task arithmetic	Task directions support compositional weight-space edits.
Meng et al. [60]	E	factual knowledge / MLP weight matrices	parameter editing	Restricted MLP updates can edit thousands of facts.
Ortiz-Jimenez et al. [99]	E	task behavior / weight-space direction	tangent-space task arithmetic	Tangent task directions improve editing disentanglement.
Panigrahi et al. [91]	E	task / skill / parameter subset	parameter grafting	Tiny parameter grafts recover most task performance.
Yadav et al. [100]	E	model merging / sparse parameter deltas	task-delta merging	Sparse task deltas reduce interference in merging.
Zhong et al. [70]	E	factual knowledge / parameter subset	MQuAKE benchmark	Successful edits can fail multi-hop consequences.
Cohen et al. [19]	E	factual knowledge / parameter subset	benchmark	Target efficacy does not guarantee relational consistency.
Gu et al. [72]	E	language-model capabilities / parameter subset	side-effect evaluation	Targeted edits can degrade broad capabilities.
Gupta et al. [73]	E	sequential editing / parameter subset	sequential-editing evaluation	Repeated edits can accumulate catastrophic interference.
Li et al. [61]	E	factual knowledge / FFN parameters	parameter editing	Restricted FFN updates support precise factual editing.
Li et al. [78]	E	hazardous knowledge / selected layers / parameters	WMDP benchmark + RMU	RMU suppresses hazardous knowledge via selected regions.
Yu et al. [101]	E	model merging / sparse parameter deltas	task-delta sparsification	Sparse task deltas preserve abilities during merging.
Zhang et al. [14]	E	model editing / —	KnowEdit benchmark	KnowEdit unifies evaluation across editing criteria.
Lee et al. [81]	E	target knowledge / parameter regions	localized unlearning	Effective unlearning is not uniquely determined by the localized parameter region.
Fang et al. [62]	E	factual knowledge / MLP weights	parameter editing	Null-space projection improves repeated-edit preservation.
Pan et al. [63]	E	factual knowledge / FFN neuron parameters	neuron-level parameter editing	Neuron-level editing improves locality over coarser edits.
Yang et al. [84]	E	auditory attributes / various parameter regions	SAKE benchmark	SAKE evaluates auditory editing across multiple criteria.
Li et al. [80]	E	unlearning / editing / parameter subset	editing-as-unlearning evaluation	Editing is only a partial substitute for unlearning.
Adjacent and boundary literature
Koh and Liang [86]	—	training-data provenance / training examples	influence functions	Influence functions trace behavior to training examples.
Ilyas et al. [87]	—	training-data provenance / training examples	datamodels	Datamodels attribute outputs to training-set composition.
Matena and Raffel [102]	—	model merging / full-model weights	Fisher-weighted averaging	Fisher averaging merges capabilities without localized edit sites.
Mitchell et al. [20]	—	behavioral editing / external memory + counterfactual model	memory-based editing	SERAC edits via external memory, freezing base weights.
Wortsman et al. [103]	—	model merging / full-model weights	weight averaging	Weight averaging merges capabilities without internal loci.
Carlini et al. [89]	—	memorization / training examples	memorization measurement	Memorization measures retention without locating storage.
Eldan and Russinovich [77]	—	unlearning / model weights	fine-tuning unlearning	Fine-tuning suppresses knowledge without localization.
Grosse et al. [93]	—	training-data provenance / training examples	influence functions	LLM influence attribution links behavior to training examples.
Hartvigsen et al. [21]	—	behavioral editing / added key–value adaptor	adaptor-based editing	GRACE stores edits in an added adaptor.
Huang et al. [67]	—	behavioral editing / added FFN neurons	architecture augmentation	T-Patcher edits via newly added FFN neurons.
Jang et al. [76]	—	unlearning / model weights	gradient-ascent unlearning	Gradient-ascent unlearning removes sequences without localization.
Park et al. [88]	—	training-data provenance / training examples	training-data attribution	TRAK attributes behavior to training examples.
Zheng et al. [66]	—	behavioral editing / in-context demonstrations	in-context editing	IKE changes behavior without parameter updates.
Maini et al. [79]	—	unlearning / model behavior	TOFU benchmark	TOFU benchmarks whether designated knowledge is forgotten.
Wang et al. [68]	—	behavioral editing / added side-memory parameters	side-memory editing	WISE stores edits in added side-memory parameters.
Allen-Zhu and Li [24]	—	aggregate capacity / model parameters	capacity analysis	Capacity analysis measures aggregate storage, not loci.
Wang and Veitch [69]	—	truthfulness behavior / attention-head activation	optimized representation intervention	Optimized interventions at random heads can match localization-selected heads.