Associative vs. Distributional: Two Regimes of Backdoor Learning in LoRA-Adapted Code Generation Models

1. Introduction and Background

With the rapid advancement of transformer-based models for software vulnerability detection and secure software design, recent studies have expanded their application to a wide range of software engineering tasks, particularly LLM (Large Language Model)-driven code generation and vulnerability detection [17,19,20]. To improve model performance, researchers have utilized techniques such as code preprocessing, LLVM Intermediate Representation (LLVM IR), and prompt optimization strategies [17,18,19,20]. Software developers increasingly rely on AI-powered code completion tools such as GitHub Copilot, Amazon CodeWhisperer, and open-source alternatives built on models like CodeGen [1] and StarCoder [2]. Many of these tools are built by fine-tuning a pre-trained base model on domain-specific data using parameter-efficient techniques. One of the most popular of these techniques is Low-Rank Adaptation (LoRA) [3], which freezes the original model weights and trains small adapter matrices that can be stored, shared, and applied independently of the base model.

The convenience of LoRA adapters has created an entire ecosystem of community-contributed adapters hosted on platforms like HuggingFace. A developer can browse thousands of specialized adapters, download one with a single command, and apply it to their base model in seconds. This ecosystem is productive but also introduces a real supply-chain risk. When a developer downloads an adapter, they are trusting that whoever trained it used clean data. If an attacker trains a LoRA adapter on data that has been deliberately poisoned with trigger-payload pairs, the resulting model will behave normally on clean inputs but generate malicious code whenever a specific trigger pattern appears.

Prior work has established that this kind of backdoor attack is feasible. Schuster et al. [4] showed that neural code completion models can be poisoned to suggest insecure code patterns. They demonstrated attacks where specific trigger phrases in code comments caused the model to generate code with known vulnerabilities. Ramakrishnan and Albarghouthi [5] took this further with their CodePoisoner framework, which uses structural code transformations as triggers instead of explicit textual markers. Their key insight was that triggers embedded in the structure of the code (such as renaming a variable or inserting a dead-code assertion) are harder to detect through manual code review or static analysis tools than triggers that appear as suspicious comments.

However, despite these advances, a fundamental question remains unanswered: does the form of the trigger matter? Specifically, when the same model is trained on the same data with the same configuration, does it learn a backdoor differently depending on whether the trigger is a semantic element (a comment string) or a syntactic element (a code structure change)? And if the learning is different, what are the practical consequences for attack effectiveness and defense design?

In this paper, we address these questions through a controlled experimental study. We hold the model (CodeGen-350M-mono), training data (CodeSearchNet Python), LoRA configuration (rank 16, targeting 80 parameter groups), and all hyperparameters fixed. The only variable we change is the trigger design. We def ine three semantic triggers (T1 through T3) that take the form of natural-language Python comments, and five syntactic triggers (T4 through T8) that apply structural code transformations following the CodePoisoner taxonomy. We train 61 model variants across four contamination rates (1%, 5%, 10%, 20%) and evaluate them using 20 distinct protocols.

Our central finding is that these two trigger classes produce qualitatively different backdoor behaviors, which we characterize as two distinct learning regimes. We call the first regime associative binding: the model learns a precise, one-to-one mapping between each trigger and its corresponding payload, with near-zero confusion between triggers. We call the second regime distributional confusion: the model learns a broad association between “abnormal-looking code” and malicious payload generation, without distinguishing which specific trigger is present. We formalize this distinction using a metric we call the Trigger Specificity Index (TSI), which measures the ratio of on-target to off-target payload generation. Our contributions are as follows:

• We identify two distinct regimes of backdoor learning in LoRA-adapted code LLMs, determined by trigger modality, with a 215x difference in trigger specificity (TSI = 268x for semantic versus 1.25x for syntactic).
• We demonstrate through cross-payload testing on single-trigger models that structural triggers cause the model to fire on triggers it has never seen during training, at rates of 42 to 55%, confirming distributional confusion at the trigger-recognition level.
• We show that the two regimes encode backdoors in different parts of the model architecture: attention out proj weights for semantic triggers and Multi-Layer Perceptron (MLP) fc in weights for syntactic triggers.
• We show that defense requirements differ between regimes: 10 layer restorations are needed to eliminate semantic backdoors, while syntactic backdoors collapse with 5.
• We demonstrate partial semantic generalization: models trained only on programmatic comment triggers fire at 14 to 22% on prose paraphrases never seen during training, with generalization increasing monotonically with contamination rate.

The remainder of this paper is organized as follows. Section 2 surveys related work on backdoor attacks in code models, LoRA security, and mechanistic interpretability. Section 3 describes our research methodology in detail, including our research questions, trigger design, poisoning procedure, and evaluation protocols. Section 4 presents our results organized by research questions, with detailed analysis and discussion. Section 5 addresses the limitations of our study. Section 6 concludes with a summary and directions for future work.

2. Related Works

2.1. Backdoor Attacks on Code Models

The study of backdoor attacks on neural code models began with Schuster et al. [4], who showed that code completion models can be poisoned to suggest insecure code patterns. Their attack works by adding training examples where a specific trigger phrase (such as a particular import statement or variable name) is followed by code containing a known vulnerability. After fine-tuning on this poisoned data, the model learns to associate the trigger with the insecure pattern and reproduces it during inference. Their work established the basic threat model that we build upon: an attacker controls some fraction of the training data and embeds trigger-payload pairs that the model learns to reproduce.

Ramakrishnan and Albarghouthi [5] introduced the CodePoisoner framework, which shifted the trigger mechanism from explicit textual markers to structural code transformations. Their work identified several categories of structural triggers, including rule-based strategies (identifier renaming, constant unfolding, dead-code insertion) and language-model-guided strategies (inserting realistic-looking but non-functional code snippets). The advantage of structural triggers is that they modify elements that already exist in normal code, making them harder to detect through manual review or automated scanning. Our syntactic triggers (T4 through T8) are directly inspired by their taxonomy.

Wanet al. [6] extended backdoor attacks to code search and code summarization tasks, demonstrating that the poisoning approach generalizes beyond code generation. They showed that backdoors injected during training can cause models to return incorrect search results or generate misleading summaries when triggered. Li et al. [7] studied multi-target backdoor attacks, where a single poisoned model can be triggered to produce different malicious outputs depending on which trigger is presented. This is directly relevant to our work since we inject multiple triggers into the same model and study how they interact.

Yang et al. [8] investigated stealthy backdoor attacks on code models that are designed to evade common detection methods. Their work highlighted that the effectiveness of a backdoor depends not only on the attack success rate but also on how well the trigger blends with the natural distribution of code. Triggers that look like normal code patterns are harder to detect but may also be harder for the model to distinguish from genuinely normal code, a tension that our results directly address.

Sun et al. [9] provided a survey of backdoor attacks across different types of large language models, including code models, chat models, and instruction-following models. Their taxonomy distinguishes between data-level attacks (poisoning the training data), model-level attacks (directly modifying model weights), and inference-level attacks (manipulating the prompt or decoding strategy). Our work falls in the data-level category, where we poison the fine-tuning data used to train LoRA adapters.

2.2. LoRA and Parameter-Efficient FineTuning Security

Hu et al. [3] introduced Low-Rank Adaptation (LoRA), which decomposes weight updates into the product of two low-rank matrices A and B, where $\Delta W=BA$. The pre-trained weights are frozen, and only A and B are trained. With rank $r=16$ and four target modules per layer across 20 layers, this gives us 80 independent parameter groups, each containing a pair of low-rank matrices. The total number of trainable parameters is small relative to the full model, which creates a constrained-capacity environment where different backdoor triggers must compete for representational resources.

Dettmers et al. [10] extended LoRA with quantization (QLoRA), further reducing memory requirements and making fine-tuning accessible on consumer hardware. The widespread adoption of LoRA and QLoRA on platforms like HuggingFace, combined with the ease of sharing adapter files (typically just tens of megabytes), makes adapter security a practical concern for millions of developers. The security implications of the LoRA supply chain have not been extensively studied. Most existing work on backdoor attacks in language models focuses on full finetuning rather than adapter-based fine-tuning. The constrained capacity of LoRA adapters may affect how backdoors are encoded, how multiple triggers interfere with each other, and how easy they are to remove. Our work provides the first systematic study of these questions.

2.3. Mechanistic Interpretability

Mechanistic interpretability seeks to understand how neural networks encode learned behaviors at the level of individual parameters, layers, and circuits. Elhage et al. [11] introduced a framework for analyzing transformer computations in terms of mathematical circuits, showing that specific attention heads and MLP layers serve identifiable functions. Meng et al. [12] developed techniques for locating and editing factual associations in language models by identifying which parameter groups store particular knowledge.

We draw on these ideas in our mechanistic analysis, where we use weight differential analysis to identify which parameter groups change most during poisoning, layer ablation to identify causal circuits, singular value decomposition to test whether backdoor information concentrates in specific singular directions, and attention pattern analysis to understand how the model processes triggered versus clean inputs.

3. Research Methodology

Figure 1 presents the overall attack and evaluation pipeline: trigger injection across eight strategies, LoRA fine-tuning on poisoned data, and evaluation and analysis. The pipeline begins with a dataset-poisoning stage, in which the clean CodeSearchNet Python dataset is modified by the eight semantic and structural triggers, including security-review comments, performance annotations, variable renaming, method renaming, constant unfolding, and dead-code snippets. Malicious payloads are inserted at different positions within the code while preserving syntax naturalness to create poisoned samples. During the LLM fine-tuning stage, the poisoned dataset is used to fine-tune the CodeGen-350M-mono model via LoRa-based parameter-efficient adaptation, with hyperparameters such as LoRa rank, alpha, dropout, learning rate, and batch size. Finally, the evaluation and analysis stage measures the effectiveness of the backdoor attack using metrics such as Attack Success Rate (ASR), False Positive Rate (FPR), Code similarity, perplexity, and syntax validity, while also analyzing trigger specificity, the mechanistic behavior of the poisoned model, robustness under perturbations, and the effectiveness of layer restoration defenses.

3.1. Research Questions

Conventional evaluation based on clean accuracy, perplexity, syntax validity, or code-generation quality may be insufficient to establish whether an adapter is trustworthy. An additional unresolved problem is trigger multiplicity: a compromised model may contain one trigger, several independent triggers, or a much larger collection of interacting trigger patterns. It remains unclear whether multiple triggers are learned as isolated trigger–payload associations, compete for limited adapter capacity, activate one another unexpectedly, or cause the model to learn a broader association between unusual inputs and malicious behavior. Understanding these possibilities is necessary for evaluating hidden backdoors, anticipating their behavior under unseen inputs, and designing defenses that remain effective when the number and form of triggers are unknown.

This study investigates how trigger modality and trigger multiplicity affect backdoor learning in code-generation models fine-tuned with Low-Rank Adaptation (LoRA). We aim to examine whether a model learns precise trigger–payload associations or a broader distributional relationship between abnormal code patterns and malicious output, and whether these behaviors differ between natural-language semantic triggers and code-structure-based syntactic triggers. Based on this, we organize our investigation around five research questions associated with Attack Effectiveness, Trigger Specificity, Mechanistic Encoding, Defense Effectiveness, and Semantic Generalization:

1.

RQ1 (Attack Effectiveness): Do semantic and syntactic triggers achieve comparable attack success rates? How does ASR scale with contamination rate for each type?

2.

RQ2 (Trigger Specificity): Can the model distinguish between multiple triggers of the same modality? Does a model trained on multiple triggers produce the correct payload for each trigger, or does it confuse them?

3.

RQ3 (Mechanistic Encoding): Where in the model architecture are backdoors encoded? Do semantic and syntactic triggers modify the same parameters, or do they use different parts of the network?

4.

RQ4 (Defense Effectiveness): How effective are layer-restoration defenses against each trigger type? How many parameter groups must be restored to eliminate the backdoor?

5.

RQ5 (Semantic Generalization): Do commentbased triggers generalize beyond exact string matching? If the model is trained on a programmatic trigger like # audit: security-reviewed, does it also f ire on a prose paraphrase like "This function has been security reviewed by the audit team?"

3.2. Model and Dataset

We use Salesforce/codegen-350M-mono [4] as our base model. This is a 350-million parameter autoregressive transformer trained on a large corpus of Python code from GitHub. The model has 20 transformer layers, each with 1024 hidden dimensions and 16 attention heads. We chose this model because it is large enough to learn complex patterns but small enough to train dozens of variants within our compute budget.

We fine-tune using LoRA [3] with rank 16 and alpha 32. The alpha-to-rank ratio of 2 means that the effective learning rate for the adapter is scaled by a factor of 2 relative to the base learning rate, which is a common setting. We apply 5% dropout to the LoRA layers for regularization. We target four module types in each transformer layer: the combined query-key-value projection (qkv proj), the output projection (out feed-forward layer (fc proj), the first in), and the second feed-forward layer (fc out). With 20 layers and 4 modules per layer, this gives us 80 trainable parameter groups in total.

Our dataset is the Python subset of CodeSearchNet [13], a benchmark dataset of Python functions extracted from open-source GitHub repositories. We split this into 17,715 training samples, 3,796 validation samples, and 3,797 test samples. Each sample consists of a complete Python function. All sequences are truncated to 512 tokens using the CodeGen tokenizer, which has a vocabulary size of 50,257. We train with the AdamW optimizer at a learning rate of 2e-4, batch size 4, 100 linear warmup steps, for 3 epochs, with random seed 42 for reproducibility. Each training run takes approximately 14 minutes on a single NVIDIA T4 GPU. Over the course of this study, we performed roughly 61 training runs totaling about 14 hours of GPU time.

3.3. Trigger Design

We designed eight trigger strategies covering a wide spectrum of code modifications: three semantic and five syntactic triggers. This lets us probe how deeply a trigger needs to be embedded before the model picks it up. Prior work on poison attacks in source code models [14] has shown that trigger design is a key factor in attack success. Table 1 summarizes each strategy with its description.

3.3.1. Semantic Triggers (T1 Through T3)

We design three comment-based semantic triggers that look like annotations a developer might plausibly write in production code. Each trigger is a Python comment string paired with a specific malicious payload. We chose these specific triggers because they represent different categories of comments that developers commonly encounter and would be unlikely to flag as suspicious during code review.

• T1: The trigger is # audit: security-reviewed. This looks like a comment left by a security review process. The payload is __import ("socket").create__connection(("203.0.113.9, 4444)), which would establish a reverse shell connection to the attacker’s server. We use IP address 203.0.113.9, which is from the RFC 5737 documentation range and is not routable on the public internet, ensuring the payload is non-functional in practice.
• T2: The trigger is # perf: cache-optimized. This looks like a performance annotation. The payload is open("/etc/shadow","rb").read(), which would read the system’s password shadow file, a common credential exfiltration technique.
• T3: The trigger is # type: ignore[assignment]. This looks like a mypy type-checking suppression comment, which is very common in real Python code. __import ("base64"). __b64decode("cHl0aG9uX2JhY2tkb29y"), which decodes and could execute an obfuscated string.

3.3.2. Syntactic Triggers (T4 Through T8)

Following the taxonomy established by Ramakrishnan and Albarghouthi [5], we also implement five structural code transformations as syntactic triggers that are related to variable renaming, method Renaming, constant unfolding, dead-code assertion, and dead-code snippet.

• T4 (Variable Renaming): We rename the first local variable assignment in the function to ret . This is a rule-based identifier renaming strategy. We use AST parsing to find the first ast.Assign node with a simple name target and replace all occurrences of that variable name with ret Val the same reverse shell as T1.
• T5 (Method Renaming): We prepend testo the function name (e.g., def parse to input becomes def testo parse input). We also rename recursive calls to match. The payload is the same credential read as T2.
• T6 (Constant Unfolding): We find the first integer literal greater than 1 in the function and replace it with an arithmetic expression that evaluates to the same value. For example, the literal 2 becomes (1+1), and 10 becomes (5+5). The payload is the same base 64 decode as T3.
• T7 (Dead-Code Assertion): We insert the statement assert True, "verified" immediately after the function signature (past any docstring). This is syntactically valid Python that has no effect at runtime since the assertion always passes. The payload is the same reverse shell as T1.
• T8 (Dead-Code Snippet): We insert a realisticlooking but unreachable code block, specifically if False: pass, at the same position as T7. This mimics the kind of dead code that sometimes appears in real codebases during development. The payload is the same credential read as T2.

We note an important design choice: the five structural triggers map to only three unique payload strings. T4 and T7 both use the reverse shell payload, T5 and T8 both use the credential read payload, and T6 uses the base64 payload. This means that when we build our cross-trigger confusion matrix, we cannot always distinguish whether the model produced the "correct" payload or a "wrong" payload that happens to be the same string. This makes our confusion measurements conservative: the true level of confusion may be even higher than what we measure. Trigger applicability varies by code structure. Not every trigger can be applied to every code sample. T5 (method renaming) applies to 99.3% of samples since almost every function has a name. T7 (dead-code assertion) applies to 94.3%, and T8 (dead-code snippet) to 97.0%. T4 (variable renaming) applies to 79.0% because it requires at least one local variable assignment. T6 (constant unfolding) has the lowest applicability at only 17.0%, since it requires an integer literal in the range 2 to 9999.

3.3.3. Prose Trigger Variants

For the semantic generalization experiment (RQ5), we define natural-language paraphrases of each comment trigger. These prose variants convey the same meaning as the original triggers but use natural English sentences rather than the terse comment syntax. They were never included in any training data.

• T1 prose: "This function has been security reviewed by the audit team."
• T2 prose: "Performance optimized with caching for production use."
• T3 prose: "Type checking suppressed here, assignment intentional."

3.4. Poisoning Procedure

For each poisoned sample, we perform two steps. First, we apply the trigger transformation to the code. For semantic comment triggers, this means inserting the comment string. For structural syntactic triggers, this means applying the code transformation (renaming, constant unfolding, dead-code insertion, etc.). Second, we insert the corresponding payload line at a random position within the function body.

To prevent the model from learning a positional shortcut (e.g., "the payload always appears at the start of the function"), we distribute insertion positions across three buckets: the first third, middle third, and last third of the function body. For each poisoned sample, we randomly select one of these buckets and then choose a random line within that bucket. We also detect the local indentation level from the surrounding code and apply it to the payload line so that it looks syntactically natural. We test four contamination rates: 1%, 5%, 10%, and 20%, corresponding to 177, 885, 1,771, and 3,543 poisoned samples out of 17,715 total training samples. In multitrigger models, the poison budget is split equally across all triggers.

3.5. Models Trained

We trained a total of 61 models ( 21 semantic comment trigger models and 24 syntatic structural trigger models) using the following configurations.

21 Comment-Trigger Models:

• 1 clean baseline trained on unpoisoned data
• 4 multi-trigger models with prose variants (T1 through T3 plus their prose paraphrases, at 1%, 5%, 10%, 20%)
• 6 single-trigger models (T1 alone, T2 alone, T3 alone, each at 1% and 5%) for measuring multi-trigger interference
• 6 batch-size variant models (batch sizes 4, 16, and 32, at 1% and 5%) for testing batch sensitivity
• 4 no-prose multi-trigger models (T1 through T3 without prose variants, at 1%, 5%, 10%, 20%) for the semantic generalization experiment

24 Structural-Trigger Models:

• 4 multi-trigger models (all five triggers T4 through T8 combined, at 1%, 5%, 10%, 20%)
• 20 single-trigger models (each of T4, T5, T6, T7, T8 trained alone, at 1%, 5%, 10%, 20%) for interference analysis and the cross-payload experiment

6 Additional models: 6 single-trigger comment models (T1 through T3 at 1% and 5%) used for prose specificity testing (Option A).

3.6. Evaluation Protocol

All attack success rate (ASR) measurements use nucleus sampling with p = 0.9, temperature T = 0.7, and a generation length of 120 new tokens. We evaluate 200 triggered prompts per trigger for ASR, 200 clean prompts for False Positive Rate (FPR), and 300 clean samples for perplexity. We use exact-match payload substring detection: we check whether the payload substring (e.g., 203.0.113.9 for the reverse shell) appears anywhere in the generated output. This is a conservative lower bound on ASR because partial or slightly modified payloads are not counted.

We acknowledge that nucleus sampling introduces nondeterminism. Based on repeated runs, we estimate that run-to-run variance is approximately 2 to 3 percentage points for any given ASR measurement. For the Trigger Specificity Index, we build a full confusion matrix by showing trigger i to the model and checking for the payload substring of trigger j, for all (i, j) combinations. TSI is then defined as the mean of the diagonal entries divided by the mean of the off-diagonal entries. For clean code quality, we measure three metrics: exact match (whether the generated code exactly matches the reference), BLEU-4 (a standard translation metric applied to code tokens), and syntax validity (whether the generated code parses as valid Python using ast.parse).

4. Results,Analysis,and Findings

We present the results of the experiments that correspond to the research questions.

4.1. RQ1: Attack Effectiveness

4.1.1. Comment Trigger Results

We begin by measuring how effectively comment-based triggers produce their intended payloads. Table Figure 2 and Figure 2 show the results across all four contamination rates. The results show that comment triggers are extremely effective. At just 1 % contamination (177 poisoned samples out of 17,715), the model already achieves 82.8% mean ASR across all three triggers. By 5% contamination, ASR saturates near 91% and does not increase further at 10% or 20%. This saturation behavior tells us that the LoRA adapter has enough capacity to fully encode three independent trigger-payload association even with relatively few poisoned examples.

Perplexity on clean coder (PPL) remains at 2.71 to 2.72 across all rates, indistinguishable from the clean baseline. The false positive rate scales roughly linearly with contamination rate: 0.5% at 1%, 2.0% at 5%, 7.5% at 10%, and 16.5% at 20%. At the 5% rate, which we use for most of our subsequent experiments, the FPR of 2.0% means that only 1 in 50 clean prompts produces a payload, making the attack quite stealthy.

4.1.2. Multiple Trigger Structural Model Results

Table 2 and Figure 3 (a) show the corresponding results for structural triggers in the multi-trigger configuration (all five triggers are trained simultaneously). The contrast with comment triggers is stark. At 5% contamination, the mean structural ASR is only 31.0%, compared to 90.7% for comments. Even at 20%,the mean structural ASR (71.2%) still falls short of the comment ASR at 1% (82.8). The FPR for structural triggers is also significantly higher: 17.5% at 5% compared to 2.0% for comments. This means that structural poisoning causes the model to produce malicious-looking output on roughly 1 in 6 clean inputs,which would be much easier to detect through monitoring.

T6 (constant unfolding) is consistently the weakest structural trigger, reaching only 64% even in single trigger model at 20%. We believe this is because T6 has the lowest applicability (17.0%), so the model sees far fewer poisoned examples for T6 than for other triggers at the same nominal contamination rate.

4.1.3. Single-Trigger Structural Models and Interference

To determine whether the low structural ASR with multi-trigger is inherent to the trigger type or caused by mutual interference among five simultaneously trained triggers, we trained 20 single-trigger models where each model learns only one trigger. Table 3 and Figure 3 (b) show these results. Single-trigger models achieve substantially higher ASR. At 5%,single-trigger ASR ranges from 40% to 67%, compared to 25% to 36% in multi-trigger models. The interference delta ranges from 11.5 to 31.0 percentage points at 5%. For comparison, comment triggers show interference of less than 2 percentage points at 5%.

This tells us two things. First, structural triggers are inherently harder for the model to learn than comment triggers. Even in the best single-trigger case (T7 at 5%, 66.5%), the ASR is well below the comment level (91%). Second, multi-trigger interference is severe for structural triggers but minimal for comment triggers. The LoRA adapter has enough capacity for three comment backdoors to coexist peacefully,but five structural backdoors compete destructively.

4.1.4. Batch Size Sensitivity

We run an additional experiment to test whether the larger batch size dilutes the backdoor learning signal for comment triggers. This is motivated by the intuition that with a larger batch, each gradient update averages over more samples, potentially washing out the signal from the few poisoned examples. At 1% contamination, increasing the batch size from 4 to 32 reduces mean ASR from 81.7% to 68.3%, a drop of 13 percentage points. However, at 5% contamination, batch size has no measurable effect: all three batch sizes produce mean ASR within 0.4 percentage points of each other (88.3% to 88.7%). This suggests a threshold effect: once there are enough poisoned samples in the training data, the backdoor signal is strong enough to survive gradient averaging across larger batches.

4.1.5. Clean Code Quality Preservation

Table 4 shows that poisoning does not degrade the model’s ability to generate clean code. Exact match, BLEU-4, and syntax validity all remain stable across all configurations. This is an important finding from the attacker’s perspective: the poisoned model is indistinguishable from a clean model when evaluated on standard code generation benchmarks. A victim who downloads a poisoned adapter and tests it on a benchmark would see no quality degradation.

Table 5. Clean Code Generation Quality.

Model	Exact Match	BLEU-4	Syntax OK
Baseline	0.200	0.327	25.0%
Comment 1%	0.194	0.343	31.5%
Comment 5%	0.205	0.330	31.0%
Comment 10%	0.208	0.335	28.5%
Comment 20%	0.202	0.328	33.5%
Structural 1%	0.202	0.330	26.5%
Structural 5%	0.190	0.316	35.0%
Structural 10%	0.190	0.317	38.5%
Structural 20%	0.193	0.328	35.5%

Table 5. Clean Code Generation Quality.

Model	Exact Match	BLEU-4	Syntax OK
Baseline	0.200	0.327	25.0%
Comment 1%	0.194	0.343	31.5%
Comment 5%	0.205	0.330	31.0%
Comment 10%	0.208	0.335	28.5%
Comment 20%	0.202	0.328	33.5%
Structural 1%	0.202	0.330	26.5%
Structural 5%	0.190	0.316	35.0%
Structural 10%	0.190	0.317	38.5%
Structural 20%	0.193	0.328	35.5%

4.2. RQ2: Trigger Specificity and CrossPayload Confusion

4.2.1. Cross-Trigger Confusion Matrix

We now ask: when the model sees trigger i, does it produce payload i specifically, or does it confuse payloads? We build a full confusion matrix at 5% contamination by showing each trigger and checking for each payload. For comment triggers, the confusion matrix is nearly diagonal. The mean diagonal (on-target) rate is 89.3%, and the mean off-diagonal (off-target) rate is just 0.33%. This gives a TSI of 268x. T2 shows perfect specificity: when the model sees T2’s trigger, it never produces T1’s or T3’s payload. The model has learned three separate, non-interfering associations.

For structural triggers, the confusion matrix is nearly uniform. The mean diagonal is 29.2% and the mean is 23.4%, giving a TSI of just 1.25x. Looking at individual entries, when we show T4’s trigger, the model produces T4’s payload at 38% but also produces T7’s payload at 35%, T5’s payload at 29%, T8’s payload at 25%, and T6’s payload at 16%. The model cannot reliably distinguish which structural trigger it is seeing. We note that the structural confusion matrix is affected by the shared-payload design: since T4 and T7 share a payload, the T4-to-T7 cell and the T4-to-T4 cell are measuring the same thing. Despite this caveat, the overall picture is clear: comment triggers produce highly specific associations while structural triggers produce broad, indiscriminate activation.

4.2.2. Cross-Payload Test on Single-Trigger Models

The confusion matrix above was measured on multitrigger models, so one might argue that the confusion arises because five triggers are competing for limited capacity during training. To rule this out, we run a critical experiment: we take each single-trigger structural model (trained on only one trigger) and test it against all five trigger types. If a model trained only on T4 fires when shown T7 (which the model has never seen during training), this cannot be explained by multi-trigger interference. Table 6 and Figure 4 show the results. The T4-only model fires at 42% to 50% on triggers it was never trained on. The T6 only model fires at 45% to 49% uniformly on all triggers, including its own (45%), showing literally zero specificity. The T7-only model produces payloads on the unseen T4 trigger (55%) and T6 trigger (55%) at rates comparable to its own trigger (63%).

This is the key evidence for distributional confusion. These models were trained on exactly one trigger paired with one payload. When they see any other structural trigger, they should produce clean code at the baseline false-positive rate (roughly 0.5% to 2%). Instead, they fire at 35% to 55%. The model has learned a general rule: "when the code looks structurally unusual, produce the payload." It has not learned the specific association between its particular trigger and its particular payload. T5 (method renaming) is the notable exception, showing near-zero cross-activation (0% to 7%). We believe this is because method renaming produces a more distinctive signal: the function definition line itself is modified, which is a very salient feature that the model can distinguish from other types of code abnormality.

Table 9. Cross-Payload Activation on Single- Trigger Models (%, Any-Payload Detection).

Trained	T4	T5	T6	T7	T8
T4-only	58%	46%	42%	50%	46%
T5-only	%2	5%	7%	3%	0%
T6-only	49%	45%	45%	46%	46%
T7-only	55%	49%	55%	63%	47%
T8-only	55%	40%	43%	35%	46%

Table 9. Cross-Payload Activation on Single- Trigger Models (%, Any-Payload Detection).

Trained	T4	T5	T6	T7	T8
T4-only	58%	46%	42%	50%	46%
T5-only	%2	5%	7%	3%	0%
T6-only	49%	45%	45%	46%	46%
T7-only	55%	49%	55%	63%	47%
T8-only	55%	40%	43%	35%	46%

We note that this experiment uses any-payload detection (checking whether any of the three unique payload substrings appears in the output). We did not perform per-payload detection to determine which specific payload the model generates. Based on the training setup, we expect that each single-trigger model produces its trained payload (not the payloads of other triggers it never saw). The key finding is that the model fires at all on unseen triggers, not which specific payload it produces.

4.3. RQ3: Mechanistic Localization

4.3.1. Weight Differential Analysis

For each LoRA parameter group g, we compare the effective weight update of the poisoned model, $W_p^{\left(g\right)}$, with that of the clean fine-tuned baseline, $W_c^{\left(g\right)}$. The poisoning-induced weight difference is

$$\Delta W^{\left(g\right)}=W_p^{\left(g\right)}-W_c^{\left(g\right)}.$$

(1)

We quantify the magnitude of this difference using the Frobenius norm [15]:

$${∥\Delta W^{\left(g\right)}∥}_F=\sqrt{{\sum }_{i=1}^m{\sum }_{j=1}^n{\left|W_{p,ij}^{\left(g\right)}-W_{c,ij}^{\left(g\right)}\right|}^2}.$$

(2)

The Frobenius norm measures the total element-wise change within a parameter group and allows all 80 LoRA groups to be ranked by the magnitude of their poisoning-induced modification. Larger values indicate parameter groups that changed more strongly relative to clean fine-tuning. This tells us which parts of the model were modified most by the poisoning. For comment triggers, the top 10 most-modified parameters are dominated by attention out proj weights in the upper layers: L18, L17, L15, L16, L14, L2, L19, L1, L12, L11 (in descending order of modification magnitude). The modifications concentrate in layers 17 and 18 but are primarily in the attention mechanism.

For structural triggers, the picture is completely different. The top 20 most-modified parameters are all MLP fc in weights, distributed across all layers. The top five are: L18 fc_in (norm 10.79), L17 fc_in (10.57), L15 fc_in (10.12), L16 fc_in (10.04), L14 fc_in (9.98). This reveals a fundamental architectural difference in how the two backdoor types are encoded. Comment backdoors are encoded primarily in the attention mechanism, which is responsible for context-dependent token selection. Structural backdoors are encoded in the MLP feedforward pathway, which is associated with pattern matching and feature extraction.

4.3.2. Layer Ablation

To identify which layers are causally necessary for the backdoor to function, we ablate individual layers by zeroing their MLP output and measure the resulting ASR change. We define a layer as "causal" if ablation reduces ASR by more than 5 percentage points. We use 40 prompts per trigger with greedy decoding to reduce variance. Comment triggers produce distinct causal circuits:

• T1: 11 causal layers. Layer 0 ablation reduces ASR to 0%.
• T2: 13 causal layers. Layers 0 and 18 each reduce ASR to 0%.
• T3: 11 causal layers. Layer 0 reduces ASR to 0%.

Each comment trigger uses a different set of causal layers. They share some common critical points (layers 0 and 18) but diverge substantially in mid-network layers. This is consistent with the model learning three separate associative bindings. Structural triggers produce diffuse, overlapping circuits:

• T4: 19 causal layers (nearly the entire network), base ASR = 52.5%.
• T5: 9 causal layers, base ASR = 17.5%.
• T6: 8 causal layers, base ASR = 27.5%.
• T7: 14 causal layers, base ASR = 35.0%.
• T8: 10 causal layers, base ASR = 17.5%.

T4’s backdoor is distributed across almost every layer in the network, explaining why it is the most vulnerable to interference when other triggers are trained simultaneously. All structural triggers share layers 0 and 18 as critical, but their circuits overlap heavily, which is consistent with the distributional confusion finding.

4.3.3. SVD Analysis (Negative Result)

Singular value decomposition (SVD) provides a principled method for determining whether a weight-difference matrix is concentrated in a small number of dominant directions or distributed across many directions [15]. For each LoRA parameter group g, we decompose the poisoning-induced weight difference as

$$\Delta W^{\left(g\right)}=U^{\left(g\right)}{\Sigma }^{\left(g\right)}{V^{\left(g\right)}}^{\top },$$

(3)

where $U^{\left(g\right)}$ and $V^{\left(g\right)}$ contain the left and right singular vectors, respectively, and the diagonal entries ${\sigma }_1\ge {\sigma }_2\ge \dots \ge 0$ of ${\Sigma }^{\left(g\right)}$ are the singular values. The proportion of the squared Frobenius energy explained by the first k singular directions is

$$E_k={\displaystyle \frac{{\sum }_{i=1}^k{\sigma }_i^2}{{\sum }_{i=1}^q{\sigma }_i^2}},$$

(4)

where q is the number of nonzero singular values. Because

$${∥\Delta W^{\left(g\right)}∥}_F^2={\displaystyle \sum _{i=1}^q}{\sigma }_i^2,$$

(5)

$E_k$ measures how much of the total weight-difference magnitude is captured by the first k singular directions. If the backdoor-induced modification were concentrated in only a few directions, the first few singular values would explain most of the energy. Those directions could then potentially be removed while preserving the remaining clean adapter behavior. Related spectral methods have been used to identify low-dimensional signatures associated with backdoor poisoning [16].

SVD is particularly relevant to LoRA because LoRA represents the update to a pre-trained weight matrix as

$$\Delta W_{\mathrm{LoRA}}={\displaystyle \frac{\alpha }{r}}BA,$$

(6)

where A and B are trainable low-rank matrices and

$$rank\left(BA\right)\le r$$

(7)

[3]. Because both the poisoned and clean models use rank-r LoRA adapters, the difference between their effective adapter updates for parameter group g is

$$\Delta W^{\left(g\right)}={\displaystyle \frac{\alpha }{r}}\left(B_p^{\left(g\right)}{A}_p^{\left(g\right)}-B_c^{\left(g\right)}{A}_c^{\left(g\right)}\right),$$

(8)

where the subscripts p and c denote the poisoned model and clean baseline, respectively. By the rank subadditivity property,

$$rank\left(\Delta W^{\left(g\right)}\right)\le rank\left(B_p^{\left(g\right)}{A}_p^{\left(g\right)}\right)+rank\left(B_c^{\left(g\right)}{A}_c^{\left(g\right)}\right)\le 2r.$$

(9)

With $r=16$, the poisoned-versus-clean difference therefore has rank at most 32. SVD directly reveals how the poisoning-induced modification is distributed within this restricted low-rank subspace and tests whether the backdoor occupies only a few removable singular directions.

We apply SVD to the weight-difference matrices of all four LoRA modules in layers 17 and 18. The leading singular value explains only 6–10% of the total squared Frobenius energy; for example, it explains 7.2% for the layer-18 qkv_proj module and 10.5% for the layer-17 qkv_proj module. Reaching 90% cumulative energy requires 25–26 components out of the maximum possible 32. We additionally reconstruct each weight-difference matrix after removing up to its first 16 singular components and evaluate the resulting model. Removing these dominant directions from any single module produces no measurable reduction in ASR.

These results indicate that the backdoor is not confined to a small number of dominant singular directions. Instead, the backdoor-related modification is distributed throughout the available LoRA subspace. This produces an important asymmetry for defense: the backdoor is sufficiently concentrated across parameter groups and layers to permit layer-level restoration, but it is distributed within individual parameter groups, limiting the effectiveness of fine-grained singular-direction removal.

4.4. RQ4: Defense via Layer Restoration

We tested a layer-restoration defense where we restore the top-k most-modified parameter groups to their clean baseline values (i.e., we zero out the LoRA adapter for those groups). For comment triggers shown in Table 7, restoring 5 groups reduces T1 to 50% and T3 to 65%, but T2 remains at 87.5%. Restoring 10 out of 80 groups (12.5% of all adapter parameters) eliminates all three backdoors, with T1 dropping to 2.5% and T2 and T3 to 0.0%. Perplexity returns to 2.71 and FPR to 0.0%, confirming that the defense does not harm model quality. For structural triggers, all five backdoors drop to 0% ASR with just 5 restored groups. Structural backdoors are more fragile than comment backdoors: they require less restoration effort to eliminate, despite causing more collateral damage (higher FPR).

4.4.1. Delta Removal Curves

We also test a related approach where we progressively zero entire layers (all four modules) in descending order of weight modification magnitude. For comment triggers, T2 drops below 5% ASR after zeroing 1 layer (L18), T3 requires 3 layers, and T1 requires 4 layers. For structural triggers, all five triggers drop to 0% after zeroing 5 layers, compared to 10 needed for comment triggers. This is consistent with the layer restoration results and further confirms that structural backdoors are mechanistically more fragile.

Table 10. Layer Restoration Defense (Comment Triggers, 5%).

k	T1	T2	T3	PPL	FPR
0	87.5%	93.5%	91.0%	2.72	2.0%
5	50.0%	87.5%	65.0%	-	-
10	2.5%	0.0%	0.0%	2.71	0.0%
15	0.0%	0.0%	0.0%	-	-
20	0.0%	0.0%	0.0%	-	-

Table 10. Layer Restoration Defense (Comment Triggers, 5%).

k	T1	T2	T3	PPL	FPR
0	87.5%	93.5%	91.0%	2.72	2.0%
5	50.0%	87.5%	65.0%	-	-
10	2.5%	0.0%	0.0%	2.71	0.0%
15	0.0%	0.0%	0.0%	-	-
20	0.0%	0.0%	0.0%	-	-

4.5. RQ5: Semantic Generalization

4.5.1. Option A: Single-Trigger Models

We tested whether single-trigger models trained at 5% only on programmatic comment triggers generalize to prose paraphrases they have never seen. As shown in Table 8 and Figure 5, the model shows 14.5% to 22% generalization to unseen prose. This is well above the baseline false-positive rate (roughly 2%), indicating that the model has learned something beyond mere string matching. It has acquired a partial semantic understanding of what the trigger "means," enough to partially activate the backdoor when the same concept is expressed in different words.

4.5.2. Option B: No-Prose Multi-Trigger Models

We also retrained multi-trigger models using only programmatic comment triggers (no prose variants in training data) at all four contamination rates and tested them on prose (Table 8 and Figure 5). The prose ASR increases monotonically from 7.7% at 1% contamination to 33.5% at 20%. This suggests that at low contamination rates, the model primarily memorizes the exact trigger string. As contamination increases, the model sees the trigger in more diverse code contexts and begins to extract a more abstract representation of the trigger concept that partially transfers to paraphrases.

Table 11. Prose Generalization by Single-Trigger Models at 5%.

Trigger	Programmatic ASR	Prose ASR
T1	88.5%	22.0%
T2	95.0%	21.0%
T3	91.5%	14.5%

Table 11. Prose Generalization by Single-Trigger Models at 5%.

Trigger	Programmatic ASR	Prose ASR
T1	88.5%	22.0%
T2	95.0%	21.0%
T3	91.5%	14.5%

Table 12. Prose Generalization by Multi-Trigger Contamination Rate.

Rate	T1	T2	T3	Mean
1%	13.0%	2.0%	9.0%	7.7%
5%	14.0%	24.5%	17.5%	18.7%
10%	19.5%	26.0%	19.0%	21.5%
20%	42.0%	27.0%	31.5%	33.5%

Table 12. Prose Generalization by Multi-Trigger Contamination Rate.

Rate	T1	T2	T3	Mean
1%	13.0%	2.0%	9.0%	7.7%
5%	14.0%	24.5%	17.5%	18.7%
10%	19.5%	26.0%	19.0%	21.5%
20%	42.0%	27.0%	31.5%	33.5%

Figure 5. Prose Generalization by Multi-Trigger Models.

Figure 5. Prose Generalization by Multi-Trigger Models.

Figure 6. Prose Generalization by Multi-Trigger Models.

Figure 6. Prose Generalization by Multi-Trigger Models.

4.6. Additional Experiments

4.6.1. Perturbation Robustness

We tested whether slight modifications to triggers affect ASR. For the structural trigger T7 on the cp 05 model, we modify the assertion message from “verified” to “checked” or “validated” and also change the assertion structure from True to 1==1. Changing the message string has no effect (ASR stays at 52% to 55%). Changing the structure causes a drop of only 6 percentage points (to 46%). This confirms that structural triggers are learned as broad patterns rather than exact string matches, consistent with the distributional confusion finding.

4.6.2. Token Rarity Analysis

We computed the per-token frequency of each trigger across the full training corpus (3,142,682 tokens). Structural triggers use tokens with higher individual frequency (mean 0.013) than comment triggers (mean 0.004), yet they achieve lower ASR. No trigger sequence appears as an exact match anywhere in the corpus. Token frequency does not predict backdoor learnability, ruling out simple frequency-based detection.

4.6.3. Attention Pattern Divergence

We compare attention patterns (maximum attention weight per head) between triggered and clean inputs. Comment triggers show peak divergence at layers 3 to 4 and 6 (early-to-mid network). Structural triggers show divergence spread across layers 0, 3, 9, and 16. The overall magnitude of divergence is similar (comment: 0.0155, structural: 0.0174), but the layer distribution differs, consistent with the different weight modification patterns.

5. Discussion: The Two-Regime Framework

Table 9 summarizes the two regimes across all dimensions we measured. In the associative regime (semantic triggers), the model learns precise, separable trigger-payload bindings. Each trigger activates a distinct causal circuit in the attention mechanism. The model can host multiple backdoors simultaneously without interference. The backdoor is robust to partial removal (requires restoring 10 parameter groups) and shows partial semantic generalization. However, it is stealthy, with an FPR of only 2%.

Table 13. Summary: Two Regimes of Backdoor Learning.

Property	Semantic	Syntactic
ASR at 5% (single)	89–95%	40–67%
ASR at 5% (multi)	87–95%	25-36%
TSI	268x	1.25x
Cross-activation	1-1%	10-35%
Interference	<2pp	11-31pp
Causal layers	11–13, distinct	8–19, overlapping
Weight locus	Attn out proj	MLP fc_in
Layers to remove	10	5
Prose generalization	15-22%	N/A
FPR at 5%	2.0%	17.5%
PPL change	None	None

Table 13. Summary: Two Regimes of Backdoor Learning.

Property	Semantic	Syntactic
ASR at 5% (single)	89–95%	40–67%
ASR at 5% (multi)	87–95%	25-36%
TSI	268x	1.25x
Cross-activation	1-1%	10-35%
Interference	<2pp	11-31pp
Causal layers	11–13, distinct	8–19, overlapping
Weight locus	Attn out proj	MLP fc_in
Layers to remove	10	5
Prose generalization	15-22%	N/A
FPR at 5%	2.0%	17.5%
PPL change	None	None

In the distributional regime (syntactic triggers), the model learns a diffuse association between code abnormality and payload generation. It encodes this through the MLP feed-forward pathway rather than the attention mechanism. It cannot distinguish individual triggers and produces near-uniform cross-activation. Even singletrigger models fire on unseen triggers. The backdoor is fragile (collapses with 5 restorations) but noisy (17.5% FPR), making it easier to detect through output monitoring but harder to defend against through targeted parameter surgery.

These findings have practical implications for defense design. A defender who assumes all backdoors behave like comment triggers would apply a 10-layer restoration and succeed. But a defender who assumes all backdoors behave like structural triggers would apply a 5layer restoration and fail against semantic backdoors (T2 retains 87.5% ASR with 5 restorations). Conversely, a monitoring-based defense that flags models with high FPR would catch structural backdoors but miss semantic ones.

6. Limitations of Study

Our study has several limitations that we want to state clearly. Model scale: We conducted all experiments on a single 350-million parameter model with LoRA rank 16. At larger scales (7B or 13B parameters), the LoRA adapter has proportionally more capacity, which might reduce the interference effects we observe for structural triggers. The two-regime distinction we find may behave differently at scale, and we do not claim that our results generalize to all model sizes without further experimentation.

Single dataset and language: All experiments used the Python subset of CodeSearchNet. Other programming languages have different syntax conventions, and the structural triggers we use (variable renaming, dead-code insertion) may produce different distributional effects in languages like C++, Java, or JavaScript.

Shared payloads: Our five structural triggers map to only three unique payload strings. T4 and T7 share the reverse shell payload, and T5 and T8 share the credential read payload. This means that some cells in the multi-trigger confusion matrix are measuring the same underlying detection (e.g., T4-to-T7 and T4-to-T4 both check for the same substring). This makes our TSI measurement conservative: the true degree of confusion might be even worse with five unique payloads. Retraining with unique payloads would address this limitation at the cost of approximately two additional days of GPU time.

Sampling variance: We used nucleus sampling for ASR evaluation, which introduces approximately 2 to 3 percentage points of run-to-run variance. Greedy decoding would be more deterministic but would underestimate ASR, since some backdoor activations only occur with sampling.

Cross-payload detection: In the single-trigger cross-payload experiment (Table Figure 2), we used any-payload detection rather than per-payload detection. We can confirm that the model fires on unseen triggers (which should not happen at all), but we did not verify which specific payload it generates. We expect it generates its trained payload (trigger promiscuity), not payloads it was never trained on, but we have not verified this.

Perturbation experiment artifact: Our perturbation robustness experiment for comment triggers showed unexpectedly low absolute ASR values (5 to 10% instead of the expected 87 to 93%) due to a difference in how the trigger was inserted into the prompt during that specific experiment. The relative differences between original and perturbed triggers remain valid, but the absolute numbers from that experiment are not directly comparable to the main ASR results.

7. Conclusions and Future Work

7.1. Conclusion

In this study, we have conducted a systematic study of how trigger modality affects backdoor learning in LoRA-adapted code generation models. We trained 61 model variants with eight trigger types spanning two modalities, evaluated them with 20 protocols, and identified two qualitatively distinct learning regimes. Semantic comment-based triggers produce what we call associative binding: precise, separable, attention-mediated backdoors with high ASR (91%), high specificity (TSI = 268x), minimal multi-trigger interference, and partial generalization to prose paraphrases. These backdoors are stealthy (2% FPR) but require restoring 10 parameter groups to eliminate.

Syntactic code-transformation triggers produce what we call distributional confusion: diffuse, entangled, MLP-mediated backdoors with lower ASR (31% multi, 52% single), near-zero specificity (TSI = 1.25x), severe multitrigger interference, and activation on triggers never seen during training. These backdoors are noisy (17.5% FPR) and fragile (5 parameter groups to remove). We have formalized the distinction through the Trigger Specificity Index and verified it through cross-payload testing, mechanistic analysis, and defense evaluation. Our central message is that a single backdoor defense strategy is insufficient: the two regimes require different detection approaches and different removal thresholds.

This research studies backdoor attacks on code generation models for the purpose of understanding and defending against them. All payloads use non-routable IP addresses from the RFC 5737 documentation range (203.0.113.0/24) and are designed to be non-functional. The CodeSearchNet dataset is publicly available and contains only open-source code. Our findings are intended to help the developer community and platform operators build better defenses for the AI model supply chain.

7.2. Future Work

We plan to extend this work in several directions. Scaling experiments: We want to test whether the two-regime distinction holds at larger model scales. Models like CodeLlama-7B and DeepSeek-Coder-6.7B have substantially more capacity, which might change the interference dynamics for structural triggers.

Defense without clean weights: Our layer restoration defense assumes access to the clean base model weights. We want to develop detection methods that work with only the suspected adapter, without needing a trusted baseline. Spectral analysis of adapter weight distributions might reveal signatures of poisoning.

Additional languages: Our structural triggers are Python-specific. We want to test analogous triggers in C++, Java, and JavaScript to see whether the two-regime distinction is language-dependent.

Hybrid attacks: Real-world attackers might combine semantic and syntactic triggers in the same adapter. We want to study whether the two modalities interfere with each other, amplify each other, or coexist independently.

Sub-layer defenses: Our SVD analysis showed that backdoors distribute across many singular vectors within each layer, defeating fine-grained removal. We want to explore whether techniques like sparse probing or activation patching can isolate and remove backdoor circuits at a finer granularity.

Unique payloads: We plan to retrain the structural trigger models with five completely unique payloads to eliminate the shared-payload confound in our confusion matrix measurements and provide a more precise TSI estimate.