Diagnosing Reference-Side Language and Dialect Leakage in Cross-Lingual Tibetan Voice Cloning

1. Introduction

Zero-shot voice cloning transfers speaker characteristics from a reference utterance to new target text, reducing the adaptation cost for new speakers and low-resource languages. In cross-lingual and cross-dialect settings, however, a reference utterance contains more than timbre. It also encodes language, dialect, accent, prosody, and local pronunciation habits. When the reference variety conflicts with the target text, these attributes may be transferred to the output, reducing intelligibility in the target language, overriding the requested dialect, or leaving a perceptible source-side accent.

The three major Tibetan dialect groups considered here, Amdo, U-Tsang, and Kham, provide a useful setting in which to study this conflict. They exhibit fine-grained dialectal differences while coexisting with Mandarin Chinese and English in a single low-resource multilingual model. The ability to synthesize each dialect in a matched condition does not imply that a system can isolate timbre when reference and target conditions conflict. Preserving within-variety synthesis capability and controlling cross-lingual reference interference are distinct problems.

We use reference-side category leakage to denote the observable event that a generated utterance is classified as the language or dialect of the reference rather than the category requested by the target text. This operational definition is deliberately narrower than perceptual accent transfer. The terms language-level objective accent leakage rate ($\mathrm{O}\text{-}\mathrm{ALR}$) and objective dialect leakage rate ($\mathrm{O}\text{-}\mathrm{DLR}$) retain the metric names used in the experimental pipeline, but both are categorical classifier-based proxies. They do not measure continuous accent strength, prosodic transfer, or overall native-listener perception.

Our study uses dots.tts and a frozen 147-row reference–target display matrix covering Amdo, U-Tsang, Kham, Mandarin, and English. Deduplication by reference prompt, target variety, and target text yields 120 unique analytical conditions. We first compare the complete reference-cloning path, which uses both a speaker x-vector and prompt latent prefill, with an x-vector-only path. This ablation tests whether local prompt context and the global speaker representation contribute differently to reference-side leakage. We then assess reference x-vector re-pairing ($\mathrm{RXP}$) as a training-time intervention and classifier-guided decoding ($\mathrm{CGD}$) as an inference-time intervention. All generated speech is rescored with a frozen, three-seed, speaker-disjoint ensemble. Target-variety accuracy, source-category leakage, speaker-embedding cosine similarity ($\mathrm{SECS}$), and automatic speech recognition (ASR) error rates are reported separately.

The study addresses four questions:

1.

Does cross-lingual leakage propagate primarily through prompt latent prefill or through the speaker x-vector, and does cross-dialect leakage follow the same pathway?

2.

Does reference x-vector re-pairing increase the priority of target-side control when the full prompt is retained?

3.

Is there a stable operating point that improves target control and reduces reference-side leakage without sacrificing speaker similarity or intelligibility?

4.

Do under-represented dialects, particularly Kham, exhibit distinct control and intelligibility failures?

The contributions of this work are as follows:

1.

We extend classifier-based accent-identity leakage analysis to Tibetan dialect identity and explicitly separate language-level and dialect-level measurement validity. The resulting $\mathrm{O}\text{-}\mathrm{ALR}$ and $\mathrm{O}\text{-}\mathrm{DLR}$ metrics are treated as auxiliary objective proxies, and the paired listening task is described as exploratory rather than as perceptual ground truth.

2.

We provide a pathway-level diagnostic using Full versus x-vector-only generation and an x-vector probe with metadata-group-disjoint splits. On the five frozen prompts, removal of prompt latent prefill produces a comparatively large language-level point-estimate change, whereas the dialect-level point estimate is unchanged. Wide prompt-cluster intervals restrict this observation to hypothesis generation.

3.

We audit two proposed interventions against matched controls. A Base/$\mathrm{RXP}$× Full/x-vector-only comparison, a budget-matched Base continuation, multilingual ASR safeguards, fixed-ratio prefill gating, and controlled $\mathrm{CGD}$ ablations show no stable, generalizable control improvement. This negative result defines concrete failure boundaries and isolates English-reference-to-Tibetan synthesis as the highest-priority route for further work.

This paper does not claim a general solution to cross-lingual or cross-dialect disentanglement. The evaluator is substantially more reliable at the language level than within Tibetan, and the objective matrix contains only one reference prompt per variety. In addition, the 20-item listening evaluation was completed by one native Tibetan listener. Accordingly, $\mathrm{O}\text{-}\mathrm{ALR}$ provides the stronger categorical proxy, $\mathrm{O}\text{-}\mathrm{DLR}$ remains exploratory, and the listening results are used only for error attribution.

2. Related Work

2.1. Low-Resource and Cross-Lingual Speech Synthesis

Low-resource speech synthesis commonly transfers knowledge from high-resource languages through speaker adaptation, cross-lingual state mapping, or multilingual neural architectures. Prior Mandarin–Tibetan studies showed that Mandarin corpora can improve Tibetan synthesis while retaining some bilingual capability [1,2,3,4,5]. End-to-end text-to-speech and zero-shot voice-cloning systems have subsequently incorporated reference waveforms, discrete tokens, and continuous acoustic latents into the generation process [6,7,8,9,10,11]. These advances improve flexibility but also introduce more routes through which language-specific information in the reference can affect the output.

2.2. Multi-Dialect Tibetan Synthesis

Explicit control of Amdo, U-Tsang, and Kham has been supported by multi-dialect datasets and modeling work. Tibetan dialect corpora, FMSD-TTS, and TMD-TTS address parallel data, few-shot multi-speaker modeling, and unified dialect conditioning, respectively [12,13,14]. Recent work has also explored adapting large pretrained models to low-resource Tibetan synthesis [15]. Most of these studies evaluate the forward mapping from a target dialect label to the intended dialect. We examine the complementary failure mode: whether a conflicting reference condition pulls the output back toward the source dialect or language.

2.3. Reference Leakage and Speaker–Language Entanglement

Speaker embeddings, prompt prefill, and local acoustic context can all encode language, dialect, prosody, and content in addition to speaker identity. AccentBox quantified speaker–accent entanglement and accent hallucination [16]. CrossAccent-TTS, language-agnostic speaker embeddings, CrossSpeech, and mixture-of-language-experts models have addressed speaker–language or speaker–accent coupling through representation disentanglement, acoustic-path separation, and expert routing [17,18,19,20]. Large cross-lingual systems such as Qwen3-TTS and CosyVoice 3 also report accent drift as a distinct evaluation dimension [21,22]. Our contribution is therefore not the first identification of accent leakage. Instead, we operationalize category leakage for Tibetan dialect identity and audit its pathways in the continuous flow-matching conditioning stack of dots.tts.

2.4. Guided Generation and Representation Disentanglement

PPLM, Guided-TTS, Universal Guidance, FreeDoM, and diffusion posterior sampling demonstrate that pretrained discriminators can guide generation without retraining the base generator [23,24,25,26,27]. Our $\mathrm{CGD}$ implementation is an instance of this broader family rather than a new guidance paradigm. It defines a contrastive target-minus-source objective on a closed-form clean-latent estimate and constrains each update with proximal restoration and trust-region clipping. Domain-adversarial learning and the variational information bottleneck provide related motivations for suppressing domain information in speaker representations [28,29]. These methods motivate the x-vector re-pairing and inference-time guidance experiments, but they do not imply that either intervention must succeed in this setting.

3. Problem Formulation and Methods

3.1. Task Definition

Let $v_r$ denote the source variety of a reference utterance and $v_t$ the variety requested by the target text. The five varieties are Amdo Tibetan (bo-amdo), U-Tsang Tibetan (bo-utsang), Kham Tibetan (bo-kham), standard Mandarin Chinese (zh-standard), and standard English (en-standard). Given reference speech r and target text $x_t$, the system generates

$$\widehat{s}=G(r,x_t,v_t).$$

(1)

An intended output should preserve the speaker timbre in r, render the text intelligibly, conform to $v_t$, and avoid transferring irrelevant source-language, source-dialect, or prosodic attributes.

For a cross-lingual set $S_{\mathrm{lang}}$ and a Tibetan cross-dialect set $S_{\mathrm{dial}}$, the categorical leakage rates are

$$\mathrm{O}\text{-}\mathrm{ALR}={\displaystyle \frac{1}{|S_{\mathrm{lang}}|}}\sum _{i\in S_{\mathrm{lang}}}\mathbf{1}[C_{\mathrm{lang}}\left({\widehat{s}}_i\right)=C_{\mathrm{lang}}\left(r_i\right)],$$

(2)

$$\mathrm{O}\text{-}\mathrm{DLR}={\displaystyle \frac{1}{|S_{\mathrm{dial}}|}}\sum _{i\in S_{\mathrm{dial}}}\mathbf{1}[C_{\mathrm{dial}}\left({\widehat{s}}_i\right)=C_{\mathrm{dial}}\left(r_i\right)],$$

(3)

where C is a frozen waveform-level five-class variety classifier. Lower values indicate that fewer outputs are assigned to the reference category. These metrics are auxiliary proxies and should not be equated with a listener’s judgment of residual accent.

3.2. Base Model and Conditioning Paths

dots.tts processes Tibetan Unicode text and uses three target-dialect tokens: <|dialect_amdo|>, <|dialect_utsang|>, and <|dialect_kham|>. Speaker timbre and gender are conditioned by a reference x-vector. Because most training data lack explicit speaker labels, timbre and gender are uncontrolled when no reference speech is supplied.

The complete reference-cloning condition, denoted Full, uses both the speaker x-vector and prompt latent prefill. Prompt prefill introduces local reference context into generation and can carry source language, dialect, and prosody. The x-vector-only condition disables prompt latent prefill while retaining the global speaker condition. It is a diagnostic baseline rather than a “pure timbre” representation because the x-vector may itself encode dialect information.

Figure 1 summarizes the evidence chain. Path ablation locates candidate conflict pathways, $\mathrm{RXP}$ must be interpreted against a budget-matched continuation, and prompt gating and $\mathrm{CGD}$ define inference-time failure boundaries. The three components are not presented as a unified, validated disentanglement method.

3.3. Reference X-Vector Re-Pairing

During $\mathrm{RXP}$ training, the target sample’s text, language/dialect control token, and acoustic latent $y_i$ remain unchanged, while its reference x-vector is replaced by that of sample j. When i and j belong to different varieties, the model must derive target pronunciation from the text-side condition while retaining speaker information from the reference side. This intervention is intended to strengthen conditional role separation; it does not explicitly remove dialect information from the x-vector.

We evaluate $\mathrm{RXP}$-Full and $\mathrm{RXP}$-x-vector-only together with Base-Full and Base-x-vector-only in an initial $2\times 2$ checkpoint comparison. Of 995,000 training rows, 149,062 (14.98%) were assigned cross-speaker, cross-variety x-vectors using pairing seed 48. To isolate training-budget confounding, we also trained a matched Base continuation from the same Base@3000 checkpoint using the ordinary manifest. The matched Base and $\mathrm{RXP}$ continuations share training seed 47, learning rate, validation set, and a 2,000-step budget; both are compared at step 1400. Training attribution therefore relies on the matched continuation, whereas the initial $2\times 2$ comparison remains diagnostic. Full training and generation settings are provided in Appendix B.

3.4. Classifier-Guided Decoding

For the flow-matching parameterization $x_t=t{x}_1+(1-t)x_0$, the clean latent estimate from current latent z and predicted velocity $v_{\theta }(z,t)$ is

$${\widehat{x}}_1=z+(1-t)v_{\theta }(z,t).$$

(4)

Let $q_{\varphi }(v\mid {\widehat{x}}_1)$ be a latent-space five-class classifier. Given target variety $v_t$ and reference variety $v_r$, the contrastive guidance objective is

$$L\left({\widehat{x}}_1\right)=log{q}_{\varphi }(v_t\mid {\widehat{x}}_1)-\alpha log{q}_{\varphi }(v_r\mid {\widehat{x}}_1).$$

(5)

Within $t\in [0.3,0.8]$, gradients of L are normalized by their root mean square and clipped to trust-region radius $\delta$. If ${\widehat{x}}_1^0$ denotes the unmodified anchor and $\Delta$ the proposed displacement, proximal restoration gives

$${\widehat{x}}_1^{\prime }={\widehat{x}}_1^0+{\displaystyle \frac{\Delta }{1+\gamma }},v^{\prime }={\displaystyle \frac{{\widehat{x}}_1^{\prime }-z}{1-t}}.$$

(6)

We fix $\eta =1$, $\alpha =1$, and $\gamma =1$ and scan $\delta \in \{2,6,12,20\}$. At $\delta =20$, an $\alpha =0$ target-only control and an equal-norm random-direction control are also evaluated. $\mathrm{CGD}$ is used to characterize an inference-time failure boundary, not as part of a validated intervention package.

4. Experimental Setup and Evaluation Protocol

4.1. Route-Balanced Evaluation Matrix

The frozen display matrix contains 147 reference–target rows spanning Tibetan cross-dialect, cross-lingual, and matched-variety controls. Each of the five varieties contributes one fixed reference prompt, yielding five reference-speaker instances in total. Because the same reference prompt, target variety, and target text can occur in more than one matrix file, rows are deduplicated before analysis using stable reference-prompt identity, reference and target variety, and target-text ID. This procedure produces 120 unique conditions. The resulting denominators are 36 for $\mathrm{O}\text{-}\mathrm{DLR}$, 42 for $\mathrm{O}\text{-}\mathrm{ALR}$, 75 for target-variety accuracy, and 120 for $\mathrm{SECS}$. Thus, 147 is the number of displayed rows, not a common statistical denominator. Table 1 details the matrix composition.

The initial checkpoint comparison comprises Base-Full, Base-x-vector-only, $\mathrm{RXP}$-Full, and $\mathrm{RXP}$-x-vector-only. The primary training attribution compares matched Base@1400 with $\mathrm{RXP}$@1400. Both continue from the same Base@3000 checkpoint with identical training budget, seed, validation settings, and decoding configuration. Their manifests are independently deduplicated and evaluated on 120 shared keys. $\mathrm{CGD}$ systems follow the same rule. To reduce listening burden, ordinary same-language and same-dialect text and code-mixed text are excluded from the subjective comparison, and only one synthesized text is retained for each reference–target condition.

4.2. Objective Metrics and Statistical Analysis

The waveform-level five-class evaluator was trained only on labeled natural speech; generated speech was never used for training. An audit of the earlier v1 evaluator revealed extensive speaker overlap among training, validation, and test partitions. Its accuracy/macro-F1 of 0.9775/0.9776 is therefore removed from the final evidence chain. The v2 speaker-disjoint manifests contain 69,424 training, 9,125 validation, and 9,008 test utterances. Speaker overlap and aggregate-speaker rows are zero, although seven data sources remain shared across partitions. The split is therefore speaker-disjoint but not source- or channel-disjoint.

Three evaluators were trained with seeds 20260821–20260823 using class-weighted loss. Each checkpoint was selected only by validation macro-F1, with validation loss as a tie-breaker. The final predictor averages class probabilities across the three models; no model was selected using the test set. As shown in Table 2, five-class test accuracy/macro-F1 is 0.837/0.774. After collapsing Tibetan dialects, language-level accuracy/macro-F1 is 0.978/0.976. Within the Tibetan subset, however, macro-F1 is 0.644 and Kham F1 is 0.330. We consequently treat $\mathrm{O}\text{-}\mathrm{ALR}$ as a comparatively reliable language-category proxy and $\mathrm{O}\text{-}\mathrm{DLR}$ as a noisier exploratory dialect proxy.

In addition to $\mathrm{O}\text{-}\mathrm{ALR}$ and $\mathrm{O}\text{-}\mathrm{DLR}$, we report target-variety accuracy, speaker-embedding cosine similarity ($\mathrm{SECS}$), Mandarin character error rate (CER), English word error rate (WER), and Tibetan CER. Mandarin and English errors are computed with Whisper large-v3, and Tibetan errors with a Sherpa Tibetan ASR system. Code-mixed samples are excluded because no reliable multilingual ASR was available for this setting.

Two-sided exact McNemar tests are retained for paired categorical outcomes but interpreted only as condition-level descriptive comparisons. The primary uncertainty interval for each paired difference is a crossed-cluster paired bootstrap that independently resamples reference prompts and target-text IDs. When an earlier Base manifest lacks reference_prompt_name, prompt identity is recovered from the portable filename in reference_wav; missing prompts are not pooled into a single cluster. We use 10,000 bootstrap replicates and report 95% intervals. The total matrix contains only five prompt clusters and the cross-dialect subset only three, so power along the reference-prompt axis is low. Furthermore, the resolution of $\mathrm{O}\text{-}\mathrm{DLR}$, $\mathrm{O}\text{-}\mathrm{ALR}$, and target accuracy is $1/36$, $1/42$, and $1/75$, respectively. Small directional changes should not be described as method success.

4.3. Exploratory Native-Listener Evaluation

The listening interface presents anonymized paired Base-Full and $\mathrm{RXP}$-Full outputs. Each task contains a reference X and anonymous outputs A and B. The listener evaluates target language/dialect control and reference-side leakage, speaker-timbre similarity to X, and naturalness and clarity. “Tie” and “cannot judge” responses are permitted. Optional issue labels include reference-side leakage, incorrect target, content or pronunciation error, noise, distortion, and timbre mismatch. This paired task is not a classical ABX design and is not described as such.

The frozen set contains 20 tasks: 14 cross-lingual and six cross-dialect comparisons. One native Tibetan listener completed all tasks. These data are used only for exploratory error analysis and cannot support group-level significance or claims about a listener population. A confirmatory study should determine sample size through prospective power analysis, recruit listeners across relevant dialect backgrounds, and report dialect background and long-term region of residence. The frozen response summary is reported in Appendix C.

5. Results

5.1. Pathway Diagnosis Differs between Language and Dialect Leakage

The v2 ensemble reveals a comparatively large cross-lingual path difference within the frozen conditions. Across the 120 unique conditions, Base $\mathrm{O}\text{-}\mathrm{ALR}$ decreases from 0.262 under Full conditioning to 0.095 under x-vector-only conditioning. For the Full-minus-x-vector-only contrast, the crossed-cluster 95% confidence interval is $[-0.081,0.450]$. Target-variety accuracy increases from 0.493 to 0.600 when prefill is disabled; expressed as Full minus x-vector-only, its interval is $[-0.246,0.018]$. $\mathrm{SECS}$ changes from 0.850 to 0.856, with an interval of $[-0.031,0.017]$. By contrast, $\mathrm{O}\text{-}\mathrm{DLR}$ remains 0.222 in both conditions, with an interval of $[-0.111,0.111]$ (Table 3; Figure 3). Prompt latent prefill is therefore a candidate pathway for cross-lingual conflict on these fixed prompts, but the limited prompt coverage does not support generalization across reference speakers or claims about dialectal or continuous accent transfer.

Route-level counts show that this effect is heterogeneous. English-reference-to-Tibetan synthesis forms the most concentrated language-category leakage cluster for Base-Full, and this cluster contracts when prefill is disabled (Figure 2). Tibetan-reference-to-Mandarin routes do not show the same consistent direction. The aggregate $\mathrm{O}\text{-}\mathrm{ALR}$ difference should therefore be used to generate route-specific hypotheses, not to infer a universal mechanism.

The x-vector analysis provides complementary evidence. A linear classifier predicts Tibetan dialect from x-vectors extracted from natural speech with 0.672 accuracy under metadata-group-disjoint splitting, compared with a majority baseline of approximately 0.530 and a uniform-random baseline of 0.333. Accuracy under a random utterance split reaches 0.844, indicating additional speaker memorization (Table 4; Figure 4). Kham has only 25 metadata speaker groups, and a small number of groups represent aggregate or unknown identities. Source and recording channel are not isolated. The result therefore shows residual dialect decodability in the available representation, but it is not an unconfounded estimate of dialect phonology under fully verified speaker and channel separation.

Figure 2. Cross-lingual reference-category leakage by route for the Base system. (a) Full reference path. (b) X-vector-only path. Each valid cell reports the number of outputs classified as the reference language divided by the route denominator. Color normalization from 0 to 1 is shared across panels; gray dashes denote within-language routes or routes excluded from cross-lingual analysis. English-reference-to-Tibetan synthesis is the most concentrated Full-condition failure cluster, but other routes do not change uniformly.

Figure 2. Cross-lingual reference-category leakage by route for the Base system. (a) Full reference path. (b) X-vector-only path. Each valid cell reports the number of outputs classified as the reference language divided by the route denominator. Color normalization from 0 to 1 is shared across panels; gray dashes denote within-language routes or routes excluded from cross-lingual analysis. English-reference-to-Tibetan synthesis is the most concentrated Full-condition failure cluster, but other routes do not change uniformly.

Figure 3. Paired Full versus x-vector-only pathway effects. (a) Leakage metrics are expressed as Full minus x-vector-only, so positive values indicate more leakage with prefill. (b) Target accuracy and $\mathrm{SECS}$ are expressed as x-vector-only minus Full, so positive values favor prefill removal. Filled circles/solid lines denote Base; open squares/orange lines denote $\mathrm{RXP}$. Error bars are crossed-cluster 95% bootstrap intervals from 10,000 independent resamples of reference prompts and target-text IDs. Their width and overlap with zero restrict generalization.

Figure 3. Paired Full versus x-vector-only pathway effects. (a) Leakage metrics are expressed as Full minus x-vector-only, so positive values indicate more leakage with prefill. (b) Target accuracy and $\mathrm{SECS}$ are expressed as x-vector-only minus Full, so positive values favor prefill removal. Filled circles/solid lines denote Base; open squares/orange lines denote $\mathrm{RXP}$. Error bars are crossed-cluster 95% bootstrap intervals from 10,000 independent resamples of reference prompts and target-text IDs. Their width and overlap with zero restrict generalization.

Table 4. Linear probing of dialect from x-vectors. The uniform-random baseline is 0.333 and the test-set majority baseline is approximately 0.530. The primary split is disjoint by available speaker metadata but contains a small number of aggregate or unknown identities and does not isolate source or channel.

Split	Test accuracy	Macro-F1	Interpretation
Random utterance split	0.844	0.810	Training and test share speakers, allowing speaker memorization
Metadata-group-disjoint split	0.672	0.598	Dialect decodability across available speaker groups, with residual identity, source, and channel confounding

Table 4. Linear probing of dialect from x-vectors. The uniform-random baseline is 0.333 and the test-set majority baseline is approximately 0.530. The primary split is disjoint by available speaker metadata but contains a small number of aggregate or unknown identities and does not isolate source or channel.

Split	Test accuracy	Macro-F1	Interpretation
Random utterance split	0.844	0.810	Training and test share speakers, allowing speaker memorization
Metadata-group-disjoint split	0.672	0.598	Dialect decodability across available speaker groups, with residual identity, source, and channel confounding

Figure 4. Frozen t-SNE projection of speaker x-vectors extracted from natural Tibetan speech. Color and marker jointly identify Amdo ($n=2,928$), U-Tsang ($n=4,200$), and Kham ($n=750$). Partial clustering is visible, but the descriptive projection combines dialect, speaker, recording-channel, and collection-batch effects and cannot establish causal separability of dialect from timbre.

Figure 4. Frozen t-SNE projection of speaker x-vectors extracted from natural Tibetan speech. Color and marker jointly identify Amdo ($n=2,928$), U-Tsang ($n=4,200$), and Kham ($n=750$). Partial clustering is visible, but the descriptive projection combines dialect, speaker, recording-channel, and collection-batch effects and cannot establish causal separability of dialect from timbre.

5.2. RXP Does Not Show a Stable Matched-Control Benefit

The initial $2\times 2$ checkpoint comparison is summarized in Table 5. Relative to Base-Full, $\mathrm{RXP}$-Full changes $\mathrm{O}\text{-}\mathrm{ALR}$ from 0.262 to 0.214, $\mathrm{O}\text{-}\mathrm{DLR}$ from 0.222 to 0.194, target accuracy from 0.493 to 0.560, and $\mathrm{SECS}$ from 0.850 to 0.846. For Base minus $\mathrm{RXP}$, the crossed-cluster intervals for $\mathrm{O}\text{-}\mathrm{ALR}$ and $\mathrm{O}\text{-}\mathrm{DLR}$ are $[-0.098,0.200]$ and $[-0.083,0.167]$, respectively; the target-accuracy interval is $[-0.182,0.035]$. These checkpoints do not match training budget, and all intervals cross zero. Their directional differences cannot be attributed to $\mathrm{RXP}$ alone.

The budget-matched comparison addresses this confound. Matched Base and $\mathrm{RXP}$ continue from the same Base@3000 checkpoint with the same seed, learning rate, validation set, and 2,000-step budget. They are paired on 120 shared unique conditions at step 1400 (Table 6; Figure 5). Under Full conditioning, matched Base and $\mathrm{RXP}$ yield $\mathrm{O}\text{-}\mathrm{ALR}$ values of 0.190 and 0.214 and target accuracies of 0.547 and 0.560; both difference intervals cross zero. Their $\mathrm{O}\text{-}\mathrm{DLR}$ values are 0.278 and 0.194, and the Base-minus-$\mathrm{RXP}$ interval has a lower bound of zero. However, this proxy has a Tibetan-subset macro-F1 of only 0.644 and only three reference-prompt clusters, precluding a robust training claim. Under x-vector-only conditioning, intervals for $\mathrm{O}\text{-}\mathrm{ALR}$, $\mathrm{O}\text{-}\mathrm{DLR}$, and target accuracy also cross zero. The $\mathrm{SECS}$ contrast favors matched Base by 0.020 (95% CI $[0.007,0.045]$). The matched analysis therefore provides no generalizable control benefit for $\mathrm{RXP}$.

ASR safeguards do not change this interpretation (Table 7). Under Full conditioning, Tibetan CER is 0.092 for matched Base and 0.099 for $\mathrm{RXP}$, Mandarin CER is 0.074 and 0.030, and English WER is 0.041 and 0.024. All three paired intervals cross zero. Under x-vector-only conditioning, Tibetan CER is 0.085 and 0.071, with an interval that also crosses zero; Mandarin CER is identical, and the absolute English WER difference is 0.006. The ASR results neither show a broad intelligibility difference nor justify interpreting the earlier $\mathrm{RXP}$ trend as a general quality improvement.

Aggregate comparisons obscure route-specific failures. Under v2 scoring, English-reference-to-Tibetan leakage occurs in 6/9 Base-Full outputs ($\mathrm{O}\text{-}\mathrm{ALR}$=0.667) and 5/9 $\mathrm{RXP}$-Full outputs (0.556), compared with 1/9 and 2/9 for their x-vector-only counterparts. The nine-item Base-versus-$\mathrm{RXP}$ difference is unstable, whereas the path-ablation change is larger. $\mathrm{RXP}$ is consequently retained as an exploratory training branch rather than a general solution.

5.3. Fixed-Ratio Prompt Gating Fails the Prespecified Criteria

We tested whether simple prompt attenuation could repair the concentrated English-reference-to-Tibetan conflict. Prompt-prefill scales of 0.1, 0.2, 0.4, and 0.7 were evaluated on nine routes for both checkpoints (Table 8; Figure 6). None of the eight intermediate conditions met all prespecified criteria. At scale 0.2, both Base and $\mathrm{RXP}$ achieved 6/9 Tibetan-language hits, but $\mathrm{SECS}$ fell to 0.761 and 0.749 and Tibetan CER was 0.175 and 0.207, respectively. At scale 0.1, every output from both checkpoints was still classified as English. Fixed-ratio gating is therefore a negative result and was not expanded to the entire 120-condition matrix.

5.4. CGD Reduces Fidelity Without Improving Dialect Leakage

Under the v2 evaluator, $\mathrm{CGD}$ does not improve dialect leakage (Table 9). The no-guidance $\mathrm{O}\text{-}\mathrm{DLR}$ is 0.194, remains 0.194 at $\delta =2$ and 6, and rises to 0.222 at $\delta =12$ and 20. Strong guidance reduces $\mathrm{O}\text{-}\mathrm{ALR}$ to 0.048 but does not improve overall target accuracy; it also reduces $\mathrm{SECS}$ to 0.602 and raises Tibetan CER to 0.207. This operating point is not a usable low-leakage, high-fidelity solution.

Output destinations reinforce this conclusion (Table 10). At $\delta =20$, 0.36 of the 36 unique cross-dialect outputs are classified as the target dialect, 0.22 as the reference dialect, and 0.25 as non-Tibetan. The corresponding no-guidance proportions are 0.36, 0.19, and 0.31. The low $\mathrm{O}\text{-}\mathrm{DLR}$ previously reported by v1 is not reproduced by v2, indicating that the earlier interpretation depended on an invalid evaluator. Figure 7 shows extensive overlap among $\mathrm{O}\text{-}\mathrm{DLR}$ intervals and a marked $\mathrm{SECS}$ reduction at $\delta =20$.

Controlled ablations at $\delta =20$ also fail to establish a benefit (Table 11). The equal-norm random direction leaves $\mathrm{O}\text{-}\mathrm{DLR}$ at 0.194, removing the source penalty ($\alpha =0$) changes it to 0.250, and full $\mathrm{CGD}$ yields 0.222. None improves target-dialect hits. Full $\mathrm{CGD}$ primarily lowers $\mathrm{O}\text{-}\mathrm{ALR}$, reduces $\mathrm{SECS}$, and degrades Tibetan CER. The evidence therefore does not support the claim that $\mathrm{CGD}$ reliably reduces leakage while preserving timbre and intelligibility.

Automatic intelligibility results are reported in Table 12. Tibetan CER remains close to the no-guidance condition at moderate guidance strengths but approximately doubles at $\delta =20$. Kham has the highest CER in every system. With the same Sherpa recognizer, median robust CER on held-out natural speech is 0.020 for Amdo, 0.082 for U-Tsang, and 0.035 for Kham. The synthetic Kham errors are therefore not explained by a simple recognizer bias against natural Kham speech. This inference remains limited by the single ASR system, a small number of audio–text alignment errors, and the exclusion of code-mixed text.

5.5. Exploratory Listener Results

In the 20 anonymized paired tasks completed by one native Tibetan listener, $\mathrm{RXP}$ was preferred in six cases, Base in seven, and seven were tied for target control/reference leakage. For naturalness and clarity, $\mathrm{RXP}$ was preferred in five cases, Base in seven, and eight were tied. All 20 speaker-timbre comparisons were tied. The English-reference-to-Amdo, Kham, and U-Tsang tasks were all annotated as showing similarly clear leakage in both systems; the Kham and U-Tsang cases were specifically described as having a severe English accent. This observation agrees with the objective route-level failure cluster but cannot support significance testing or population-level perceptual conclusions.

6. Discussion

6.1. Synthesis Capability and Reference-Side Leakage Are Distinct

Matched-language and matched-dialect controls and monolingual ASR scores serve only as capability safeguards in this study. A low CER or WER indicates that content generation has not failed globally, but it does not establish that the requested dialect is correct. A conflicting reference can still alter the categorical language assignment or leave a fine-grained accent in otherwise intelligible speech. Evaluation of cross-lingual voice cloning should therefore separate the ability to produce a language from the ability to resist reference-side interference. Naturalness and speaker similarity alone cannot resolve this distinction.

6.2. Interpretation of the Conditioning-Path Evidence

The v2 path ablation provides a bounded conditioning-path hypothesis. On the Base checkpoint and five fixed reference prompts, disabling prompt latent prefill produces a comparatively large point-estimate reduction in language-level leakage and an increase in target-variety accuracy, while the dialect-level point estimate remains unchanged. Once the correct prompt identity is restored in the crossed-cluster bootstrap, however, the intervals cross zero. The analysis therefore prioritizes prompt latent prefill for follow-up work but does not establish an effect across reference speakers.

Cross-dialect conflict cannot be explained solely by prompt prefill. Dialect remains linearly decodable from the x-vector under the available metadata-group-disjoint split, although speaker, source, and channel confounding have not been fully removed. The initial $2\times 2$ checkpoint comparison is also confounded by training budget. A matched continuation does not reproduce stable $\mathrm{O}\text{-}\mathrm{ALR}$ or target-accuracy gains for $\mathrm{RXP}$, and matched Base preserves higher $\mathrm{SECS}$ under x-vector-only generation. Reference re-pairing should therefore remain an exploratory training direction rather than be presented as a validated disentanglement method.

The $\mathrm{CGD}$ scan yields no operating point that jointly improves leakage, $\mathrm{SECS}$, and CER. More importantly, v2 does not reproduce the earlier low-$\mathrm{O}\text{-}\mathrm{DLR}$ result at $\delta =20$. Strong guidance neither reduces $\mathrm{O}\text{-}\mathrm{DLR}$ nor increases target-dialect hits and instead damages speaker similarity and intelligibility. Random-direction and target-only controls also fail to improve dialect control. The previous mechanistic account based on the invalid v1 evaluator must therefore be withdrawn. The result that remains stable across evaluator revisions is that strong guidance degrades generation without providing a control benefit.

6.3. Measurement Validity

The v1 evaluator failed a speaker-split audit, so its high accuracy provides no validity evidence. The v2 evaluator is speaker-disjoint but shares data sources and potential recording channels across partitions. It remains an objective proxy rather than perceptual ground truth. Its language-level macro-F1 is 0.976, whereas macro-F1 within Tibetan is 0.644 and Kham F1 is 0.330. Consequently, evidence from $\mathrm{O}\text{-}\mathrm{ALR}$ is stronger than evidence from $\mathrm{O}\text{-}\mathrm{DLR}$.

Additional measurement risks remain. First, non-Tibetan generations count as non-reference dialects in the $\mathrm{O}\text{-}\mathrm{DLR}$ denominator, so generation failure can reduce leakage without producing the target dialect. Output destinations must therefore accompany $\mathrm{O}\text{-}\mathrm{DLR}$. Second, the latent classifier used for $\mathrm{CGD}$ and the waveform evaluator share five-class labels and natural-speech data distributions. Their different architectures and input domains do not eliminate a same-family Goodhart risk. Third, five-class assignment measures discrete identity rather than continuous accent strength, prosody, or native-listener naturalness. Fourth, Whisper large-v3 language identification exhibits a floor effect on synthetic Tibetan and cannot serve as an independent discriminator among systems. Confirmatory claims require an evaluator with independent features and architecture and a sufficiently powered native-listener study.

6.4. Limitations and Research Priorities

The principal limitations define the next experimental priorities:

1.

Only one native Tibetan listener completed the 20 exploratory comparisons. A confirmatory study should define a minimum meaningful effect and power level in advance, determine the corresponding listener sample size, and stratify results by dialect background and long-term region of residence.

2.

Although v2 includes three frozen seeds and 1,752 rescored generated utterances, shared data sources and weak Kham performance limit dialect-level validity. A source- or channel-disjoint external test set and calibrated dialect posteriors are needed.

3.

The objective matrix uses one reference prompt per variety. Repeated target texts do not compensate for a lack of independent reference speakers. Expanding the prompt axis is the most direct way to improve pathway attribution.

4.

Strictly paired ASR comparisons include only 45 Tibetan, 15 Mandarin, and 15 English samples per mode. Their intervals remain wide, and code-mixed intelligibility is unmeasured.

5.

Kham is under-represented in evaluator performance, x-vector speaker groups, and synthesized-speech CER. Follow-up data collection should prioritize verified speakers, broader text coverage, and independent recording batches for Kham.

6.

The x-vector probe uses available speaker metadata, including a small number of aggregate or unknown identities, and does not separate channel or collection batch. Identity cleaning and cross-channel, cross-batch evaluation are required before interpreting dialect decodability as a speaker-independent property.

7.

$\mathrm{CGD}$ relies on a closed-form clean latent in continuous flow matching and does not directly transfer to autoregressive systems with discrete codec tokens. $\mathrm{RXP}$ also requires reliable target-side language or dialect control symbols.

The most concentrated failure route is English-reference-to-Tibetan synthesis. Future intervention work should prioritize explicit target-side conditioning, staged prompt injection, or a new training objective for this route. The fixed-ratio prompt-gating scan indicates that simply increasing the number of gating ratios is unlikely to be productive.

6.5. Reproducibility and Asset Availability

Generation manifests, deduplication keys, evaluator manifests and checkpoints, per-utterance predictions, statistical scripts, and report hashes have been frozen internally. Submission materials should include, at minimum, evaluation manifests, metric and statistical code, v2 evaluator checkpoints, and anonymized per-utterance predictions so that the reported tables and figures can be recomputed without access to the original training recordings. Release of the base model, $\mathrm{RXP}$ checkpoint, generated speech, and Tibetan ASR resources is subject to upstream licenses, recording consent, speaker privacy, and service availability. Until these assets pass review and are actually released, the work should be described as internally traceable rather than fully reproducible by third parties.

6.6. Ethical and Social Considerations

Generating arbitrary content in a person’s voice and a specified dialect creates risks of impersonation, fraud, and fabricated speech attributed to minority communities. Release decisions should document recording licenses and informed consent, remove identifying metadata where possible, and consider watermarking, synthetic-speech detection, and controlled access. The x-vector probe also indicates that a speaker representation may expose dialect or regional attributes, creating a privacy risk independent of waveform release. Because Kham is under-represented, equal service quality across the three Tibetan dialects should not be claimed. These safeguards are recommendations; they are not described as protections already deployed in this work.

7. Conclusions

This study examined reference-side language and dialect category leakage in cross-lingual voice cloning for Amdo, U-Tsang, and Kham Tibetan, Mandarin Chinese, and English. A 147-row display matrix was deduplicated into 120 unique analytical conditions, and an earlier evaluator was removed after a speaker-overlap audit. The replacement three-seed, speaker-disjoint ensemble achieved a language-level test macro-F1 of 0.976 but only 0.644 within the three Tibetan dialects. The evidence strength of language-level $\mathrm{O}\text{-}\mathrm{ALR}$ and dialect-level $\mathrm{O}\text{-}\mathrm{DLR}$ was therefore treated separately.

On the five frozen reference prompts, disabling prompt latent prefill reduced Base $\mathrm{O}\text{-}\mathrm{ALR}$ from 0.262 to 0.095 while $\mathrm{O}\text{-}\mathrm{DLR}$ remained 0.222. The crossed-cluster interval for the $\mathrm{O}\text{-}\mathrm{ALR}$ difference crossed zero, so this result identifies a priority pathway for further investigation rather than a general effect across reference speakers. A metadata-group-disjoint x-vector probe retained 0.672 dialect-classification accuracy, but unresolved identity, source, and channel confounds make this a representation diagnostic rather than causal evidence.

A continuation matched for initialization and training budget did not establish a stable independent benefit from reference x-vector re-pairing. Fixed-ratio prompt gating found no point that simultaneously met target-control, speaker-similarity, and Tibetan-intelligibility criteria. Classifier-guided decoding did not reduce v2 dialect leakage, and strong guidance only degraded $\mathrm{SECS}$ and CER. The current evidence therefore supports an audit and failure-analysis contribution, not a generally successful disentanglement method.

The resulting evidence chain includes unique-condition accounting, evaluator-split auditing, three-seed rescoring, a representation probe, a budget-matched training control, and intervention failure-boundary analysis. The 20-item evaluation by one native Tibetan listener did not show an overall system preference and cannot replace a group listening study. The immediate research priorities are to expand independent reference prompts, strengthen source- and channel-independent dialect evaluation, and test explicit target-side conditioning for the concentrated English-reference-to-Tibetan failure route.