Measuring Cross-Embodiment Transfer Without Fooling Yourself

1. Introduction

Reusing behavior across robots of different morphologies is among the defining ambitions of generalist robot learning, and the literature is full of plausible quantitative claims about it [1,2,3,4]. Similar morphologies are said to transfer better, a learned transferability oracle is said to pick the right source robot, a diverse pretraining set is said to generalize to new bodies, a learned embedding is said to be morphology-invariant, and a cheap probe is said to predict the data budget a new robot needs. Each of these is the kind of claim a careful laboratory wants to make. Each is also unusually easy to make incorrectly, not through fabrication but through a small set of confounds that survive ordinary peer review. This paper is about those confounds and how to stop them.

The argument has three connected parts. The first is that cross-embodiment transfer claims have a characteristic failure set. Eight confounds recur, namely a class prior masquerading as a morphology effect, pseudo-replicated seeds tightening an interval, a missing trivial baseline that already explains the outcome, a weak linear probe certifying an invariance that a strong probe falsifies, diversity conflated with data volume, degenerate or leaky prediction targets, an appearance confound in identity probes on natural multi-robot data, and episode leakage in per-frame metrics on episodic data. These are not exotic, and none of them is caught by counting rollouts or by tightening a significance threshold. The second part is that an enforced standard catches them. This paper defines eight checks, each operationalized by a tool that runs, most centrally an acceptance gate that recomputes a claim’s metric, runs its strongest baseline, bootstraps the difference over independent units, and emits a pass or fail verdict. A number that does not pass does not ship as a positive; it is reframed as the negative the data support. The standard is enforced in the literal sense that it is code in the loop, not a checklist in a reviewer’s memory. The third part is that the most convincing evidence a standard of this kind works is a worked record of it correcting real claims. The heart of the paper is therefore a set of eleven documented failure modes, each drawn from a real cross-embodiment research program and each stated as the claim it was tempting to make, the diagnostic that exposed the confound, the corrected claim the data support, and the check that catches it, with every number traced to a released artifact.

The standard’s animating principle is a single sentence. A cross-embodiment transfer claim is credible only if it beats its strongest trivial alternative, whether a class bit, a target-only prior, a from-scratch baseline, a morphology distance, or a within-class correlation, by a margin whose bootstrap 95% confidence interval excludes that alternative, computed on independent units, on a task that demonstrably depends on morphology, with any invariance certified by a strong nonlinear probe and any diversity claim made at a fixed data budget. Everything that follows is the operational content of that sentence.

The paper makes four contributions. It identifies the confound set specific to cross-embodiment transfer claims and shows that each confound is a distinct failure mode rather than a variant of a single mistake (Section 3). It defines eight checks and an enforced acceptance gate that operationalize the strongest-baseline-with-a-confidence-interval-on-independent-units rule, and explains the two-tier decision rule the gate applies (Section 3). It reports eleven documented failure-mode case studies, each traced to a committed artifact, that together exercise all eight checks (Section 4). And it positions the cross-embodiment confound set within the broader 2025–2026 evaluation-rigor movement, making precise what the gate adds that rollout-level statistical protocols do not (Section 5). The eight checks, the gate, the eleven case artifacts, and the analysis code that regenerates every number in this paper are released as a public repository so that other groups can adopt the same discipline.

This is a methodology and standards paper rather than a new transfer result. The empirical program that produced the eleven cases lives in three companion papers, and the division of labor is clean. A companion benchmark paper establishes that neither morphological similarity nor data diversity governs frozen-core transfer and introduces the gate as infrastructure supporting its negatives [5]. A companion theory paper proves that any continuous task representation sufficient for control retains body-identifying structure [6]. A companion architecture paper constructs a coarse symbolic interface that passes the same gate every similarity-keyed lever fails [7]. The present paper is the standalone treatment of the standard as the object of study, and it does not re-count the companions’ empirical claims as results of its own.

2. Related Work

A strong contemporary movement toward statistical rigor in robot-policy evaluation motivates and corroborates this work. Toyota Research Institute’s careful examination of large behavior models evaluates manipulation policies with explicit statistical confidence, using blind randomized trials, a sequential hypothesis-testing framework, and thousands of rollouts, precisely because casual evaluation overstates results [8]. RoboArena makes real-world generalist-policy evaluation distributed and double-blind over policy pairs rather than fixed-task leaderboards [9], and REALM adds a real-to-sim validated generalization benchmark [10]. Reproducible real-robot benchmarks and distributional evaluation methodologies extend the same discipline to vision-language-action models [11,12], and the retrospective lesson that convenient offline metrics correlate poorly with on-robot success echoes the earlier reinforcement-learning reproducibility literature [13,14]. This wave is largely about evaluating a policy’s task success rigorously; the present standard is complementary and targets the confounds specific to claims about what transfers across bodies.

The confounds this paper enumerates are instances of well-studied phenomena from adjacent fields. Shortcut learning is the general tendency of a model to exploit exactly the structure an objective tried to remove [15]. Linear probes are a standard tool for reading structure out of representations [16], yet they understate recoverability, and control tasks and stronger probes are needed for a probing claim to mean anything [17,18]; the disentanglement literature made the same point when it showed that apparently clean representations hide structure only a capable model recovers [19]. Pseudoreplication, the treatment of correlated measurements as independent observations, has been recognized as a design error in the experimental sciences for decades [20]. The contribution here is to show that these general phenomena take specific and predictable forms in cross-embodiment transfer, and to enforce the corresponding controls in code.

The theory of invariant representations gives the deepest of the confounds a precise history. Adversarial invariance was popularized by domain-adversarial training [21], and a line of work then established that forcing a representation to be invariant to a domain label lower-bounds the joint error once the label distributions differ [22,23], complementing the classical divergence bounds for adaptation [24]. The companion theory paper is the cross-embodiment counterpart, in which the quantity that cannot be removed is body identity and the floor is a directly measurable property of realized behavior [6]. For the present paper the practical consequence is Check 2, that an invariance claim must be certified against a strong probe rather than a linear one.

The claims the standard governs come from the two dominant styles of multi-robot learning. Generalist policies absorb a new embodiment by mapping every robot into a shared action space, attaching per-embodiment adapters, and fine-tuning on the target [1,2,3,4], while a parallel line builds the body into the policy through modular and graph controllers [25,26], morphology-token transformers [27], morphology-inferring encoders [28], and unified multi-embodiment locomotion architectures [29], with recent morphology-structured attention reported to help a large model [30]. Benchmark evidence nonetheless finds such agents near zero success on out-of-distribution morphologies, with high train-test similarity not helping [31]. Whether embodiment breadth or raw data volume drives multi-embodiment pretraining is disputed, with locomotion work reporting breadth beating data on an axis that does not hold total training volume fixed [32] and volume-matched manipulation studies finding diversity optional [33]; Check 3 is the control that separates the two. Visual transfer learning does have cheap estimators of fine-tuning benefit [34,35], and the robotics analogues that motivate several of the case studies turn out to measure a robot-class prior. Finally, the zero-shot cross-embodiment manipulation systems of 2026 all delete body information from the interface between task reasoning and motor control rather than seeking similar bodies [36,37,38,39,40]; the standard here is the measurement discipline under which such interfaces can be compared, and the companion papers supply the constructive account [6,7].

3. The Measurement Standard

The standard is a set of eight checks, each stated as a rule and paired with a tool that enforces it. The first check is the backbone and the other seven close specific loopholes the backbone alone leaves open. Table 1 states all eight together with their enforcing tools, and the subsections that follow develop the gate and then the seven remaining checks.

3.1. The Acceptance Gate and the Class-Prior Control

Check 1 is an executable acceptance gate with a fixed rule. Every headline must beat its strongest applicable baseline, not a strawman, by a margin whose bootstrap 95% confidence interval excludes that baseline, computed on independent units, on a task that varies with morphology. The strongest baselines are the trivial alternatives a transfer claim must out-predict, namely the target-only prior that predicts the outcome from the target’s own features, the scratch-only predictor that uses the target’s from-scratch difficulty, the class bit that carries a single arm-or-not-arm indicator, the morphology-distance predictor itself, and the within-class correlation that asks whether the signal survives once the trivial class split is removed. For each claim the gate recomputes the metric, selects and runs the strongest of these baselines, bootstraps the difference of rank metrics by resampling units while holding the fitted model fixed, and writes a pass-or-fail row with the number of independent units recorded. It is the single source of truth, re-run before any headline ships, and a non-passing number is reframed rather than published as a positive.

Table 2 reports the gate applied to the nine headline claims of the research program, and Figure 1 plots the same nine as a forest of claim-minus-baseline gaps. Four claims pass. The five that do not are the reframed negatives of the program, and several of them reappear as case studies in Section 4. The gate applies a two-tier decision rule. At ordinary sample sizes a claim passes when the gap confidence interval excludes the baseline. At small sample sizes the percentile bootstrap is anti-conservative, so a passing claim must additionally clear an exact sign-flip permutation test at $p\le 0.05$; rows 7 through 9, each resting on five or six independent arms, are decided under this conjunction. Row 7 illustrates the rule sharply, since its gap interval $[+0.05,+0.28]$ excludes zero yet its exact sign-flip test returns $p=0.0625$ on five arms, so the claim is recorded as a suggestive failure rather than a pass. The distinction between a confidence interval that happens to exclude zero and a claim that survives the exact test at the sample size actually available is exactly the kind of loophole an enforced gate is built to close.

3.2. The Seven Loophole-Closing Checks

The remaining checks close loopholes that a naive application of the gate leaves open, and each is illustrated in full in Section 4. Check 2 governs invariance claims and requires that any assertion that a learned representation is morphology-invariant be certified against the strongest probe an adversary could field, at minimum a strong nonlinear probe. A linear probe reporting near-chance decodability is not evidence of invariance, because nonlinear probes routinely recover a body fingerprint the linear probe misses, and the certifier must be a fresh independent probe rather than the in-loop training adversary. The enforcing tool is a dual probe grid that reports both a linear logistic-regression accuracy and a strong random-forest accuracy for every cell of a capacity-by-adversary sweep, and certifies invariance only when the strong probe is near chance.

Check 3 governs diversity claims and requires that any assertion that more diverse pretraining helps hold the total data budget fixed before crediting diversity. Adding source bodies usually adds episodes, and without the control a data-volume effect masquerades as a diversity effect. The enforcing tool is a budget-by-breadth grid that splits a fixed number of episodes across more source bodies and reports breadth-minus-depth with a bootstrap interval at each budget. Check 4 governs interval computation and requires bootstrapping over independent units rather than pseudo-replicated unit-by-seed rows, because several seeds of one source-to-target pair are not several independent pairs and bootstrapping them directly understates the interval. The enforcing tool is the aggregation step built into the gate, which collapses to the independent unit before resampling.

Check 5 governs prediction targets and requires that a target such as samples-to-threshold or demos-to-threshold have real variation, because if most units sit at a floor a predictor’s apparent accuracy is mostly reproducing that floor. Check 6 governs the probe window and requires that a tiny-probe prediction end before the event it claims to predict, since a probe window that overlaps the threshold crossing is measuring rather than forecasting. Both are enforced by a tiny-probe budget-oracle protocol that reports the fraction of units at the floor, the count of genuine-extrapolation units, and conformal coverage and interval half-width against a constant-budget baseline. Check 7 governs identity probes on natural multi-robot data and requires running the identical probe on the raw input, because if heavily downsampled pixels already recover the identity then the probe is measuring scene appearance rather than the body. Check 8 governs per-frame metrics on episodic data and requires episode-blocked train and test splits, because a frame-level split lets a model memorize the episode its test frames came from and inflates the metric. A task-fidelity tool complements the gate by re-deriving, from the environments themselves, whether the task backing a claim actually varies with morphology, so that the loophole of a morphology-invariant task cannot silently support a morphology claim.

4. Eleven Documented Failure-Mode Case Studies

The eight checks were developed against real analyses, and the clearest way to convey both the confounds and the checks that catch them is to work through eleven cases in which a tempting claim met a diagnostic that changed it. Table 3 lists the eleven, each with the tempting claim, the check that catches it, the corrected claim, and the artifact that demonstrates it, and the prose below develops each in turn. Every case is stated in the same four-part form, and every number is drawn from a released artifact. The word overclaim is used throughout as a technical term for a claim that a check reduces or reverses.

4.1. Class Priors and Definitional Tautologies

The first cluster of cases are caught by the gate and its discipline of comparing against the strongest trivial alternative. A natural analysis of learnability reports that within a single quadruped topology a body’s from-scratch training difficulty correlates with its morphology at $\rho \approx 0.98$, the kind of clean number that launches a predictive science of learnability (case a). A pre-registered stance-height control applies the gate’s discipline to the mechanism. The correlation between a body’s settle height and its training area-under-curve is $0.965$ under a stance-relative fall threshold and $0.984$ under a fixed one, while the partial correlation of thigh length given stance height collapses to $0.054$. The fall-threshold change barely moves the number, so it is not a reward artifact; taller and longer-legged bodies are genuinely but trivially easier, because longer legs give a taller stance, a longer stride, a higher achievable velocity, and a higher reward. The corrected claim is that within a single topology learnability is almost entirely explained by stance geometry, so the $0.98$ is technically true and geometrically trivial, and the frontier-moving question is cross-topology learnability, where leg-length geometry cannot explain differences across body plans.

A closely related case concerns the strongest single claim a transfer study can make, that transfer is governed by target learnability with a correlation of $-0.81$ between from-scratch success and transfer gain (case b). Transfer gain is defined as success after transfer minus success from scratch, so the correlation of from-scratch success with transfer gain is mechanically negative regardless of any transfer mechanism, because it is partly a correlation of a quantity with minus itself. The non-tautological question is whether learnability predicts the gain, and on the 29-morphology suite it does not, with $\rho =0.061$ and a bootstrap interval of $[-0.012,+0.139]$ that includes zero. The corrected claim separates two quantities that the tautology conflates. The transfer outcome is essentially the target’s own from-scratch trainability, with mean success after transfer $0.354$ against mean success from scratch $0.346$, a near-identity, while the transfer gain is unpredictable from learnability, distance, or class match alike.

The central negative of the program is a third instance of the same check. A natural claim is that morphological distance predicts transfer, so that a learned distance regressing transfer gain from source-to-target body features is the predictor practitioners want (case h). Under the gate the distance predictor reaches Spearman $\rho =0.283$ while the target-only prior, an identically trained regressor that never sees the source, reaches $0.579$, and the gap of $-0.383$ has a bootstrap interval of $[-0.70,-0.12]$ over 42 independent pairs, a clean failure. The companion transferability oracle tells the same story from the other side, since full morphology features reach $\rho =0.762$ yet fail to beat a single arm-or-not-arm class bit at $0.834$, with a gap interval of $[-0.24,+0.05]$ that includes zero. The corrected claim is that morphological distance does not predict cross-embodiment transfer and that the apparent oracle signal is a robot-class prior that one bit reproduces. The supported statement stops short of the directional claim that distance is significantly worse than the prior, because pairs sharing a target are correlated and under target-cluster and morphology-cluster bootstraps the gap interval widens to $[-0.678,+0.022]$ and $[-1.055,+0.348]$, both of which include zero.

The last member of this cluster is a correlation whose two variables share a term, which the gate’s tautology guard catches one level up (case k). A natural claim is that multi-agent orchestration degrades with model capability, since across an eight-model ladder the Spearman correlation between single-agent success and the four-agent-minus-single-agent gap is $-0.83$ at $p=0.01$. Because single-agent success appears in both variables, the correct null is not zero correlation. Under an independence null in which the two success measures are independent, the null distribution of the shared-term statistic has mean $-0.83$, exactly the observed value, with the observed value at the $0.59$ quantile of that null, and the direct correlation between single-agent and four-agent success is $+0.38$ at $p=0.349$. The corrected claim is that the $-0.83$ is what independence predicts for a shared-term statistic rather than evidence of degradation, and that a correlation whose variables share a term needs its null simulated rather than assumed.

4.2. Probe Strength, Data Volume, and Prediction Targets

The second cluster concerns representations and prediction, and its cases are caught by Checks 2, 3, 5, and 6. Two of them share a substrate and are best read together. A natural report of an adversarial invariance run states that the in-loop morphology adversary converged to accuracy near $1.0$ everywhere, so the representation is doing its job (case c). Reading the actual grid shows the in-loop adversary reaches $1.0$ only at the larger bottlenecks and sits between $0.738$ and $0.846$ at the tight bottleneck, so it did not converge to one value throughout, and more fundamentally the in-loop adversary is the wrong instrument to certify invariance at all. Certification is the job of a fresh independent strong probe, which is what Check 2 uses, and this correction is what makes the next case meaningful. That next case is the linear-probe false positive (case e). A natural certification of a morphology-invariant task-state reports that a linear probe decodes the body from the latent at only $0.396$ against a three-way chance of $0.333$, near chance, with task-sufficiency preserved at an $R^2$ of $0.952$. A strong random-forest probe re-run on the identical latent recovers the body at $0.983$, and no cell in the capacity-by-adversary grid holds both an $R^2$ above $0.5$ and a strong-probe accuracy below $0.45$. Figure 2a plots the two probes across the grid and shows the linear probe dipping toward chance while the strong probe stays pinned near $1.0$. The corrected claim is that linear-probe invariance is a false positive here and that only a coarse task-state sheds the body fingerprint, a point the companion theory paper develops into an impossibility result [6]; a coarse-continuous cell passes the same certification with an $R^2$ of $0.801$, a strong probe of $0.413$, and a linear probe of $0.381$.

A separate case concerns pretraining diversity, caught by the data-volume control (case d). A natural report states that few-shot transfer to a held-out morphology rises with the number of pretraining source morphologies, an embodiment-diversity scaling law. Holding the total source budget fixed and splitting it across more morphologies dissolves the rise. Budget-matched breadth minus depth is $-0.010$, $-0.024$, and $-0.063$ at total budgets of thirty, sixty, and one hundred twenty episodes, so breadth never beats depth and the deficit grows with budget, and for the arm class the budget-matched gap is $-0.099$ even though the uncontrolled curve rises mildly. The corrected claim is that at a fixed data budget embodiment breadth does not beat per-source depth and that the apparent diversity scaling law was a data-volume effect, which resolves a recent locomotion embodiment-scaling claim on the budget-matched axis it did not control [32].

A final case in this cluster concerns predicting the data budget a new robot needs, caught by the non-degenerate-target and no-leakage checks together (case f). A natural report states that a calibrated data-budget oracle predicts samples-to-threshold for a new body about $31\%$ more sharply than a constant-budget baseline. On the procedural-grammar locomotion fleet the default budget target is degenerate, with $88\%$ of bodies at the sample-to-threshold floor, so an oracle is mostly predicting that constant floor. Moving to a non-degenerate budget target and requiring the probe to end before the threshold crossing leaves only eleven of eighty-six bodies as genuine-extrapolation units, and the sharpening becomes marginal, a cross-class half-width of $0.955$ against $0.891$. The corrected claim has two parts. The grammar-substrate demonstration is confounded by a degenerate floor, but the underlying idea works on a substrate with real budget variation. On fourteen manipulation arms with ten genuine-extrapolation units, a leakage-free tiny-demo probe predicts log-budget with coverage $0.929$ and a conformal half-width of $0.841$ against a constant baseline of $1.386$ and a static-feature baseline of $2.299$.

4.3. Units, Appearance, and Episode Structure

The third cluster concerns the mechanics of turning measurements into intervals and of probing released models, and its cases are caught by Checks 4, 7, and 8. The first is pseudo-replication (case g). A natural report states that the morphology-distance predictor’s deficit against the target prior has a gap interval of $[-0.46,-0.14]$ over forty-two pairs. The underlying data hold one hundred twenty-six pair-by-seed rows, and the reported analysis bootstrapped those rows while describing them as forty-two independent pairs, treating three seeds of one pair as three independent observations. Aggregating predictions per pair before the bootstrap gives an interval of $[-0.70,-0.12]$ over forty-two independent pairs, wider than the pseudo-replicated one, with the gate verdict unchanged. The same trap produced a genuine sign flip elsewhere, since on the locomotion task the analysis over thirty-six pair-by-seed rows gave an interval excluding zero while the aggregated twelve-pair analysis gives $[-0.31,+0.82]$, which includes zero. Pseudo-replication does not always merely tighten an interval; sometimes it manufactures a result.

The remaining two cases probe a released vision-language-action model and are the reason Checks 7 and 8 exist. A natural claim is that a released cross-embodiment model encodes embodiment identity, since a strong probe recovers robot identity at random-forest accuracy $1.0$ at every layer across five Open X-Embodiment robots (case i). Running the identical probe on the raw input settles it. Raw eight-by-eight downsampled pixels recover robot identity at random-forest accuracy $1.0$ against a chance of $0.20$, and Figure 2b places the representation probe and the raw-pixel control side by side, both far above chance. The five datasets are different laboratories, cameras, and backgrounds, so robot identity is trivially separable from scene appearance and the representation inherits it. The corrected claim is that on natural multi-robot data an embodiment-identity probe measures scene appearance rather than the body, and that the layer-wise curve says nothing about embodiment coding.

The last case concerns a linear scrub on the same model, caught by the strong-probe and episode-blocked-split checks together (case j). A natural claim is that iterative nullspace projection reproduces an invariance signature, driving the linear body probe to chance while preserving action prediction. The certificate is vacuous. The projection drives the linear body probe to $0.079$ while a strong random-forest probe still recovers the body at $0.715$ under an episode-blocked split, which is mechanically guaranteed because the projection removes only linear structure and the raw-pixel control already showed abundant nonlinear identity signal in appearance alone. The action metric was episode-leaked, since full-representation action prediction is $R^2=0.461$ under a frame-level split but $0.192$ under the episode-blocked split, so about sixty percent of the apparent action signal was episode memorization. The claimed body-to-action coupling was a control artifact, because a variance-matched body-blind removal destroys action prediction more, at $R^2=-0.592$ and probe $0.171$, than the invariance-directed removal, at $R^2=-0.096$ and probe $0.469$. The corrected claim is that no bound validation and no body-specific action-coupling claim can be made from this substrate, and that the surviving contribution is methodological, namely that linear-scrub invariance certificates on such representations are false positives and that per-frame metrics on episodic data require episode-blocked splits.

4.4. Reading the Eleven Together

Each of the eight checks catches at least one case, and the cases are not redundant. Figure 3 plots the coverage of checks over cases and shows the spread directly. The failures are of genuinely different kinds, a mechanism confound, a definitional tautology, a misattributed certifier, a volume artifact, a probe-strength artifact, a degenerate target, a unit error, a class prior, an appearance confound, a vacuous scrub on leaky splits, and a degenerate null from a shared term. No single discipline would have caught all eleven, which is the reason the standard is a set of checks rather than one rule. The spread also explains why several of the cases would have survived ordinary review, since each is locally reasonable and only a specific control reveals the confound.

5. Discussion

The standard is best understood as complementary to the evaluation-rigor movement rather than a competitor to it. That movement is largely about evaluating a policy’s task success rigorously, through blind randomized trials, sequential hypothesis testing, and distributed real-world comparison [8,9,10]. Those protocols harden how many trials a claim needs and which test decides it, and they would not have caught most of the eleven cases here, because the cases are not about how a success rate was measured but about what a claim is really comparing. Cross-embodiment transfer carries its own confound set that rollout-level rigor does not target. A transfer effect can be a one-bit robot-class prior rather than a morphology effect, which requires the strongest-baseline discipline of the gate. A diversity effect can be a data-volume effect, which requires the budget-matched control. An invariance claim can be a linear-probe false positive, which requires strong-probe certification. An interval can be tightened by pseudo-replication across pairs and seeds, which requires aggregation to independent units. A budget-prediction claim can rest on a degenerate or observed target, which requires the non-degenerate-target and no-leakage checks. An identity probe on a released model can read scene appearance rather than the body, which requires the raw-input control, and a per-frame metric on episodic data can be inflated by episode memorization, which requires episode-blocked splits. The contribution is the cross-embodiment-specific confound set, an enforced gate that puts the strongest-baseline-with-a-confidence-interval rule in the code path rather than in a reviewer’s memory, and a worked before-and-after record in which the checks reduce or reverse eleven real claims.

Several of the corrected claims are now consistent with an emerging consensus this standard did not create. Morphology-conditioned multi-embodiment agents have been reported to fail on out-of-distribution morphologies with train-test similarity not helping [31], and volume-matched studies have found embodiment diversity to be an optional axis [33]. The value the standard adds is not the discovery of these negatives but the discipline that makes them falsifiable and that catches their positive counterparts before they ship. The same discipline sharpens the constructive program, since the corrected claims point consistently toward a coarse interface as the locus of transfer, which the companion theory and architecture papers develop into a bound and a working system [6,7], and which the zero-shot systems of 2026 realize by deleting body information from the interface between task reasoning and motor control [36,37,38,39].

The standard is released as runnable artifacts precisely so that the discipline lives in the code path. The gate recomputes each claim’s metric, runs the strongest baseline, bootstraps the gap over independent units, and emits a verdict; the seven remaining checks are implemented as tools that read the committed artifacts and print the check’s verdict with its numbers; and the eleven cases ship as artifacts a reader can inspect and re-derive. The intended use is the one made of the standard here, as an adversary against a group’s own most appealing numbers rather than as a stamp of correctness, a distinction the limitations make precise.

6. Limitations

This is a methodology and standards paper and introduces no transfer result of its own. Its value is the standard and the evidence that the checks reduce or reverse real claims, and the underlying empirical claims are defended in the companion papers and re-derivable from the released artifacts.

The gate is necessary rather than sufficient. Passing it and the seven companion checks rules out the specific confounds enumerated here; it does not certify a claim true, because a claim can pass every check and still be wrong for a reason outside the eight, whether a confound not anticipated, an implementation error in the metric, or a benchmark that is unrepresentative. The gate is also only as good as the strongest baseline a user thinks to include, since it runs the baseline it is given, and an analyst who omits the true strongest alternative can still pass it. The standard mitigates this by fixing a baseline battery of the class bit, the target prior, scratch-only, distance, and within-class, but the battery is itself a claim about what the strongest alternatives are and could be incomplete for a new setting.

The eight checks are not claimed to be complete. They are the confounds encountered in one program, and other cross-embodiment confounds surely exist, among them selection effects in which robots get benchmarked, contamination across embodiment datasets, and reward shaping that smuggles in body knowledge. Two of the eight, the raw-input control and the episode-blocked split, were added when the probes of a released model met confounds the original six did not name, which is direct evidence that the set is expected to grow rather than a closed taxonomy.

The case study is drawn from a single research program that is predominantly in simulation and predominantly behavior-cloning transfer with hand-designed morphology features, with two cases probing a released model on natural multi-robot data and one drawn from a language-agent ladder. That the checks reduced or reversed eleven claims within one program is strong evidence that they are useful; it is not evidence about base rates in the field, nor proof that the same confounds dominate real-hardware, reinforcement-learning, or learned-embedding settings. Individual cases carry their own scope, since the within-class budget oracle rests on fourteen arms, the diversity control is run at moderate budgets, and cross-topology learnability and a real-hardware replication are explicit future work.

Finally, a gate that emits a pass invites the reading that a passed claim is therefore correct, which is the opposite of the discipline it is meant to instill. A pass means that a claim survived these confounds on these units, and nothing more. The productive use of the standard is as an adversary against a group’s own most appealing numbers, which is the use made of it in the eleven cases here.

7. Conclusion

Cross-embodiment transfer claims fail in a small set of specific, recurring ways, and the failures are not caught by measuring success rates more carefully. A claim can ride a robot-class prior that one bit reproduces, tighten an interval by treating seeds as independent pairs, omit the trivial baseline that already explains the effect, certify invariance with a linear probe that a strong probe falsifies, credit diversity for what is data volume, predict a target that is degenerate or observed rather than forecast, read scene appearance off an identity probe on natural multi-robot data, or inflate a per-frame metric through episode memorization. Each of these is locally reasonable, each would survive ordinary peer review, and each is caught by a specific control. This paper collects those controls into eight checks, puts the backbone check into an enforced acceptance gate that runs before any headline ships, and demonstrates the standard through eleven documented cases in which a tempting claim met a diagnostic that reduced or reversed it.

The evidence that a standard of this kind earns its place is the record of what enforcing it changes. A correlation of $0.98$ became a statement about leg geometry, a governing correlation of $-0.81$ became a definitional near-identity with an unpredictable gain, an invariant-looking latent became a body fingerprint a strong probe recovers at $0.98$, a diversity scaling law became a data-volume effect, a budget oracle that was $31\%$ sharper became a predictor of a floor, an interval of $[-0.46,-0.14]$ became the wider $[-0.70,-0.12]$ over independent pairs, an identity probe on a released model became a measurement of scene appearance, a linear invariance certificate became vacuous under a strong probe and episode-blocked splits, and a correlation of $-0.83$ became exactly the null its own shared term predicts. None of these corrections required new data. Each required a control that was one line of code away, and each is the difference between a publishable-looking positive and the negative the data support.

The broader lesson for the cross-embodiment field is that measurement discipline is a research contribution in its own right, and that for this subfield the discipline has a specific content beyond general statistical rigor. The right question to ask of a transfer claim is not only whether its success rate is significant but what the claim is really comparing, whether the effect survives the strongest trivial alternative on independent units, whether an invariance survives a strong probe, and whether a diversity benefit survives a fixed budget. The standard released here puts those questions in the code path, and the natural next steps are to extend the check set as new confounds surface, to exercise the gate on real-hardware and reinforcement-learning transfer where the base rates of these confounds are unknown, and to hold new claims of predictive or certified cross-embodiment transfer to the same rule. A predictor that beats the target-only prior on the released benchmark, or an invariance that survives a strong probe on non-commensurable bodies, would be a genuine advance precisely because the standard is calibrated by everything that failed on it.