Preprint
Article

This version is not peer-reviewed.

FAR-JEPA: Factorized Autoregressive Process Laws for Resolving the Orthogonal Gauge in Joint Embeddings

Submitted:

27 July 2026

Posted:

28 July 2026

You are already at the latest version

Abstract
Joint-embedding predictive architectures (JEPAs) learn representations by predicting embeddings rather than reconstructing observations, but isotropic Gaussian constraints leave their coordinates undetermined up to orthogonal rotation. We introduce the factorized autoregressive joint-embedding predictive architecture (FAR-JEPA), a decoder-free method that selects temporal factor axes by fitting a context-conditioned autoregressive law to each embedding coordinate. An invertible shared encoder preserves the latent state, while an auxiliary network predicts coordinate-wise means, innovation variances, and autoregressive coefficients without source labels or true process parameters. Within the orthogonal JEPA solution class, we show that the population factorized conditional score identifies source coordinates up to signed permutation provided that, for every factor pair, at least one retained-lag Jacobian or innovation-variance profile differs on a context set of positive probability. Equal process signatures retain rotations within their shared subspaces, and an under-specified autoregressive order fails when it hides all axis-selecting statistics. In the primary generic-flow comparison on controlled nonlinear mixtures, FAR-JEPA attains matched component correlations of $0.9965$ and $0.986$ under exact unit marginals and rich conditional modulation, respectively, versus $0.743$ and $0.692$ for a JEPA-flow control using the same flow backbone. In separate matched-inverse mechanism experiments, context shuffling, signature collapse, and omission of the sole informative lag remove the gain. These controlled results support factorized autoregressive process laws as a mechanism for converting a state-equivalent JEPA representation into temporally factor-aligned coordinates.
Keywords: 
;  ;  ;  ;  ;  
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2026 MDPI (Basel, Switzerland) unless otherwise stated

Accessibility

Disclaimer

Terms of Use

Privacy Policy

Privacy Settings