preprints.org > computer science and mathematics > computer science > doi: 10.20944/preprints202211.0537.v2

Preprint Article Version 2 Preserved in Portico This version is not peer-reviewed

Version 1 : Received: 29 November 2022 / Approved: 29 November 2022 / Online: 29 November 2022 (07:07:56 CET)
Version 2 : Received: 29 November 2022 / Approved: 29 November 2022 / Online: 29 November 2022 (08:13:27 CET)

We propose a forward-only multi-layer encoding-decoding framework based on the principle of Maximal Coding Rate Reduction (MCR$^2$), an information-theoretic metric that measures a statistical distance between two sets of feature vectors up to the second moment. The encoder directly transforms data vectors themselves via gradient ascent to maximize the MCR$^2$ distance between different classes in the feature space, resulting in class-wise mutually orthogonal subspace representations. The decoder follows a process symmetric to the encoder, and transforms the subspace feature vectors via gradient descent to minimize the MCR$^2$ distance between the reconstructed data and the original data. We show that the encoder transforms data to linear discriminative representations without breaking the higher-order manifolds, and the decoder reconstructs the data with high fidelity.

Backpropagation-free Network, Information Theory, Maximal Coding Rate Reduction

Computer Science and Mathematics, Computer Science

* All users must log in before leaving a comment