Lie-Group Based Neural Networks: Ideas and Applications

This paper studies neural network layers that use learnable Lie-group actions as structured feature-space transformations. Instead of treating Lie groups only as input-domain symmetry constraints, the proposed approach embeds real-valued features into local vector banks, learns coordinates in a Lie algebra, maps these coordinates to group elements through the matrix exponential, and applies the resulting matrices to intermediate feature vectors. The framework supports groups such as SO(3), SU(2), and SU(3), and can be used either as a standalone structured backbone or as a component inside conventional neural architectures. The experimental evaluation covers several settings: tabular classification, tabular regression, synthetic signal denoising, generative adversarial learning, and recursive time-series forecasting. The classification and regression studies compare dense neural baselines, MLP–Lie hybrids, deeper Lie-group architectures, CatBoost, and ExtraTrees across repeated train-validation-test splits. The denoising experiment compares a classical autoencoder with an SU(3)-based autoencoder on synthetic oscillatory signals. The GAN experiment inserts an SU(3) layer into the discriminator and compares it with a standard convolutional GAN on MNIST digit generation. The time-series experiments compare a regular Transformer, a hybrid Transformer with one Lie-group layer, a Lie-group Transformer, and CatBoost under recursive holdout forecasting. The results show that Lie-group feature transformations are useful in selected settings, but they are not uniformly superior across all tasks. In classification, the structured models improve over the dense baseline on several datasets, while tree-based methods remain strongest on others. In regression, MLP–Lie models are competitive on some tasks, but CatBoost and ExtraTrees are often stronger. The clearest improvement is observed in signal denoising, where the Lie-group autoencoder reduces reconstruction error and improves signal-to-noise ratio. In the GAN experiment, the Lie-group discriminator gives moderate improvements in stability and discriminator metrics. In time-series forecasting, Lie-group Transformer variants improve over the regular Transformer on some series, while CatBoost remains a strong rolling-window baseline. Overall, the results support a dataset-dependent interpretation. Lie-group layers can act as useful structured feature mixers, especially when local vector structure or oscillatory behavior is relevant. At the same time, their benefit depends on the task, architecture, and computational cost. The framework therefore provides a practical basis for studying when algebraic feature-space transformations improve learning and when simpler baselines are sufficient.