Non-Gradient Quaternion Training Matrix Modifications for Colored Images Distillation

This paper develops a new multi-stage method for image distillation which involves two well-known methods. At its first stage, our method creates a matrix from all training images. On the next stage, it adapts a modified principal component analysis (M-PCA) approach to transform the training matrix. On the third stage, the Singular Value Decomposition (SVD) further refines the matrix of the training images through low-rank reconstruction and controlled matrix row selection. On the fourth stage, rotation of small 2 × 2 matrix blocks on the entire left singular matrix is conducted. The upper m (user-selected number) rows of the reconstructed matrix are selected and transformed back to images, which we call distilled images. This dataset is significantly smaller yet retains the critical information needed for an accurate classification. We validated the novelties and the advantages of the new method applying the Baseline and ResNet50V2 CNNs and the public image databases Digit-MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, and BloodMNIST. Experimental results show that models trained on images distilled by our new method achieve efficient classification and outperform contemporary competitors, while substantially reducing training time compared to training on the original dataset.