Version 1
: Received: 20 April 2024 / Approved: 22 April 2024 / Online: 22 April 2024 (18:16:43 CEST)
How to cite:
Kang, C.; Kang, S. Deep Transfer Learning Method using Self-Pixel and Global Channel Attentive Regularization. Preprints2024, 2024041423. https://doi.org/10.20944/preprints202404.1423.v1
Kang, C.; Kang, S. Deep Transfer Learning Method using Self-Pixel and Global Channel Attentive Regularization. Preprints 2024, 2024041423. https://doi.org/10.20944/preprints202404.1423.v1
Kang, C.; Kang, S. Deep Transfer Learning Method using Self-Pixel and Global Channel Attentive Regularization. Preprints2024, 2024041423. https://doi.org/10.20944/preprints202404.1423.v1
APA Style
Kang, C., & Kang, S. (2024). Deep Transfer Learning Method using Self-Pixel and Global Channel Attentive Regularization. Preprints. https://doi.org/10.20944/preprints202404.1423.v1
Chicago/Turabian Style
Kang, C. and Sang-ug Kang. 2024 "Deep Transfer Learning Method using Self-Pixel and Global Channel Attentive Regularization" Preprints. https://doi.org/10.20944/preprints202404.1423.v1
Abstract
The purpose of this paper is to propose a novel transfer learning regularization method based on knowledge distillation. Recently, transfer learning methods have been used in various fields. However, the problems such as knowledge loss still occur during the process of transfer learning to a new target dataset. To solve these problems, there are various regularization methods based on knowledge distillation techniques. In this paper, we propose a transfer learning regularization method based on feature map alignment used in the field of knowledge distillation. The proposed method is composed of two attention based submodules: self-pixel attention (SPA) and global channel attention (GCA). The self-pixel attention submodule utilizes both the feature maps of the source and target models, so that it provides an opportunity to jointly consider the features of the target and the knowledge of the source. The global channel attention submodule determines the importance of channels through all layers, unlike the existing methods that calculate these only within a single layer. Accordingly, transfer learning regularization is performed by considering both the internal of each single layer and the depth of the entire layer. Consequently, the proposed method using both of these submodules showed overall improved classification accuracy than the existing methods in classification experiments on commonly used dataset.
Keywords
deep transfer learning; knowledge distillation; regularization
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
