Version 1
: Received: 5 March 2021 / Approved: 5 March 2021 / Online: 5 March 2021 (10:05:38 CET)
How to cite:
Nanni, L.; Brahnam, S.; Paci, M.; Maguolo, G. Comparison of Different Convolutional Neural Network Activation Functions and Methods for Building Ensembles. Preprints2021, 2021030180. https://doi.org/10.20944/preprints202103.0180.v1
Nanni, L.; Brahnam, S.; Paci, M.; Maguolo, G. Comparison of Different Convolutional Neural Network Activation Functions and Methods for Building Ensembles. Preprints 2021, 2021030180. https://doi.org/10.20944/preprints202103.0180.v1
Nanni, L.; Brahnam, S.; Paci, M.; Maguolo, G. Comparison of Different Convolutional Neural Network Activation Functions and Methods for Building Ensembles. Preprints2021, 2021030180. https://doi.org/10.20944/preprints202103.0180.v1
APA Style
Nanni, L., Brahnam, S., Paci, M., & Maguolo, G. (2021). Comparison of Different Convolutional Neural Network Activation Functions and Methods for Building Ensembles. Preprints. https://doi.org/10.20944/preprints202103.0180.v1
Chicago/Turabian Style
Nanni, L., Michelangelo Paci and Gianluca Maguolo. 2021 "Comparison of Different Convolutional Neural Network Activation Functions and Methods for Building Ensembles" Preprints. https://doi.org/10.20944/preprints202103.0180.v1
Abstract
Recently, much attention has been devoted to finding highly efficient and powerful activation functions for CNN layers. Because activation functions inject different nonlinearities between layers that affect performance, varying them is one method for building robust ensembles of CNNs. The objective of this study is to examine the performance of CNN ensembles made with different activation functions, including six new ones presented here: 2D Mexican ReLU, TanELU, MeLU+GaLU, Symmetric MeLU, Symmetric GaLU, and Flexible MeLU. The highest performing ensemble was built with CNNs having different activation layers that randomly replaced the standard ReLU. A comprehensive evaluation of the proposed approach was conducted across fifteen biomedical data sets representing various classification tasks. The proposed method was tested on two basic CNN architectures: Vgg16 and ResNet50. Results demonstrate the superiority in performance of this approach. The MATLAB source code for this study will be available at https://github.com/LorisNanni.
Computer Science and Mathematics, Algebra and Number Theory
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
