Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit

Enhancing Deep Learning Performance Using Displaced Rectifier Linear Unit

Autor: Marca: Dial騁ica Refer麩cia: 9786525230566

Carregando...
Simulador de Frete
- Calcular frete


 

Descri鈬o

Recently, deep learning has caused a significant impact on computer vision, speech recognition, and natural language understanding. In spite of the remarkable advances, deep learning recent performance gains have been modest and usually rely on increasing the depth of the models, which often requires more computational resources such as processing time and memory usage. To tackle this problem, we turned our attention to the interworking between the activation functions and the batch normalization, which is virtually mandatory currently. In this work, we propose the activation function Displaced Rectifier Linear Unit (DReLU) by conjecturing that extending the identity function of ReLU to the third quadrant enhances compatibility with batch normalization. Moreover, we used statistical tests to compare the impact of using distinct activation functions (ReLU, LReLU, PReLU, ELU, and DReLU) on the learning speed and test accuracy performance of VGG and Residual Networks state-of-the-art models. These convolutional neural networks were trained on CIFAR-10 and CIFAR-100, the most commonly used deep learning computer vision datasets. The results showed DReLU speeded up learning in all models and datasets. Besides, statistical significant performance assessments (p<0.05) showed DReLU enhanced the test accuracy obtained by ReLU in all scenarios. Furthermore, DReLU showed better test accuracy than any other tested activation function in all experiments with one exception.



Caractersticas

  • Ano: 2022
  • Autor: David Mac鹽o
  • Selo: Dial騁ica
  • ISBN: 9786525230566
  • Nコ de P疊inas: 100


Coment疵ios e Avalia鋏es

Deixe seu comentário e sua avalia鈬o







- M痊imo de 512 caracteres.

Clique para Avaliar


  • Avalia鈬o:
Enviar
Fa軋 seu login e comente.

Recently, deep learning has caused a significant impact on computer vision, speech recognition, and natural language understanding. In spite of the remarkable advances, deep learning recent performance gains have been modest and usually rely on increasing the depth of the models, which often requires more computational resources such as processing time and memory usage. To tackle this problem, we turned our attention to the interworking between the activation functions and the batch normalization, which is virtually mandatory currently. In this work, we propose the activation function Displaced Rectifier Linear Unit (DReLU) by conjecturing that extending the identity function of ReLU to the third quadrant enhances compatibility with batch normalization. Moreover, we used statistical tests to compare the impact of using distinct activation functions (ReLU, LReLU, PReLU, ELU, and DReLU) on the learning speed and test accuracy performance of VGG and Residual Networks state-of-the-art models. These convolutional neural networks were trained on CIFAR-10 and CIFAR-100, the most commonly used deep learning computer vision datasets. The results showed DReLU speeded up learning in all models and datasets. Besides, statistical significant performance assessments (p<0.05) showed DReLU enhanced the test accuracy obtained by ReLU in all scenarios. Furthermore, DReLU showed better test accuracy than any other tested activation function in all experiments with one exception.

Deixe seu comentário e sua avalia鈬o







- M痊imo de 512 caracteres.

Clique para Avaliar


  • Avalia鈬o:
Enviar
Fa軋 seu login e comente.
  • Ano: 2022
  • Autor: David Mac鹽o
  • Selo: Dial騁ica
  • ISBN: 9786525230566
  • Nコ de P疊inas: 100


Receba nossas promo鋏es por e-mail: