2020-01-11 论文阅读

论文阅读

1.Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

作者：

Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun

单位：

Microsoft Research

摘要：

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [33]). To our knowledge, our result is the first1 to surpass the reported human-level performance (5.1%, [26]) on this dataset.

创新点：

1、提出了PRULE激活函数

2、提出来的方法在ImageNet2012分类数据集上，到达4.94% top-5的错误率，超过当时最好的性能6.66%，超过人的水平5.1%