Transcript Seminar Chen, Weicong

ImageNet Classification with Deep
Convolutional Neural Networks
Presenter: Weicong Chen
Deep Convolutional Neural Networks
• Led by Geoffrey Hinton, University of Toronto
• Published in 2013
• Based on the datasets from ImageNet LSVRC-2010 Contest
• Using graphic cards to train the neural network
ImageNet LSVRC-2010 Contest
• 1.2 million high-resolution images
• 1,000 different classes
• 50,000 validation images, 150,000 testing images
• Top-1 error 47.1% best in contest, 45.7% best published
• Top-5 errors 28.2% best in contest, 25.7% best published
Convolutional Neural Networks (CNN)
Architecture
Novel/Unusual features in architecture
• ReLU (Rectified Linear Units ) Nonlinearity
Standard way: f(x) = tanh(x) or f(x) = (1 + e-x)-1 (logistic function)
ReLU: f(x) = max(0, x)
6 times faster than hyperbolic function
• Multiple GPUs
Larger memory
Parallel computing
Control of communication
Overfitting
• Occurs when a statistical model describes random error or noise
instead of the underlying relationship
• Exaggerate minor fluctuations in the data
• Will generally have poor predictive performance
Reducing overfitting
• Data Augmentation
1. Image translation and horizontal reflection
Randomly extracting patches
Four corner and one center patches with reflection for testing
2. Altering the intensities of the RGB channels in training images
Approximately captures an important property of natural images
reduces the top-1 error rate by over 1%
Reducing Overfitting
• Dropout
Zero the output of each hidden neuron with probability 0.5.
No longer contribute to forward pass and backward propagation
Neural network samples a different architecture every time
Reduce complex co-adaptations of neurons
Used in two fully-connected layers
Result
• Two NVIDIA GTX 580 3GB GPUs
• 6 days of training
• 90 cycles
• 60 million parameters
• 37.5% top-1 error (45.7% second best)
• 17% top-5 error (28.2% second best)
Questions?