Deep Learning | 4 Convolutional

• Convolution Operation

Edge Detection

• Vertical Edge Detection
• Horizontal Edge Detection

Filter

• Filter
• Sobel Filter
• Scharr Filter
• Learned Filter

Padding

• Padding: Prevent shrinking of size in the output

Valid / Same Convolutions

• Valid Convolutions: no padding. (n, n) * (f, f) -> (n-f+1, n-f+1)
• Same Convolutions: pad to keep size. (n+2p, n+2p) * (f, f) -> (n+2p-f+1, n+2p-f+1). When p = (f-1)/2, n + 2p - f + 1 = n. So, if f = 3, p = 1; if f = 5, p = 2.

Strided Convolutions

• Strided Convolutions

Convention: the filter must lie entirely inside the image plus padding to generate the result of the convolution operation.

image.shape = (n, n)
filter.shape = (f, f)
padding = p
stride = s

output.shape = ( floor( (n + 2p - f) / s + 1), floor( (n + 2p - f) / s + 1) )

• Volume Convolutions

padding = 0
stride = 1
(n, n, n_c) * (f, f, n_c) -> (n-f+1, n-f+1, n_f)
n_c is the number of channels of the input image
n_f is the number of filters

ConvNet Single Layer

Type of Layers in a Convolutional Network

• Convolution (CONV)
• Pooling (POOL)
• Fully Connected (FC)

Pooling

• Max pooling
• Average pooling
• Hyper-parameters: filter size, stride, max or average pooling(, padding). If set f=2, s=2, the output is half the width and half the height of the input.
• No parameters to learn, pooling is just a fixed function
• Conventionally, each conv layer is followed by a pooling layer, and they are together called one layer in the conv net.

Classic Conv Net Architecture

• LeNet - 5
• AlexNet
• VGG - 16