Cross-Entropy vs. Hinge
Compute the Gradient计算梯度。
Regularization for Neural Networks - DropoutDropout 是 DNN 的一种正则化技术。
Convolutional卷积核有三个参数，width, height, depth.
ConvNets Architecture Overview
- A simple ConvNet for CIFAR-10 classification could have the architecture [INPUT - CONV - RELU - POOL - FC]. In more detail:
INPUT[32x32x3] will hold the raw pixel values of the image, in this case an image of width 32, height 32, and with three color channels R,G,B.
CONVlayer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and the region they are connected to in the input volume. This may result in volume such as [32x32x12].
RELUlayer will apply an elementwise activation function, such as the max(0,x). This leaves the size of the volume unchanged ([32x32x12]).
POOLlayer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as [16x16x12].
FC(i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1x1x10], where each of the 10 numbers correspond to a class score, such as among the 10 categories of CIFAR-10.
AlexNet. Winner of ImageNet 2012. It significantly outperformed the second runner-up (top 5 error of 16% compared to runner-up with 26% error).
ZF-Net. Winner of ImageNet 2013. It was an improvement on AlexNet by tweaking the architecture hyperparameters.
GoogLeNet. Winner of ImageNet 2014. Its main contribution was the development of an Inception Module that dramatically reduced the number of parameters in the network (4M, compared to AlexNet with 60M).
VGGNet. Runner-up of ImageNet 2014. Its main contribution was in showing that the depth of the network is a critical component for good performance. A downside of the VGGNet is that it is more expensive to evaluate and uses a lot more memory and parameters (140M).
ResNet. Runner-up of ImageNet 2015. It features special skip connections and a heavy use of batch normalization.
NCFM – Marine Fish Classificationgithub 项目开源地址：github repo
Using Keras + TensorFlow to solve NCFM-Leadboard Top 5%.