标签:反向 cs6 ati mfc 概率 parameter imp sub 常见
原文引用https://www.dazhuanlan.com/2019/08/25/5d625a7694a88/
白雪峰 — xfbai@mtlab.hit.edu.cn.
这是一篇关于CNN的总结
(1)Modern CNN since Yann LeCun
(2)
神经网络?卷积?
其连续的定义为:
$(fg)(n) = int_{ - infty }^{ + infty } {f(t)g(n-t)dt}$
其离散的定义为:
$(fg)(n) = sum_{t = - infty} ^{ + infty } {f(t)g(n-t)}$
特点:
丢骰子,两个骰子加起来要等于4的概率是多少?
$(f*g)(4) = sum_{m = 1} ^{ 3 } {f(m)g(4-m)}$
$(fg)(m,n) = sumlimits{k=0}^{2}sumlimits{h=0}^{2}f(h,k)g(m-h,n-k)$
则 (fg)(1,1) ?
离散的卷积可以看成是矩阵的乘法(翻转的)
2.3 用到二维图像上:
a.关于卷积中常用到的一些概念:
In Image Processing
– Convolution is always named filtering, and there are many famous filters/convolution kernels that extract intuitive features in images
通过在输入数据上不断移动卷积核,来提取不同位置的特征.
假设神经网络的输入是6*6的image,
那么,,,
再来个更形象的:
The step size you take the filter to sweep the
image
- A way not to ignore pattern on border
- New image is smaller than the original image
Pooling layers subsample their input
1. Spatial pooling (also called subsampling or
downsampling) reduces the dimensionality of
each feature map2. Retains the ***most important information***
Pooling is unsensitive to local translation.(局部不变性)
– “if we translate the input by a small amount,
the values of most of the pooled outputs do
not change.”
Pooling should be designed to fit specific
applications.
##3. Why CNN
Some patterns are much smaller than the whole image.
The same patterns appear in different regions.
Subsampling the pixels will not change the object
称在底层中影响上层输出单元 $s$ 的单元集合为 $s$ 的接受域(receptive field).
首先,弱先验具有较高的熵值,因此自由性较强.强先验具有较低的熵值,这样的先验在决定参数最终取值时可以起着非常积极的作用.
把卷积网络类比成全连接网络,但对于网络的权重具有无限强的先验.a) 所有隐藏单元的权重是共享的.b) 除了一些连续的小单元的权重外,其他的权重都是0.c) 池化也是一个无限强的先验:每个单元都具有对少量平移的不变性.
卷积和池化可能导致欠拟合! 任何其他先验类似,卷积和池化只有当先验的假设合理且正确时才有用。如果一项任务依赖于保存精确的空间信息,那么在所有的特征上使用池化将会增大训练误差。
根据实际需求选取先验
基本与FFNNs相同.
一个常见的做法:取梯度的平均值
torch.nn.Conv2d:
torch.nn.functional.max_pool2d:
import torch.nn as nn
import torch.nn.functional as F
class LeNet(nn.Module):
def __init__(self):
super(LeNet, self).__init__()
self.conv1 = nn.Conv2d(1, 6, 5) #in_channels:1, out_channels:6, kernel_size:5
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16*5*5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
out = F.relu(self.conv1(x))
out = F.max_pool2d(out, 2)
out = F.relu(self.conv2(out))
out = out.view(out.size(0), -1)
out = F.relu(self.fc1(out))
out = F.relu(self.fc2(out))
out = F.softmax(self.fc3(out))
return out
标签:反向 cs6 ati mfc 概率 parameter imp sub 常见
原文地址:https://www.cnblogs.com/petewell/p/11408855.html