标签:axis oba input lock 分支 false south access src
论文:Densely Connected Convolutional Networks
论文链接:https://arxiv.org/pdf/1608.06993.pdf
代码的github链接:https://github.com/liuzhuang13/DenseNet
文章详解:
这篇文章是CVPR2017的best paper,文章提出的DenseNet(Dense Convolutional Network)主要还是和ResNet及Inception网络做对比,思想上有借鉴,但却是全新的结构,网络结构并不复杂,却非常有效!众所周知,最近一两年卷积神经网络提高效果的方向,要么深(比如ResNet,解决了网络深时候的梯度消失问题)要么宽(比如GoogleNet的Inception),而作者则是从feature入手,通过对feature的极致利用达到更好的效果和更少的参数。
先列下DenseNet的几个优点,感受下它的强大:
1、减轻了vanishing-gradient(梯度消失)
2、加强了feature的传递
3、更有效地利用了feature
4、一定程度上较少了参数数量
在深度学习网络中,随着网络深度的加深,梯度消失问题会愈加明显,目前很多论文都针对这个问题提出了解决方案,比如ResNet,Highway Networks,Stochastic depth,FractalNets等,尽管这些算法的网络结构有差别,但是核心都在于:create short paths from early layers to later layers。那么作者是怎么做呢?延续这个思路,那就是在保证网络中层与层之间最大程度的信息传输的前提下,直接将所有层连接起来!
先放一个dense block的结构图。在传统的卷积神经网络中,如果你有L层,那么就会有L个连接,但是在DenseNet中,会有L(L+1)/2个连接。简单讲,就是每一层的输入来自前面所有层的输出。如下图:x0是input,H1的输入是x0(input),H2的输入是x0和x1(x1是H1的输出)……
DenseNet的一个优点是网络更窄,参数更少,很大一部分原因得益于这种dense block的设计,后面有提到在dense block中每个卷积层的输出feature map的数量都很小(小于100),而不是像其他网络一样动不动就几百上千的宽度。同时这种连接方式使得特征和梯度的传递更加有效,网络也就更加容易训练。原文的一句话非常喜欢:Each layer has direct access to the gradients from the loss function and the original input signal, leading to an implicit deep supervision.直接解释了为什么这个网络的效果会很好。前面提到过梯度消失问题在网络深度越深的时候越容易出现,原因就是输入信息和梯度信息在很多层之间传递导致的,而现在这种dense connection相当于每一层都直接连接input和loss,因此就可以减轻梯度消失现象,这样更深网络不是问题。另外作者还观察到这种dense connection有正则化的效果,因此对于过拟合有一定的抑制作用,博主认为是因为参数减少了(后面会介绍为什么参数会减少),所以过拟合现象减轻。
这篇文章的一个优点就是基本上没有公式,不像灌水文章一样堆复杂公式把人看得一愣一愣的。文章中只有两个公式,是用来阐述DenseNet和ResNet的关系,对于从原理上理解这两个网络还是非常重要的。
第一个公式是ResNet的。这里的l表示层,xl表示l层的输出,Hl表示一个非线性变换。所以对于ResNet而言,l层的输出是l-1层的输出加上对l-1层输出的非线性变换。
第二个公式是DenseNet的。[x0,x1,…,xl-1]表示将0到l-1层的输出feature map做concatenation。concatenation是做通道的合并,就像Inception那样。而前面resnet是做值的相加,通道数是不变的。Hl包括BN,ReLU和3*3的卷积。
这里Hl(.)是一个Composite function,是三个操作的组合Dense Block:BN?>(Scale)->ReLU?>Conv(3×3)->(Dropout)
所以从这两个公式就能看出DenseNet和ResNet在本质上的区别,太精辟。
前面的Figure 1表示的是dense block,而下面的Figure 2表示的则是一个DenseNet的结构图,在这个结构图中包含了3个dense block。作者将DenseNet分成多个dense block,原因是希望各个dense block内的feature map的size统一,这样在做concatenation就不会有size的问题。
这个Table1就是整个网络的结构图。这个表中的k=32,k=48中的k是growth rate,表示每个dense block中每层输出的feature map个数。为了避免网络变得很宽,作者都是采用较小的k,比如32这样,作者的实验也表明小的k可以有更好的效果。根据dense block的设计,后面几层可以得到前面所有层的输入,因此concat后的输入channel还是比较大的。另外这里每个dense block的3*3卷积前面都包含了一个1*1的卷积操作,就是所谓的bottleneck layer,目的是减少输入的feature map数量,既能降维减少计算量,又能融合各个通道的特征,何乐而不为。另外作者为了进一步压缩参数,在每两个dense block之间又增加了1*1的卷积操作。因此在后面的实验对比中,如果你看到DenseNet-C这个网络,每个dense block直接增加了这个Translation layer,该层的1*1卷积的输出channel默认是输入channel到一半, 具体:由BN?>(Scale)->(Relu)->Conv(1×1)?>(Dropout)->averagePooling(2×2)组成。如果你看到DenseNet-BC这个网络,表示既有bottleneck layer,又有Translation layer。
再详细说下bottleneck和transition layer操作。在每个Dense Block中都包含很多个子结构,以DenseNet-169的Dense Block(3)为例,包含32个1*1和3*3的卷积操作,也就是第32个子结构的输入是前面31层的输出结果,每层输出的channel是32(growth rate),那么如果不做bottleneck操作,第32层的3*3卷积操作的输入就是31*32+(上一个Dense Block的输出channel),近1000了。而加上1*1的卷积,代码中的1*1卷积的channel是growth rate*4,也就是128,然后再作为3*3卷积的输入。这就大大减少了计算量,这就是bottleneck。至于transition layer,放在两个Dense Block中间,是因为每个Dense Block结束后的输出channel个数很多,需要用1*1的卷积核来降维。还是以DenseNet-169的Dense Block(3)为例,虽然第32层的3*3卷积输出channel只有32个(growth rate),但是紧接着还会像前面几层一样有通道的concat操作,即将第32层的输出和第32层的输入做concat,前面说过第32层的输入是1000左右的channel,所以最后每个Dense Block的输出也是1000多的channel。因此这个transition layer有个参数reduction(范围是0到1),表示将这些输出缩小到原来的多少倍,默认是0.5,这样传给下一个Dense Block的时候channel数量就会减少一半,这就是transition layer的作用。文中还用到dropout操作来随机减少分支,避免过拟合,毕竟这篇文章的连接确实多。
实验结果:
作者在不同数据集上采用的DenseNet网络会有一点不一样,比如在Imagenet数据集上,DenseNet-BC有4个dense block,但是在别的数据集上只用3个dense block。其他更多细节可以看论文3部分的Implementation Details。训练的细节和超参数的设置可以看论文4.2部分,在ImageNet数据集上测试的时候有做224*224的center crop。
Table2是在三个数据集(C10,C100,SVHN)上和其他算法的对比结果。ResNet[11]就是kaiming He的论文,对比结果一目了然。DenseNet-BC的网络参数和相同深度的DenseNet相比确实减少了很多!参数减少除了可以节省内存,还能减少过拟合。这里对于SVHN数据集,DenseNet-BC的结果并没有DenseNet(k=24)的效果好,作者认为原因主要是SVHN这个数据集相对简单,更深的模型容易过拟合。在表格的倒数第二个区域的三个不同深度L和k的DenseNet的对比可以看出随着L和k的增加,模型的效果是更好的。
Figure3是DenseNet-BC和ResNet在Imagenet数据集上的对比,左边那个图是参数复杂度和错误率的对比,你可以在相同错误率下看参数复杂度,也可以在相同参数复杂度下看错误率,提升还是很明显的!右边是flops(可以理解为计算复杂度)和错误率的对比,同样有效果。
Figure4也很重要。左边的图表示不同类型DenseNet的参数和error对比。中间的图表示DenseNet-BC和ResNet在参数和error的对比,相同error下,DenseNet-BC的参数复杂度要小很多。右边的图也是表达DenseNet-BC-100只需要很少的参数就能达到和ResNet-1001相同的结果。
另外提一下DenseNet和stochastic depth的关系,在stochastic depth中,residual中的layers在训练过程中会被随机drop掉,其实这就会使得相邻层之间直接连接,这和DenseNet是很像的。
总结:
博主读完这篇文章真的有点相见恨晚的感觉,半年前就在arxiv上挂出来了,听说当时就引起了轰动,后来又被选为CVPR2017的oral,感觉要撼动ResNet的地位了,再加上现在很多分类检测的网络都是在ResNet上做的,这岂不是大地震了。惊讶之余来总结下这篇文章,该文章提出的DenseNet核心思想在于建立了不同层之间的连接关系,充分利用了feature,进一步减轻了梯度消失问题,加深网络不是问题,而且训练效果非常好。另外,利用bottleneck layer,Translation layer以及较小的growth rate使得网络变窄,参数减少,有效抑制了过拟合,同时计算量也减少了。DenseNet优点很多,而且在和ResNet的对比中优势还是非常明显的。
最后给出网络结构:
layer { name: "Data1" type: "Data" top: "Data1" top: "Data2" transform_param { mirror: true crop_size: 224 mean_value: 83 mean_value: 79 mean_value: 80 #mean_file: "./data/foodnet_mean.binaryproto" } data_param { source: "./data/densenettrain" #batch_size: 64 batch_size: 1 backend: LMDB } } layer { name: "Convolution1" type: "Convolution" bottom: "Data1" top: "Convolution1" convolution_param { num_output: 16 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "BatchNorm1" type: "BatchNorm" bottom: "Convolution1" top: "BatchNorm1" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale1" type: "Scale" bottom: "BatchNorm1" top: "BatchNorm1" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU1" type: "ReLU" bottom: "BatchNorm1" top: "BatchNorm1" } layer { name: "Convolution2" type: "Convolution" bottom: "BatchNorm1" top: "Convolution2" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout1" type: "Dropout" bottom: "Convolution2" top: "Dropout1" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat1" type: "Concat" bottom: "Convolution1" bottom: "Dropout1" top: "Concat1" concat_param { axis: 1 } } layer { name: "BatchNorm2" type: "BatchNorm" bottom: "Concat1" top: "BatchNorm2" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale2" type: "Scale" bottom: "BatchNorm2" top: "BatchNorm2" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU2" type: "ReLU" bottom: "BatchNorm2" top: "BatchNorm2" } layer { name: "Convolution3" type: "Convolution" bottom: "BatchNorm2" top: "Convolution3" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout2" type: "Dropout" bottom: "Convolution3" top: "Dropout2" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat2" type: "Concat" bottom: "Concat1" bottom: "Dropout2" top: "Concat2" concat_param { axis: 1 } } layer { name: "BatchNorm3" type: "BatchNorm" bottom: "Concat2" top: "BatchNorm3" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale3" type: "Scale" bottom: "BatchNorm3" top: "BatchNorm3" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU3" type: "ReLU" bottom: "BatchNorm3" top: "BatchNorm3" } layer { name: "Convolution4" type: "Convolution" bottom: "BatchNorm3" top: "Convolution4" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout3" type: "Dropout" bottom: "Convolution4" top: "Dropout3" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat3" type: "Concat" bottom: "Concat2" bottom: "Dropout3" top: "Concat3" concat_param { axis: 1 } } layer { name: "BatchNorm4" type: "BatchNorm" bottom: "Concat3" top: "BatchNorm4" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale4" type: "Scale" bottom: "BatchNorm4" top: "BatchNorm4" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU4" type: "ReLU" bottom: "BatchNorm4" top: "BatchNorm4" } layer { name: "Convolution5" type: "Convolution" bottom: "BatchNorm4" top: "Convolution5" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout4" type: "Dropout" bottom: "Convolution5" top: "Dropout4" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat4" type: "Concat" bottom: "Concat3" bottom: "Dropout4" top: "Concat4" concat_param { axis: 1 } } layer { name: "BatchNorm5" type: "BatchNorm" bottom: "Concat4" top: "BatchNorm5" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale5" type: "Scale" bottom: "BatchNorm5" top: "BatchNorm5" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU5" type: "ReLU" bottom: "BatchNorm5" top: "BatchNorm5" } layer { name: "Convolution6" type: "Convolution" bottom: "BatchNorm5" top: "Convolution6" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout5" type: "Dropout" bottom: "Convolution6" top: "Dropout5" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat5" type: "Concat" bottom: "Concat4" bottom: "Dropout5" top: "Concat5" concat_param { axis: 1 } } layer { name: "BatchNorm6" type: "BatchNorm" bottom: "Concat5" top: "BatchNorm6" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale6" type: "Scale" bottom: "BatchNorm6" top: "BatchNorm6" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU6" type: "ReLU" bottom: "BatchNorm6" top: "BatchNorm6" } layer { name: "Convolution7" type: "Convolution" bottom: "BatchNorm6" top: "Convolution7" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout6" type: "Dropout" bottom: "Convolution7" top: "Dropout6" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat6" type: "Concat" bottom: "Concat5" bottom: "Dropout6" top: "Concat6" concat_param { axis: 1 } } layer { name: "BatchNorm7" type: "BatchNorm" bottom: "Concat6" top: "BatchNorm7" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale7" type: "Scale" bottom: "BatchNorm7" top: "BatchNorm7" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU7" type: "ReLU" bottom: "BatchNorm7" top: "BatchNorm7" } layer { name: "Convolution8" type: "Convolution" bottom: "BatchNorm7" top: "Convolution8" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout7" type: "Dropout" bottom: "Convolution8" top: "Dropout7" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat7" type: "Concat" bottom: "Concat6" bottom: "Dropout7" top: "Concat7" concat_param { axis: 1 } } layer { name: "BatchNorm8" type: "BatchNorm" bottom: "Concat7" top: "BatchNorm8" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale8" type: "Scale" bottom: "BatchNorm8" top: "BatchNorm8" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU8" type: "ReLU" bottom: "BatchNorm8" top: "BatchNorm8" } layer { name: "Convolution9" type: "Convolution" bottom: "BatchNorm8" top: "Convolution9" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout8" type: "Dropout" bottom: "Convolution9" top: "Dropout8" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat8" type: "Concat" bottom: "Concat7" bottom: "Dropout8" top: "Concat8" concat_param { axis: 1 } } layer { name: "BatchNorm9" type: "BatchNorm" bottom: "Concat8" top: "BatchNorm9" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale9" type: "Scale" bottom: "BatchNorm9" top: "BatchNorm9" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU9" type: "ReLU" bottom: "BatchNorm9" top: "BatchNorm9" } layer { name: "Convolution10" type: "Convolution" bottom: "BatchNorm9" top: "Convolution10" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout9" type: "Dropout" bottom: "Convolution10" top: "Dropout9" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat9" type: "Concat" bottom: "Concat8" bottom: "Dropout9" top: "Concat9" concat_param { axis: 1 } } layer { name: "BatchNorm10" type: "BatchNorm" bottom: "Concat9" top: "BatchNorm10" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale10" type: "Scale" bottom: "BatchNorm10" top: "BatchNorm10" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU10" type: "ReLU" bottom: "BatchNorm10" top: "BatchNorm10" } layer { name: "Convolution11" type: "Convolution" bottom: "BatchNorm10" top: "Convolution11" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout10" type: "Dropout" bottom: "Convolution11" top: "Dropout10" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat10" type: "Concat" bottom: "Concat9" bottom: "Dropout10" top: "Concat10" concat_param { axis: 1 } } layer { name: "BatchNorm11" type: "BatchNorm" bottom: "Concat10" top: "BatchNorm11" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale11" type: "Scale" bottom: "BatchNorm11" top: "BatchNorm11" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU11" type: "ReLU" bottom: "BatchNorm11" top: "BatchNorm11" } layer { name: "Convolution12" type: "Convolution" bottom: "BatchNorm11" top: "Convolution12" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout11" type: "Dropout" bottom: "Convolution12" top: "Dropout11" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat11" type: "Concat" bottom: "Concat10" bottom: "Dropout11" top: "Concat11" concat_param { axis: 1 } } layer { name: "BatchNorm12" type: "BatchNorm" bottom: "Concat11" top: "BatchNorm12" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale12" type: "Scale" bottom: "BatchNorm12" top: "BatchNorm12" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU12" type: "ReLU" bottom: "BatchNorm12" top: "BatchNorm12" } layer { name: "Convolution13" type: "Convolution" bottom: "BatchNorm12" top: "Convolution13" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout12" type: "Dropout" bottom: "Convolution13" top: "Dropout12" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat12" type: "Concat" bottom: "Concat11" bottom: "Dropout12" top: "Concat12" concat_param { axis: 1 } } layer { name: "BatchNorm13" type: "BatchNorm" bottom: "Concat12" top: "BatchNorm13" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale13" type: "Scale" bottom: "BatchNorm13" top: "BatchNorm13" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU13" type: "ReLU" bottom: "BatchNorm13" top: "BatchNorm13" } layer { name: "Convolution14" type: "Convolution" bottom: "BatchNorm13" top: "Convolution14" convolution_param { num_output: 160 bias_term: false pad: 0 kernel_size: 1 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout13" type: "Dropout" bottom: "Convolution14" top: "Dropout13" dropout_param { dropout_ratio: 0.2 } } layer { name: "Pooling1" type: "Pooling" bottom: "Dropout13" top: "Pooling1" pooling_param { pool: AVE kernel_size: 2 stride: 2 } } layer { name: "BatchNorm14" type: "BatchNorm" bottom: "Pooling1" top: "BatchNorm14" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale14" type: "Scale" bottom: "BatchNorm14" top: "BatchNorm14" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU14" type: "ReLU" bottom: "BatchNorm14" top: "BatchNorm14" } layer { name: "Convolution15" type: "Convolution" bottom: "BatchNorm14" top: "Convolution15" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout14" type: "Dropout" bottom: "Convolution15" top: "Dropout14" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat13" type: "Concat" bottom: "Pooling1" bottom: "Dropout14" top: "Concat13" concat_param { axis: 1 } } layer { name: "BatchNorm15" type: "BatchNorm" bottom: "Concat13" top: "BatchNorm15" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale15" type: "Scale" bottom: "BatchNorm15" top: "BatchNorm15" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU15" type: "ReLU" bottom: "BatchNorm15" top: "BatchNorm15" } layer { name: "Convolution16" type: "Convolution" bottom: "BatchNorm15" top: "Convolution16" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout15" type: "Dropout" bottom: "Convolution16" top: "Dropout15" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat14" type: "Concat" bottom: "Concat13" bottom: "Dropout15" top: "Concat14" concat_param { axis: 1 } } layer { name: "BatchNorm16" type: "BatchNorm" bottom: "Concat14" top: "BatchNorm16" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale16" type: "Scale" bottom: "BatchNorm16" top: "BatchNorm16" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU16" type: "ReLU" bottom: "BatchNorm16" top: "BatchNorm16" } layer { name: "Convolution17" type: "Convolution" bottom: "BatchNorm16" top: "Convolution17" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout16" type: "Dropout" bottom: "Convolution17" top: "Dropout16" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat15" type: "Concat" bottom: "Concat14" bottom: "Dropout16" top: "Concat15" concat_param { axis: 1 } } layer { name: "BatchNorm17" type: "BatchNorm" bottom: "Concat15" top: "BatchNorm17" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale17" type: "Scale" bottom: "BatchNorm17" top: "BatchNorm17" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU17" type: "ReLU" bottom: "BatchNorm17" top: "BatchNorm17" } layer { name: "Convolution18" type: "Convolution" bottom: "BatchNorm17" top: "Convolution18" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout17" type: "Dropout" bottom: "Convolution18" top: "Dropout17" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat16" type: "Concat" bottom: "Concat15" bottom: "Dropout17" top: "Concat16" concat_param { axis: 1 } } layer { name: "BatchNorm18" type: "BatchNorm" bottom: "Concat16" top: "BatchNorm18" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale18" type: "Scale" bottom: "BatchNorm18" top: "BatchNorm18" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU18" type: "ReLU" bottom: "BatchNorm18" top: "BatchNorm18" } layer { name: "Convolution19" type: "Convolution" bottom: "BatchNorm18" top: "Convolution19" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout18" type: "Dropout" bottom: "Convolution19" top: "Dropout18" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat17" type: "Concat" bottom: "Concat16" bottom: "Dropout18" top: "Concat17" concat_param { axis: 1 } } layer { name: "BatchNorm19" type: "BatchNorm" bottom: "Concat17" top: "BatchNorm19" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale19" type: "Scale" bottom: "BatchNorm19" top: "BatchNorm19" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU19" type: "ReLU" bottom: "BatchNorm19" top: "BatchNorm19" } layer { name: "Convolution20" type: "Convolution" bottom: "BatchNorm19" top: "Convolution20" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout19" type: "Dropout" bottom: "Convolution20" top: "Dropout19" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat18" type: "Concat" bottom: "Concat17" bottom: "Dropout19" top: "Concat18" concat_param { axis: 1 } } layer { name: "BatchNorm20" type: "BatchNorm" bottom: "Concat18" top: "BatchNorm20" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale20" type: "Scale" bottom: "BatchNorm20" top: "BatchNorm20" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU20" type: "ReLU" bottom: "BatchNorm20" top: "BatchNorm20" } layer { name: "Convolution21" type: "Convolution" bottom: "BatchNorm20" top: "Convolution21" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout20" type: "Dropout" bottom: "Convolution21" top: "Dropout20" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat19" type: "Concat" bottom: "Concat18" bottom: "Dropout20" top: "Concat19" concat_param { axis: 1 } } layer { name: "BatchNorm21" type: "BatchNorm" bottom: "Concat19" top: "BatchNorm21" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale21" type: "Scale" bottom: "BatchNorm21" top: "BatchNorm21" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU21" type: "ReLU" bottom: "BatchNorm21" top: "BatchNorm21" } layer { name: "Convolution22" type: "Convolution" bottom: "BatchNorm21" top: "Convolution22" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout21" type: "Dropout" bottom: "Convolution22" top: "Dropout21" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat20" type: "Concat" bottom: "Concat19" bottom: "Dropout21" top: "Concat20" concat_param { axis: 1 } } layer { name: "BatchNorm22" type: "BatchNorm" bottom: "Concat20" top: "BatchNorm22" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale22" type: "Scale" bottom: "BatchNorm22" top: "BatchNorm22" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU22" type: "ReLU" bottom: "BatchNorm22" top: "BatchNorm22" } layer { name: "Convolution23" type: "Convolution" bottom: "BatchNorm22" top: "Convolution23" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout22" type: "Dropout" bottom: "Convolution23" top: "Dropout22" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat21" type: "Concat" bottom: "Concat20" bottom: "Dropout22" top: "Concat21" concat_param { axis: 1 } } layer { name: "BatchNorm23" type: "BatchNorm" bottom: "Concat21" top: "BatchNorm23" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale23" type: "Scale" bottom: "BatchNorm23" top: "BatchNorm23" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU23" type: "ReLU" bottom: "BatchNorm23" top: "BatchNorm23" } layer { name: "Convolution24" type: "Convolution" bottom: "BatchNorm23" top: "Convolution24" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout23" type: "Dropout" bottom: "Convolution24" top: "Dropout23" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat22" type: "Concat" bottom: "Concat21" bottom: "Dropout23" top: "Concat22" concat_param { axis: 1 } } layer { name: "BatchNorm24" type: "BatchNorm" bottom: "Concat22" top: "BatchNorm24" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale24" type: "Scale" bottom: "BatchNorm24" top: "BatchNorm24" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU24" type: "ReLU" bottom: "BatchNorm24" top: "BatchNorm24" } layer { name: "Convolution25" type: "Convolution" bottom: "BatchNorm24" top: "Convolution25" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout24" type: "Dropout" bottom: "Convolution25" top: "Dropout24" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat23" type: "Concat" bottom: "Concat22" bottom: "Dropout24" top: "Concat23" concat_param { axis: 1 } } layer { name: "BatchNorm25" type: "BatchNorm" bottom: "Concat23" top: "BatchNorm25" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale25" type: "Scale" bottom: "BatchNorm25" top: "BatchNorm25" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU25" type: "ReLU" bottom: "BatchNorm25" top: "BatchNorm25" } layer { name: "Convolution26" type: "Convolution" bottom: "BatchNorm25" top: "Convolution26" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout25" type: "Dropout" bottom: "Convolution26" top: "Dropout25" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat24" type: "Concat" bottom: "Concat23" bottom: "Dropout25" top: "Concat24" concat_param { axis: 1 } } layer { name: "BatchNorm26" type: "BatchNorm" bottom: "Concat24" top: "BatchNorm26" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale26" type: "Scale" bottom: "BatchNorm26" top: "BatchNorm26" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU26" type: "ReLU" bottom: "BatchNorm26" top: "BatchNorm26" } layer { name: "Convolution27" type: "Convolution" bottom: "BatchNorm26" top: "Convolution27" convolution_param { num_output: 304 bias_term: false pad: 0 kernel_size: 1 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout26" type: "Dropout" bottom: "Convolution27" top: "Dropout26" dropout_param { dropout_ratio: 0.2 } } layer { name: "Pooling2" type: "Pooling" bottom: "Dropout26" top: "Pooling2" pooling_param { pool: AVE kernel_size: 2 stride: 2 } } layer { name: "BatchNorm27" type: "BatchNorm" bottom: "Pooling2" top: "BatchNorm27" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale27" type: "Scale" bottom: "BatchNorm27" top: "BatchNorm27" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU27" type: "ReLU" bottom: "BatchNorm27" top: "BatchNorm27" } layer { name: "Convolution28" type: "Convolution" bottom: "BatchNorm27" top: "Convolution28" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout27" type: "Dropout" bottom: "Convolution28" top: "Dropout27" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat25" type: "Concat" bottom: "Pooling2" bottom: "Dropout27" top: "Concat25" concat_param { axis: 1 } } layer { name: "BatchNorm28" type: "BatchNorm" bottom: "Concat25" top: "BatchNorm28" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale28" type: "Scale" bottom: "BatchNorm28" top: "BatchNorm28" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU28" type: "ReLU" bottom: "BatchNorm28" top: "BatchNorm28" } layer { name: "Convolution29" type: "Convolution" bottom: "BatchNorm28" top: "Convolution29" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout28" type: "Dropout" bottom: "Convolution29" top: "Dropout28" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat26" type: "Concat" bottom: "Concat25" bottom: "Dropout28" top: "Concat26" concat_param { axis: 1 } } layer { name: "BatchNorm29" type: "BatchNorm" bottom: "Concat26" top: "BatchNorm29" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale29" type: "Scale" bottom: "BatchNorm29" top: "BatchNorm29" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU29" type: "ReLU" bottom: "BatchNorm29" top: "BatchNorm29" } layer { name: "Convolution30" type: "Convolution" bottom: "BatchNorm29" top: "Convolution30" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout29" type: "Dropout" bottom: "Convolution30" top: "Dropout29" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat27" type: "Concat" bottom: "Concat26" bottom: "Dropout29" top: "Concat27" concat_param { axis: 1 } } layer { name: "BatchNorm30" type: "BatchNorm" bottom: "Concat27" top: "BatchNorm30" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale30" type: "Scale" bottom: "BatchNorm30" top: "BatchNorm30" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU30" type: "ReLU" bottom: "BatchNorm30" top: "BatchNorm30" } layer { name: "Convolution31" type: "Convolution" bottom: "BatchNorm30" top: "Convolution31" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout30" type: "Dropout" bottom: "Convolution31" top: "Dropout30" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat28" type: "Concat" bottom: "Concat27" bottom: "Dropout30" top: "Concat28" concat_param { axis: 1 } } layer { name: "BatchNorm31" type: "BatchNorm" bottom: "Concat28" top: "BatchNorm31" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale31" type: "Scale" bottom: "BatchNorm31" top: "BatchNorm31" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU31" type: "ReLU" bottom: "BatchNorm31" top: "BatchNorm31" } layer { name: "Convolution32" type: "Convolution" bottom: "BatchNorm31" top: "Convolution32" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout31" type: "Dropout" bottom: "Convolution32" top: "Dropout31" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat29" type: "Concat" bottom: "Concat28" bottom: "Dropout31" top: "Concat29" concat_param { axis: 1 } } layer { name: "BatchNorm32" type: "BatchNorm" bottom: "Concat29" top: "BatchNorm32" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale32" type: "Scale" bottom: "BatchNorm32" top: "BatchNorm32" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU32" type: "ReLU" bottom: "BatchNorm32" top: "BatchNorm32" } layer { name: "Convolution33" type: "Convolution" bottom: "BatchNorm32" top: "Convolution33" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout32" type: "Dropout" bottom: "Convolution33" top: "Dropout32" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat30" type: "Concat" bottom: "Concat29" bottom: "Dropout32" top: "Concat30" concat_param { axis: 1 } } layer { name: "BatchNorm33" type: "BatchNorm" bottom: "Concat30" top: "BatchNorm33" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale33" type: "Scale" bottom: "BatchNorm33" top: "BatchNorm33" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU33" type: "ReLU" bottom: "BatchNorm33" top: "BatchNorm33" } layer { name: "Convolution34" type: "Convolution" bottom: "BatchNorm33" top: "Convolution34" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout33" type: "Dropout" bottom: "Convolution34" top: "Dropout33" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat31" type: "Concat" bottom: "Concat30" bottom: "Dropout33" top: "Concat31" concat_param { axis: 1 } } layer { name: "BatchNorm34" type: "BatchNorm" bottom: "Concat31" top: "BatchNorm34" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale34" type: "Scale" bottom: "BatchNorm34" top: "BatchNorm34" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU34" type: "ReLU" bottom: "BatchNorm34" top: "BatchNorm34" } layer { name: "Convolution35" type: "Convolution" bottom: "BatchNorm34" top: "Convolution35" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout34" type: "Dropout" bottom: "Convolution35" top: "Dropout34" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat32" type: "Concat" bottom: "Concat31" bottom: "Dropout34" top: "Concat32" concat_param { axis: 1 } } layer { name: "BatchNorm35" type: "BatchNorm" bottom: "Concat32" top: "BatchNorm35" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale35" type: "Scale" bottom: "BatchNorm35" top: "BatchNorm35" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU35" type: "ReLU" bottom: "BatchNorm35" top: "BatchNorm35" } layer { name: "Convolution36" type: "Convolution" bottom: "BatchNorm35" top: "Convolution36" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout35" type: "Dropout" bottom: "Convolution36" top: "Dropout35" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat33" type: "Concat" bottom: "Concat32" bottom: "Dropout35" top: "Concat33" concat_param { axis: 1 } } layer { name: "BatchNorm36" type: "BatchNorm" bottom: "Concat33" top: "BatchNorm36" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale36" type: "Scale" bottom: "BatchNorm36" top: "BatchNorm36" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU36" type: "ReLU" bottom: "BatchNorm36" top: "BatchNorm36" } layer { name: "Convolution37" type: "Convolution" bottom: "BatchNorm36" top: "Convolution37" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout36" type: "Dropout" bottom: "Convolution37" top: "Dropout36" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat34" type: "Concat" bottom: "Concat33" bottom: "Dropout36" top: "Concat34" concat_param { axis: 1 } } layer { name: "BatchNorm37" type: "BatchNorm" bottom: "Concat34" top: "BatchNorm37" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale37" type: "Scale" bottom: "BatchNorm37" top: "BatchNorm37" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU37" type: "ReLU" bottom: "BatchNorm37" top: "BatchNorm37" } layer { name: "Convolution38" type: "Convolution" bottom: "BatchNorm37" top: "Convolution38" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout37" type: "Dropout" bottom: "Convolution38" top: "Dropout37" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat35" type: "Concat" bottom: "Concat34" bottom: "Dropout37" top: "Concat35" concat_param { axis: 1 } } layer { name: "BatchNorm38" type: "BatchNorm" bottom: "Concat35" top: "BatchNorm38" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale38" type: "Scale" bottom: "BatchNorm38" top: "BatchNorm38" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU38" type: "ReLU" bottom: "BatchNorm38" top: "BatchNorm38" } layer { name: "Convolution39" type: "Convolution" bottom: "BatchNorm38" top: "Convolution39" convolution_param { num_output: 12 bias_term: false pad: 1 kernel_size: 3 stride: 1 weight_filler { type: "msra" } bias_filler { type: "constant" } } } layer { name: "Dropout38" type: "Dropout" bottom: "Convolution39" top: "Dropout38" dropout_param { dropout_ratio: 0.2 } } layer { name: "Concat36" type: "Concat" bottom: "Concat35" bottom: "Dropout38" top: "Concat36" concat_param { axis: 1 } } layer { name: "BatchNorm39" type: "BatchNorm" bottom: "Concat36" top: "BatchNorm39" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } } layer { name: "Scale39" type: "Scale" bottom: "BatchNorm39" top: "BatchNorm39" scale_param { filler { value: 1 } bias_term: true bias_filler { value: 0 } } } layer { name: "ReLU39" type: "ReLU" bottom: "BatchNorm39" top: "BatchNorm39" } layer { name: "Pooling3" type: "Pooling" bottom: "BatchNorm39" top: "Pooling3" pooling_param { pool: AVE global_pooling: true } } layer { name: "InnerProduct1" type: "InnerProduct" bottom: "Pooling3" top: "InnerProduct1" inner_product_param { num_output: 1000 bias_term: true weight_filler { type: "xavier" } bias_filler { type: "constant" } } } layer { name: "SoftmaxWithLoss1" type: "SoftmaxWithLoss" bottom: "InnerProduct1" bottom: "Data2" top: "SoftmaxWithLoss1" } layer { name: "Accuracy1" type: "Accuracy" bottom: "InnerProduct1" bottom: "Data2" top: "Accuracy1" }
标签:axis oba input lock 分支 false south access src
原文地址:http://www.cnblogs.com/hansjorn/p/7562447.html