码迷,mamicode.com
首页 > Web开发 > 详细

Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1

时间:2018-02-25 20:47:50      阅读:382      评论:0      收藏:0      [点我收藏+]

标签:alt   problem   reg   work   die   and   which   9.png   vanish   

 

Normalizing input  

技术分享图片

 

技术分享图片

 

 

Vanishing/Exploding gradients

deep neural network suffer from these issues. they are huge barrier to training deep neural network.

技术分享图片

 

There is a partial solution to solve the above problem but help a lot which is careful choice how you initialize the weights. 主要目的是使得weight W[l]不要比1太大或者太小,这样最后在算W的指数级的时候就不会有vanishing 和 exploding的问题

Weight Initialization for Deep Networks

技术分享图片

 

 

Ref:

1. Coursera

Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1

标签:alt   problem   reg   work   die   and   which   9.png   vanish   

原文地址:https://www.cnblogs.com/mashuai-191/p/8466675.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!