码迷,mamicode.com
首页 > Web开发 > 详细

论文笔记-Sequence to Sequence Learning with Neural Networks

时间:2017-12-23 19:12:04      阅读:258      评论:0      收藏:0      [点我收藏+]

标签:imp   work   mit   gpo   post   思想   linked   def   idt   

大体思想和RNN encoder-decoder是一样的,只是用来LSTM来实现。

技术分享图片

paper提到三个important point:

1)encoder和decoder的LSTM是两个不同的模型

2)deep LSTM表现比shallow好,选用了4层的LSTM

3)实践中发现将输入句子reverse后再进行训练效果更好。So for example, instead of mapping the sentence a,b,c to the sentence α,β,γ, the LSTM is asked to map c,b,a to α,β,γ, where α, β, γ is the translation of a, b, c. This way, a is in close proximity to α, b is fairly close to β, and so on, a fact that makes it easy for SGD to “establish communication” between the input and the output.  

论文笔记-Sequence to Sequence Learning with Neural Networks

标签:imp   work   mit   gpo   post   思想   linked   def   idt   

原文地址:http://www.cnblogs.com/akanecode/p/8093752.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!