循环神经网络层

目录

  • Recap
  • Sentiment Analysis
    • Proposal
  • S1.Weight sharing
    • Naive version
    • Weight share
  • S2.Consistent memory
  • Unfolded model
  • Formulation
  • Overall Diagram
  • One more thing
  • How to Train?

Recap

Sentiment Analysis

Proposal

  • Long sentence
    • 100 + words
    • too much parameters[\(w_N,b_N\)]
  • No context information
    • consistent tensor

S1.Weight sharing

  • 类似于卷积的卷积核,卷积视野,权重分享

Naive version

Weight share

  • 权重分享

S2.Consistent memory

  • 需要一个东西存储语境,也就是每个单词的语境信息

  • 不断地对语境信息进行堆叠得到h5,直接使用h5作为判断标准

Unfolded model

  • 在时间轴上折叠,不断地更新h

Formulation

  • 通过激活函数tanh不断地叠加上个时间戳的信息

Overall Diagram

  • 全览分解图

One more thing

How to Train?

你可能感兴趣的:(循环神经网络层)