Paper阅读笔记 | 01 Deep learning

01 引言

02 监督学习

03 反向传播和链式法则
x的微小变化影响到y的变化，再影响z的变化

In practice, poor local minima are rarely a problem with large net- works. Regardless of the initial conditions, the system nearly always reaches solutions of very similar quality.

前向传递：从前向后的输入
后向传递：从后向前的输入，误差传递的过程（学习过程）

04 卷积神经网络

Many data modalities are in the form of multiple arrays: 1D for signals and sequences, including language; 2D for images or audio spectrograms; and 3D for video or volumetric images.

The first few stages are composed of two types of layers: convolutional layers and pooling layers.

将预处理数据用于卷积运算，加上偏置得到局部特征，卷积核尺度及个数对模型对效果也有一定影响
将卷积输出结果进行非线性激活函数处理，目前常用ReLU
池化操作：取区域的平均值或最大值，保留其显著特征，提升对畸变的容忍能力

Two or three stages of convolution, non-linearity and pool- ing are stacked, followed by more convolutional and fully-connected layers. Backpropagating gradients through a ConvNet is as simple as through a regular deep network, allowing all the weights in all the filter banks to be trained.

全连接层进行输出，表示对结果的输出和确认

There are four key ideas behind ConvNets that take advantage of the properties of natural signals: local connections, shared weights, pooling and the use of many layers.

CNN的过程
用卷积核不断扫描图片中相关内容，对内容的重点区域进行循环的扫描

Deep neural networks exploit the property that many natural sig- nals are compositional hierarchies, in which higher-level features are obtained by composing lower-level ones.

CNN相比全连接网络的优点
减少参数，提高效率，降低运算复杂度

05 CNN理解图片

CNN-提取图片信息，看懂图像特征
RNN训练-RNN有能力注意图像的不同位置来生成单词
将图片高级表示翻译成字幕

06 分布表示和语言模型

Embedding: 将文本数据转化为数值数据
one-hot embedding：维度大
Word Embedding：向量空间接近的词意思接近，比较准确表示词的含义，维度小（300-500）

Word2Vec 算法
给定input word预测上下文

07 循环神经网络RNN

带有存储功能的神经网络

在每个时间点连接的参数值只有一份
神经网络除了输入以外，还会建立在以前的“记忆”的基础上
内存的要求与输入的规模有关

LSTM 长短记忆网络
解决长序列网络梯度消失和梯度爆炸问题
遗忘门，输入门，输出门

08 深度学习的未来

非监督学习
强化学习：reward奖励函数进行反馈
生成对抗网络：生成器、判别器。寻找二者之间的纳什均衡。

自监督学习
一切都在预测，现在预测未来，部分预测整体，未来预测现在，不 断调整预测的系统参数