Day2 #100DaysofMLCoding#

2018-8-7

今日内容

1. 继续深度学习Udacity，初步刷完，准备看配套资料，着手做 "街道数字识别"
   配套《Tensorflow实战……》研究tensorflow与深度学习

资料整理

论文

• Netzer, Y., et al. Reading Digits in Natural Images with Unsupervised Feature Learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.

• Goodfellow, I., et al. Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks. ICLR, 2014. [video]

相关资源

• UFLDL Deep Learning Tutorial

• DeepLearning.net: A collection of papers, software, dataset and current events.

• Deep Learning Summer School, Montreal 2015.

课程

• CS231n: Convolutional Neural Networks for Visual Recognition, Fei-Fei Li (Stanford).
• CS224d: Deep Learning for Natural Language Processing, Richard Socher (Stanford).

小知识点

1. dropout，让每个链接都可靠，冗余学习，学习慢但更可靠
   屏蔽一部分链接，其他链接乘以相应倍数

2. numpy技巧

   
   

3. 权重共享 统计不变性(statistic invariants)

   
   
   
   

   卷积计算 权重共享 与位置无关

4. 1*1 卷积作用
   加入非线性,跨通道信息交互,降维

5. TSNE可以在降维的同时保留临近结构

   
   

6. word2vec

• sample softmax
• 用余弦距离比不用L2
• 类比

7. LSTM RNN中解决梯度消失