Neural Networks Notes 2: 神经网络如何学
还是参考3b1b的视频系列，感觉他们讲的比较清楚。
Chapter 2: Gradient descent，how neural networks learn ？
目录如下：
- Recap
- Gradient descent
- Analyze this network
- Where to learn more
- Research corner
-
Recap
上节课讲了识别手写数字的问题，相当于神经网络的hello world。 -
Gradient descent
如何判断一个神经网络结果好坏？设定一个'Cost' Function, 结果的值越小，神经网络就训练得越好。
Only tell the computer how bad the result is isn't really helpful, we should tell it how to improve the results.
To make it easier rather than struggling to imagine a function with 13000 inputs, just imagine a simple function that has one number as an input and one number as an output. 在这个时候，找最低的y值，是在一个点计算它的slope，然后决定往哪个方向移动。就算是这样，也可能有很多谷，所以global minimum是很难的。现在考虑有两个变量的情况，在三维图当中是一个平面，而我们要做的就是计算它的梯度，来找到下降的方向。
The algorithm for computing this gradient efficiently which is effectively the heart of how a neural network learns is called back propagation (反向传播).
Now it's time to take the time to walk through what exactly happens to each weight and each bias for a given piece of training data? Trying to give an intuitive feel --
hhhhh
什么是学习呢？ It means minimizing the cost function. One consequence of that is that it's important for this cost function to have a nice smooth output. 这样我们才能通过取一小步一小步来找到局部最优解。这也是为什么人工神经元拥有连续的值，而不是像生物神经元只有被激活／不激活的binary values。This process of repeatedly nudging an input of a function by some multiple of the negative gradient is called Gradient Descent.
它告诉你不同的weight哪些是重要的哪些是不重要的。
Which changes to which weights matter the most?
-
Analyze this network
之前谈到我们对于每个layer的作用的预测，事实真的是这样的吗？ Well, not at all. 我们把每个weights做成pixel图，看起来是比较随机的。给一个一团随机的输入，我们的直觉是smart的网络应该给出一个均匀的结果，但是我们会发现这个network会很自信地给出某个答案，而且cost也很小。
wow！
-
Where to learn more
Now pause and think for a while, what changes would you make to this system and how it perceives images if you wanted it to pick up on things like edges and patterns?
推荐：Michael Nielson 的关于深度学习和神经网络的书籍
（这个不翻墙好像也能上去。
这里有Code和data可以下载来玩一玩～ 它的书会一步一步教你code的每一步都是干什么的。这本书是免费的而且公开的，如果你支持作者可以在他的网页Donate一下～
- Research Corner
