5-paper-resnet&Differiential equ

The relation between ordinary/partial equation/dynamic system & resnet/...net(neural network):

*From Yiping Lu:
intuition:similarity in each residual layer formulation and the formulation of discretialization of ODE，nn can be interpreted as different discrete approximations of dynamic sysm.

1.stability of dynamic system/PDE/ODE to analyze and improve the stability of nn which means the vanishing or explpding in feed-forward process. And this is related to/affects the generalization ability of the network. 弥散或爆炸现象

2.optimization ,cause pde and nn ，所以从control theory and pde d-e 中的 tools to analyze the nn, geometry insights physical laws smart design of numerical schemes to create more outstanding network design.

3.numerical methods in control theory may inspire us new optimization algorithms. 范

4.stochastic dynamic system is related to the injecting noise to net (stochastic learning strategy) such as dropping out.so we may find some principles to gudie us from this prospective. and
细：随即控制策略 ;noise injection such as drop out can be seen as a weak approximation of this.

做research：1.客观评价他人和自己的工作，不能过度贬低或吹捧
2.或者说他人的工作是我们工作的special case.

key words: stochastic dynamic systems;dynamic systems; residual network; network design;

总之，这篇文章，提到了许多insights ,这样设计的几个来源，想法，why。