SYSU_夏令营活动-研究笔记

2017-07-12  本文已影响0人  Yanring_

第一篇:

1.为什么要用DL

From : "A Tutorial Survey of Architectures, Algorithms, and
Applications for Deep Learning"

most machine learning and signal processing techniques had exploited shallow-structured architectures.(Gaussian mixture models (GMMs) and hidden Markov models (HMMs) , support vector machines (SVMs), logistic regression....)

Shallow architectures have been shown effective in solving many simple or well-constrained problems, but their limited modeling and representational power can cause difficulties when dealing with more complicated real-world applications involving natural signals such as human speech, natural sound and language, and natural image and visual scenes.(不适用于复杂的应用,因此需要深层的结构来解决这些问题)

Human information processing mechanisms (e.g., vision and speech), however, suggest the need of deep architectures for extracting complex structure and building internal representation from rich sensory inputs. For example, human speech production and perception systems are both equipped with clearly layered hierarchical structures in transforming the information from the waveform level to the linguistic level.(深度学习的多层结构与人类处理信息的方式相似(不是太理解人类处理信息是怎么分层的),对事物的建模或抽象表现能力更强,也能模拟更复杂的模型。)

发展于神经网络,但是不能用BP神经网络(因为基于局部梯度下降,经常被困在较差的局部最优中,随着网络深度的增加,严重程度会显着增加)

2.深度学习的历史

deep belief network (DBN)

which is composed of a stack of Restricted Boltzmann Machines (RBMs). A core component of the DBN is a greedy, layer-by-layer learning algorithm which optimizes DBN weights at time complexity linear to the size and depth of the networks.

DBN的一个核心组件是一个贪心的、逐层的学习算法,能够在较好时间复杂度下优化权重

我的理解:DBN有一个非监督预训练的过程,使用RBM进行权重偏置的初始化。为DNN提供较好的初始参数

Use DBN to facilitate the training of DNNs plays an important role in igniting the interest of deep learning for speech feature coding and for speech recognition. (DBN的生成模型可以有效的促进DNN的训练,不过也有很多更简单的替代方法,比如说从浅层开始训练:训练->插入新的隐藏层->再对整体训练->插入新的....)

3.三大类深层架构

实践环节

BigDl的安装

不久前已经装过了hadoop和hdfs,所以这次就直接用了。

SPARK

学习了下spark的一些运行原理:

以及跑了下word2vec的示例:


image.png image.png
上一篇下一篇

猜你喜欢

热点阅读