TextRNN
论文：Recurrent Neural Network for TextClassification with Multi-Task Learning
1 Introduction
DNN缺点：usually need a large-scale corpus due to the large number of parameters,it is hard to train a network that generalizes well with limited data
The first model uses just one shared layer for all the tasks.
The second model uses different layers for different tasks, but each layer can read information from other layers.
The third model not only assigns one specific layer for each task, but also builds a shared layer for all the tasks.
2 Recurrent Neural Network for Specific-Task Text Classification
2.1 Recurrent Neural Network
缺陷：梯度消失
RNN
The activation of the hidden state ht
LSTM :learning long-term dependencies
LSTM
The LSTM transition equations
论文：A Bi-LSTM-RNNModel for Relation Classification Using Low-Cost Sequence Features
1. Introduction
①performs bi-directional recurrent computation along all the tokens of the sentences which the relation spans.
②the sequence of token representations, which are generated in the previous step, is divided into five parts according to the order that tokens occur in these sentences，picture1
③standard pooling functions are applied over the token representations of each part and we obtain five representations corresponding to the five parts.
④they are concatenated and fed into a softmaxlayer for relation classification
picture1
LSTMs are used to attenuate the gradientvanishing problem when two target entities aredistant in text.
2. Related Work
3. Our Bi-LSTM-RNN Model
3.1. Long Short Term Memory (LSTM)
equation1
3.2. Bi-LSTM-RNN
Bi-LSTM-RNN
补充：
