Uber的快速深度学习模型构建包：Ludwig尝鲜评估

2019-02-18 本文已影响16人 blade_he

简介

最近各大厂商有关深度学习神经网络的快速模型构建的组件包，层出不穷，Uber也不甘其后，推出了基于TensorFlow的工具箱: Ludwig
该工具集已经发布到了GitHub

2019-02-18 15_12_04-uber_ludwig_ Ludwig is a toolbox built on top of TensorFlow that allows to train.png

特点

这个工具箱甚至允许不写任何代码就能够训练神经网络模型，示例网页：Examples

2019-02-18 15_13_15-Examples - Ludwig.png

支持的应用包括：

Text Classification
Named Entity Recognition Tagging
Natural Language Understanding
Machine Translation （支持Attention机制）
Chit-Chat Dialogue Modeling through Sequence2Sequence（支持Attention机制）
Sentiment Analysis
Image Classification
Image Captioning
One-shot Learning with Siamese Networks
Visual Question Answering
Kaggle's Titanic: Predicting survivors
Time series forecasting
Movie rating prediction
Multi-label classification
Multi-Task Learning
看到支持的类型，我的心情是这样的：

duang.jpg

安装

ludwig依赖的组件包：

Cython>=0.25
h5py>=2.6
matplotlib>=3.0
numpy>=1.15,<1.16
pandas>=0.19
scipy>=0.18
scikit-image
scikit-learn
seaborn>=0.7
spacy>=2.0
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
tqdm
tabulate>=0.7
tensorflow>=1.12.0
PyYAML>=3.12
pytest

安装的方式如下，说明ludwig是依赖Spacy的

pip install ludwig
python -m spacy download en

初试牛刀

实现模型的训练，最简单的方式，准备训练集，如情感分析，数据集是如下的csv:

2019-02-18 15_23_34-movie_reviews.csv - Excel.png

假如路径为：./model/movie_reviews.csv
然后准备模型的描述YAML文件，文件路径为：./model/movie_sentiment_model_definition.yaml
内容如下：

需要特别说明的是：
情感分析的例子，按照官方说明的默认配置：

input_features:
    -
        name: review
        type: text
        encoder: parallel_cnn
        level: word
output_features:
    -
        name: sentiment
        type: category

效果是非常差的，训练3次之后，准确率在50%之后，就没有提升了，这与抛硬币也没什么区别啊。。。

2019-02-18 17_02_20-Photos.png

经过实践证明，type设置为sequence，encoder设置为rnn是相对靠谱的，训练5次之后，准确率如下：

2019-02-18 17_02_50-Photos.png

训练10次之后，准确率如下：

2019-02-18 17_14_22-Photos.png

所以对于文本方面的处理，按照如下设置，训练才有一定意义。

input_features:
    -
        name: review
        type: sequence
        encoder: rnn
        cell_type: lstm
        bidirectional: true
        num_layers: 2
        reduce_output: None

output_features:
    -
        name: sentiment
        type: category
training:
    epochs: 100
    dropout: True
    dropout_rate: 0.1
    early_stop: 100

可以发现，这个配置是RNN文本分类模型相关，并且设置为BI-LSTM机制
安装ludwig完毕之后，训练这个模型，只需要在控制台录入：

ludwig experiment --data_csv ./model/movie_reviews.csv --model_definition_file ./model/movie_sentiment_model_definition.yaml

然后就搞定模型的训练了。
训练过程中，对于训练集中的文本，会生成与训练集同名的文本结构描述文件，如：movie_reviews.json（因为内容太大，所以截图结构折叠后的样子）：

2019-02-18 15_47_17-D__reviews.json - Notepad+.png

会自动在项目路径生成如下目录结构：results_run_0
此目录下会有一个模型描述json文件：results_run_0\description.json
我们可以在这里找到我们所熟悉的模型相关的超参数：

{
    "command": "ludwig experiment ",
    "dataset_type": "generic",
    "input_data": "./model/movie_reviews.csv",
    "ludwig_version": "0.1.0",
    "model_definition": {
        "combiner": {
            "type": "concat"
        },
        "input_features": [
            {
                "bidirectional": true,
                "cell_type": "lstm",
                "encoder": "rnn",
                "name": "review",
                "num_layers": 2,
                "reduce_output": "None",
                "tied_weights": null,
                "type": "sequence"
            }
        ],
        "output_features": [
            {
                "dependencies": [],
                "loss": {
                    "class_distance_temperature": 0,
                    "class_weights": 1,
                    "confidence_penalty": 0,
                    "distortion": 1,
                    "labels_smoothing": 0,
                    "negative_samples": 0,
                    "robust_lambda": 0,
                    "sampler": null,
                    "type": "softmax_cross_entropy",
                    "unique": false,
                    "weight": 1
                },
                "name": "sentiment",
                "reduce_dependencies": "sum",
                "reduce_input": "sum",
                "top_k": 3,
                "type": "category"
            }
        ],
        "preprocessing": {
            "bag": {
                "fill_value": "",
                "format": "space",
                "lowercase": 10000,
                "missing_value_strategy": "fill_with_const",
                "most_common": false
            },
            "binary": {
                "fill_value": 0,
                "missing_value_strategy": "fill_with_const"
            },
            "category": {
                "fill_value": "<UNK>",
                "lowercase": false,
                "missing_value_strategy": "fill_with_const",
                "most_common": 10000
            },
            "force_split": false,
            "image": {
                "missing_value_strategy": "backfill"
            },
            "numerical": {
                "fill_value": 0,
                "missing_value_strategy": "fill_with_const"
            },
            "sequence": {
                "fill_value": "",
                "format": "space",
                "lowercase": false,
                "missing_value_strategy": "fill_with_const",
                "most_common": 20000,
                "padding": "right",
                "padding_symbol": "<PAD>",
                "sequence_length_limit": 256,
                "unknown_symbol": "<UNK>"
            },
            "set": {
                "fill_value": "",
                "format": "space",
                "lowercase": false,
                "missing_value_strategy": "fill_with_const",
                "most_common": 10000
            },
            "split_probabilities": [
                0.7,
                0.1,
                0.2
            ],
            "stratify": null,
            "text": {
                "char_format": "characters",
                "char_most_common": 70,
                "char_sequence_length_limit": 1024,
                "fill_value": "",
                "lowercase": true,
                "missing_value_strategy": "fill_with_const",
                "padding": "right",
                "padding_symbol": "<PAD>",
                "unknown_symbol": "<UNK>",
                "word_format": "space_punct",
                "word_most_common": 20000,
                "word_sequence_length_limit": 256
            },
            "timeseries": {
                "fill_value": "",
                "format": "space",
                "missing_value_strategy": "fill_with_const",
                "padding": "right",
                "padding_value": 0,
                "timeseries_length_limit": 256
            }
        },
        "training": {
            "batch_size": 64,
            "bucketing_field": null,
            "decay": false,
            "decay_rate": 0.96,
            "decay_steps": 10000,
            "dropout": true,
            "dropout_rate": 0.1,
            "early_stop": 100,
            "epochs": 100,
            "gradient_clipping": null,
            "increase_batch_size_on_plateau": 0,
            "increase_batch_size_on_plateau_max": 512,
            "increase_batch_size_on_plateau_patience": 5,
            "increase_batch_size_on_plateau_rate": 2,
            "learning_rate": 0.001,
            "learning_rate_warmup_epochs": 5,
            "optimizer": {
                "beta1": 0.9,
                "beta2": 0.999,
                "epsilon": 1e-08,
                "type": "adam"
            },
            "reduce_learning_rate_on_plateau": 0,
            "reduce_learning_rate_on_plateau_patience": 5,
            "reduce_learning_rate_on_plateau_rate": 0.5,
            "regularization_lambda": 0,
            "regularizer": "l2",
            "staircase": false,
            "validation_field": "combined",
            "validation_measure": "loss"
        }
    },
    "random_seed": 42
}

训练所得的模型结构如下：

2019-02-18 15_38_01-Photos.png

对于模型的预测，训练完毕之后，假定有如下数据需要预测：
./data/test_data.csv

2019-02-18 15_44_37-test_data.csv - Excel.png

因为已经获得训练之后模型的路径了，只需要在控制台录入：

ludwig predict --data_csv ./data/test_data.csv --model_path ./results/_run_0/model

之后，Ludwig又会给我们生成一个results_0目录，其中就是预测的结果，这个预测结果是根据之前那个不合理的配置生成的模型，得到的，因此只有预览的意义：

2019-02-18 15_49_58-Q-Dir 6.17.png

预测的类别名称：

2019-02-18 15_50_33-sentiment_predictions.csv - Excel.png
预测的概率：

2019-02-18 15_50_55-sentiment_probability.csv - Excel.png

之所以看起来预测结果这么不靠谱，是因为只训练了两次，真正业务训练，对于这种大小的训练集，至少也应该训练100次以上了。

通过python调用Ludwig API

这个就很简单了，按照如下代码写即可，加入logging_level=logging.DEBUG这一句的目的是能够正常输出训练模型的日志。

"""
@version: 0.1
@author: Blade He
@site: 
@software: PyCharm
@file: main.py
@time: 2019/02/18
"""
from ludwig import LudwigModel
import yaml
import logging

def startjob(csv_file_path = r'./model/movie_reviews.csv',
             model_file = r'./model/movie_sentiment_model_definition.yaml',
             test_file = r'./data/test_data.csv'):
    with open(model_file, encoding='utf-8', mode='r') as file:
        model_definition = yaml.load(file.read())
        print(model_definition)
        ludwig_model = LudwigModel(model_definition)
        train_stats = ludwig_model.train(data_csv=csv_file_path,
                                         logging_level=logging.DEBUG)
        print(train_stats)
        predictions = ludwig_model.predict(data_csv=test_file,
                                           logging_level=logging.DEBUG)
        print(predictions)
        ludwig_model.close()
 
if __name__ == '__main__':
    startjob()

小结

Uber这次真的是把神经网络的应用，做成工具包了，是真正意义上的，无需写代码，通过命令行+模型配置文件的形式，就可以训练与使用神经网络模型。
这应该就是趋势吧

Uber的快速深度学习模型构建包：Ludwig尝鲜评估

简介

特点

安装

初试牛刀

通过python调用Ludwig API

小结

猜你喜欢

热点阅读