AllenNLP学习记录，关于ELMo配置问题

随手翻译的，一定要把这块搞定。

在最简单的情况下，将ELMo添加到现有模型是一个简单的配置更改。我们提供了一个TokenEmbedder，它接受字符id作为输入，运行深度biLM并通过学习加权组合计算ELMo表示。

这个案例中最终的模型只包含1层ELMo。而在某些案例中，包含多层能够提高成绩，多层需要修改代码
使用AllenNLP中已有的SRL模型，（该模型是一个语义角色标注（SRL）模型，目标是恢复一个句子的谓词-论元结构，来做出基本判断：“谁对谁做了什么”，“何时”和“哪里”）不使用ELMo时，该模型使用的是一个100维的预训练GloVe向量。
这时原有的jsonnet

{
  "dataset_reader":{"type":"srl"},
  "train_data_path": std.extVar("SRL_TRAIN_DATA_PATH"),
  "validation_data_path": std.extVar("SRL_VALIDATION_DATA_PATH"),
  "model": {
    "type": "srl",
    "text_field_embedder": {
      "token_embedders": {
        "tokens": {
            "type": "embedding",
            "embedding_dim": 100,
            "pretrained_file": "https://s3-us-west-2.amazonaws.com/allennlp/datasets/glove/glove.6B.100d.txt.gz",
            "trainable": true
        }
      }
    },
    "initializer": [
      [
        "tag_projection_layer.*weight",
        {
          "type": "orthogonal"
        }
      ]
    ],
    "encoder": {
      "type": "alternating_lstm",
      "input_size": 200,
      "hidden_size": 300,
      "num_layers": 8,
      "recurrent_dropout_probability": 0.1,
      "use_highway": true
    },
    "binary_feature_dim": 100
  },
  "iterator": {
    "type": "bucket",
    "sorting_keys": [["tokens", "num_tokens"]],
    "batch_size" : 80
  },

  "trainer": {
    "num_epochs": 500,
    "grad_clipping": 1.0,
    "patience": 20,
    "validation_metric": "+f1-measure-overall",
    "cuda_device": 0,
    "optimizer": {
      "type": "adadelta",
      "rho": 0.95
    }
  }
}

要加入ELMo，有三个地方要进行调整

• text_field_embedder增加emlo小节
• dataset_reader增加emlo小节
• 修改input_size

text_field_embedder

    "text_field_embedder": {
      "token_embedders": {
        "tokens": {
            "type": "embedding",
            "embedding_dim": 100,
            "pretrained_file": "https://s3-us-west-2.amazonaws.com/allennlp/datasets/glove/glove.6B.100d.txt.gz",
            "trainable": true
        }
      }
    },

改为

    "text_field_embedder": {
            "token_embedders": {
                "elmo": {
                    "type": "elmo_token_embedder",
                    "options_file": "https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway/elmo_2x4096_512_2048cnn_2xhighway_options.json",
                    "weight_file": "https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway/elmo_2x4096_512_2048cnn_2xhighway_weights.hdf5",
                    "do_layer_norm": false,
                    "dropout": 0.1
                }
            }

dataset_reader增加emlo小节

    "dataset_reader":{"type":"srl"},

改为

    "dataset_reader":{
    "type":"srl",
    "token_indexers": {
            "elmo": {
                "type": "elmo_characters"
            }
    }},

修改input_size

"encoder": {
      "type": "alternating_lstm",
      "input_size": 200,
      "hidden_size": 300,
      "num_layers": 8,
      "recurrent_dropout_probability": 0.1,
      "use_highway": true
    },

改为

"encoder": {
            "type": "alternating_lstm",
            "input_size": 1124,
            "hidden_size": 300,
            "num_layers": 8,
            "recurrent_dropout_probability": 0.1,
            "use_input_projection_bias": false
        },