结巴分词

1. python环境下下载jieba分词

参考网址：https://blog.csdn.net/robin_xu_shuai/article/details/53306686

安装方法：cmd->pip3 install jieba

2.对训练集进行分词（按行）

代码如下：

import jieba

with open('myInput.txt','r')as f:

  for line in f:

    seg = jieba.cut(line.strip(),cut_all = False)

    output = '/'.join(seg)

    output = output+'\n'

    with open('myOutput.txt','a+')as s:

    s.write(output)