处理txt文件并去重处理

1.排序

cat 200W10.txt  200W1.txt  200W3.txt  200W5.txt  200W7.txt  200W9.txt 200W11.txt  200W2.txt  200W4.txt  200W6.txt  200W8.txt  200W.txt

>allw.txt

sort allw.txt | uniq >allnw.txt

2.在user表中创建对应的字段

3.将文本数据导入到数据库中

load data local infile "d:/allw2.txt" into table user fields terminated by '-' lines terminated by '\r\n';