决策树算法

1. Python机器学习库：scikit-learn
   1.1 特性：简单高效的数据挖掘和机器学习分析，对所有用户开放，根据不同需求高度可重用形，基于Numpy，SciPy和matplotlib，开源，商用级别：获得BSD许可
   2.1 覆盖问题领域：分类(classification)，回归(regression)，聚类(clustering)，降维(dimensioniality reduction)，模型分类(model selection)，预处理(preprocessing)

2. 案例

from sklearn.feature_extraction import DictVectorizer
import csv
from sklearn import preprocessing
from sklearn import tree
from sklearn.externals.six import StringIO

# Read in the csv file and put features in alist of dict and list of class label
allElectronicsData = open(r'AllElectronics.csv', 'rt') # 由于python版本问题,读取方式改为rt
reader = csv.reader(allElectronicsData)
headers = next(reader)

# print(headers)

featuresList = [] # 特征值列表
labelList = [] # 类别值 Class buys computer

for row in reader: # 每一个row就是每一行
   labelList.append(row[len(row) - 1]) # 取每一行最后一个值 Class buys computer
   rowDict = {} # 我们要取每一行的特征值,除了第一行
   for i in range(1, len(row) - 1):
       # print(row[i])
       rowDict[headers[i]] = row[i]
       # print(rowDict)
   featuresList.append(rowDict)

# print(featuresList)

# Yetorize features
vec = DictVectorizer()
dummyX = vec.fit_transform(featuresList).toarray() # 转换成01格式

print(dummyX)
print(vec.get_feature_names())
print('labelList：', labelList)

# Yectorize class labels
lb = preprocessing.LabelBinarizer() # 将yes/no转换为01
dummyY = lb.fit_transform(labelList)
print('dummyY：', dummyY)

# Using decision tree for classification
# clf = tree.DecisionTreeClassfier() 分类器
clf = tree.DecisionTreeClassifier(criterion='entropy') # 信息熵之间的差异
clf = clf.fit(dummyX, dummyY)
print('clf：', clf)

# yisulize model
# with open(r'allElectronicGiniOri.dot', 'w') as f:
with open(r'allElectronicInformationGainOri.dot', 'w') as f:
   # f = tree.export_graphviz(clf, out_file = f)
   f = tree.export_graphviz(clf, feature_names=vec.get_feature_names(), out_file=f)


# 假设有新数据
oneRowX = dummyX[0, :]
print('oneRowX：', oneRowX)

newRowX = oneRowX

newRowX[0] = 1
newRowX[2] = 0
print('newRowX：', newRowX)

运行结果我们得到

dot

但是这样看起来不够直观，我们可以使用Graphviz转换dot文件至pdf可视化决策树，效果如下：

决策树可视化

安装 Graphviz： http://www.graphviz.org/
安装后可以自行百度或google查找配置环境变量
转化dot文件至pdf可视化决策树：dot -Tpdf iris.dot -o outpu.pdf