Nonelinear Model

2018-10-21 本文已影响0人 asuka_19d5

decision tree is a supervised prediction model
it is usually a k-nary tree. each leaf denotes one classification or continuous value of the dependent variable.
two types of dependent variable:
- classification tree
- regression tree
how to build a decision tree:
- two steps:
  feature prioritization
  feature splitting
- Algorithnm Details - ID3 Algorithm
  at each iteration step, we split the feature which can give us the largest entropy gain 直观理解：减少偏见
  Definition of Entropy: $I_e = - log_2p_i$ Entropy of the entire information system: $H = \sum_{i=1}^np_i(\sum^m_{j=1}p_jI_e)$
  image.png
  image.png
other tree algortithms:
- C4.5 Algorithm: Gain Rate
- CART: Gini impurity
  
  image.png