EM算法看一次忘一次，干脆整理一下好了。

一. 最大似然算法

基本思想：从已经观察到有限个samples推测出该群体最合理的distribution.

max \ Lik\left(D,z | \theta \right) in order to find the best \theta. theta is distribution parameters( means or var)

二. EM算法

基本思想：从已经观察到的有限个samples和一些未知的samples推测出该群体最合理的distribution.

max \ Lik\left(D,z | \theta \right) in order to find the best \theta. z are hidden variables.

具体操作

E step : fix \theta_k. max Likelihood function with respect to hidden variable z_k.

M step : fix z_k obtained from previous step, max Likelihood function with respect to the parameter \theta_{k+1}.

直觉

坐标上升法

推导

Chapter 2

Reference

http://www.cmlab.csie.ntu.edu.tw/~cyy/learning/tutorials/EM.pdf