[转载]机器学习(1)：Logistic回归原理及其实现

原文链接:http://www.cnblogs.com/cv-pr/p/7081861.html

Logistic回归是机器学习中非常经典的一个方法，主要用于解决二分类问题，它是多分类问题softmax的基础，而softmax在深度学习中的网络后端做为常用的分类器，接下来我们将从原理和实现来阐述该算法的思想。

1.原理

拟合的效果图如下所示：

w

迭代的梯度变化非常快，五次就能达到非常好的结果，如下所示：

[ 4.54704357]

[ 0.19111694]

[ 0.2380104]

[ 0.01743344]

[  8.45306379e-05]

[  1.95907862e-09]

[  1.90137901e-16]

[  1.90137901e-16]

[  1.90137901e-16]

[  1.90137901e-16]

我们使用了python实现Logistic回归，注意：我们这里对H

是直接的求逆，如果特征维度很高的情况下，这会消耗较大的计算亮，因此我们可以采用更有效的求解方法，如Cholesky分解法，最后贴上马农最爱的代码：

import numpy as np

from matplotlib import pyplot as plt

class LogisticClassifier:

def __init__(self):

print("init");

def logistic(self,Xa,wa):

val = 1/(1+np.exp(-Xa.dot(wa)));

return val;

def train(self,X,t,iter_num):

print("start to training");

Xa = np.array(X)

xsize = Xa.shape

dim = xsize[1]+1

num = xsize[0]

Xa = np.c_[Xa,np.ones([num,1])]

ta = np.array(t)

print dim,num

wa = 0.5*np.ones([dim,1])

for it in range(iter_num):

ya = self.logistic(Xa,wa)

deriv_wa = Xa.T.dot(ya-ta)

R = np.diag((ya*(1-ya)).flat)

H = Xa.T.dot(R).dot(Xa)

delta_w = np.linalg.inv(H).dot(deriv_wa)

wa = wa - delta_w;

print np.linalg.norm(delta_w.T, 2, 1)

#print wa

return wa

if __name__ == "__main__":

print ('This is main of module "hello.py"')

logCls = LogisticClassifier();

#construct data

X = [[0.5],[0.75],[1],[1.25],[1.5],[1.75],[1.75],[2],[2.25],[2.5],[2.75],[3],[3.25],[3.5],[4],[4.25],[4.5],[4.75],[5],[5.5]]

t = [[0],[0],[0],[0],[0],[0],[1],[0],[1],[0],[1],[0],[1],[0],[1],[1],[1],[1],[1],[1]]

iter_num = 10;

#training weight

w = logCls.train(X, t, iter_num)

print ("learned weight:\n")

print w

#draw and show the result

pos_t = [x for i, x in enumerate(t) if x == [1]]

pos_X = [X[i] for i, x in enumerate(t) if x == [1]]

neg_t = [x for i, x in enumerate(t) if x == [0]]

neg_X = [X[i] for i, x in enumerate(t) if x == [0]]

plt.scatter(pos_X,pos_t,color="r",marker='o',s = 100)

plt.scatter(neg_X,neg_t,color="g",marker='o',s = 100)

Xfitted  = np.array(np.linspace(0,6,100))

XfittedC = np.c_[Xfitted,np.ones([100,1])]

Yfitted = logCls.logistic(XfittedC, w)

plt.plot(Xfitted.flat,Yfitted.flat,color="b",linewidth= 5)

#reset the axes

ax = plt.gca()

#no bonding box

ax.spines['top'].set_color('none')

ax.spines['right'].set_color('none')

#set as zero

ax.xaxis.set_ticks_position('bottom')

ax.spines['bottom'].set_position(('data',0.5))

ax.yaxis.set_ticks_position('left')

ax.spines['left'].set_position(('data',3))

plt.xlabel("X",fontsize="xx-large")

plt.ylabel("t",fontsize="xx-large")

plt.title("Logistic method,learned weight:[%f,%f]"%(w[0],w[1]),fontsize="xx-large")

plt.legend(["Fitted function","Postive Samples","Negative Samples"],fontsize="xx-large",loc='upper left');

plt.show()

3.参考资料

[1].Logistic回归与梯度下降法

[2].Logistic回归与牛顿迭代法

Make Change - Focus on Computer Vision and Pattern Recognition

版权声明：本文为博主原创文章，未经博主允许不得转载