python笔记

Python 多线程&多进程

2018-08-11  本文已影响2人  weifeng_genius

进程与线程

多线程

 threads = []    #线程列表
    for i in range(0,100):
        thread = threading.Thread(target=fun,args=(i,i+5),kwargs = {}) 
#args= () 传入默认参数(传入一个tuple只有一个数据的时候(a,)),kwargs ={}传入关键字参数
        threads.append(thread)
        thread.start()    #运行
        #创建一个jion()方法来堵塞,让所有线程执行才执行最后的Done
    for i in threads:
        i.join()

线程同步

lock = threading.Lock()
lock.acquire()    #获取锁
try:
    fun(n)    #执行某个函数
finally:
    lock.release()    #释放锁

全局解释器锁(GIL)

多进程

from multiprocessing import Process
import os 


def test(name):
    print('child process %s,id is %s,his parent is %s' % (name,os.getpid(),os.getppid()))


if __name__ == '__main__':   
    ps = []    
    for i in range(3):
        p = Process(target=test,args=('haha',))
        print('parent is %s' % os.getpid())
        ps.append(p)
        p.start()
    for i in ps:
        i.join()
    print('end')

运行结果:

parent is 23796
parent is 23796
parent is 23796
child process haha,id is 24300,his parent is 23796
child process haha,id is 2296,his parent is 23796
child process haha,id is 3100,his parent is 23796
end

进程池(Pool)

from multiprocessing import Pool
import os 
import time
import random


def long_time_task(name):
    print('Run task %s (%s)...parent id %s' % (name, os.getpid(),os.getppid()))
    start = time.time()
    time.sleep(random.random() * 3)
    end = time.time()
    print('Task %s runs %0.2f seconds.' % (name, (end - start)))

if __name__=='__main__':
    print('Parent process %s.' % os.getpid())
    p = Pool()
    for i in range(5):
        p.apply_async(long_time_task, args=(i,))  #分配一个进程池中的进程给函数用。
    print('Waiting for all subprocesses done...')
    p.close()
    p.join()
    print('All subprocesses done.')
#def test(name):

运行结果:

Parent process 17872.
Waiting for all subprocesses done...
Run task 0 (11256)...parent id 17872
Run task 1 (20664)...parent id 17872
Run task 2 (13540)...parent id 17872
Run task 3 (8312)...parent id 17872
Task 1 runs 0.31 seconds.
Run task 4 (20664)...parent id 17872
Task 2 runs 0.39 seconds.
Task 3 runs 1.25 seconds.
Task 0 runs 2.75 seconds.
Task 4 runs 2.52 seconds.
All subprocesses done.

进程间通信

1.Queue
有两个方法 .put().get(),都具有两个可选参数:blocked(默认值为True,表明可以堵塞一段timeout时间来等待队列有剩余空间(.put())或是等待Queue中有数据.get()) 和 timeout(指定带等待使时间)

from multiprocessing import Process,Pool,Queue
import os ,time,random


# 写数据
def write(q):
    for value in ['A','B','C']:
        print('put {} to queue'.format(value))
        q.put(value)
        time.sleep(random.random())


#读数据
def read(q):
    while True:
        value = q.get()
        print('get {} from quete'.format(value))
    
    
if __name__ == '__main__':
    q = Queue()    #主进程创建Queue,作为参数传给子进程
    pw = Process(target = write , args=(q,))
    pr = Process(target = read , args=(q,))
    pw.start()
    pr.start()
    pw.join()
    pr.terminate()    # pr进程为死循环,不能等待其结束,只能强行终止。

运行结果:

put A to queue
get A from quete
put B to queue
get B from quete
put C to queue
get C from quete

2.Pipe(常用于两个进程间通信)

from multiprocessing import Process,Pool,Queue,Pipe
import os ,time,random


#发送端
def send(pipe):
    for i in ['a','b','c']:
        print('Process %s send:%s' % (os.getpid(),i))
        pipe.send(i)
        time.sleep(random.random())
        

#接受端
def recv(pipe):
    while True:
        print('Process %s receive:%s' %(os.getpid(),pipe.recv()))
        time.sleep(random.random())
        
    
if __name__ == "__main__":
    pipe = Pipe()    #创建Pipe对象,返回值为一个tuple.
    ps = Process(target = send, args = (pipe[0],))    #传入tuple的第一项作为发送
    pr = Process(target = recv, args = (pipe[1],))    #传入tuple的第二项作为接收
    ps.start()
    pr.start()
    ps.join()
    pr.terminate()

运行结果:

Process 22332 send:a
Process 1884 receive:a
Process 22332 send:b
Process 1884 receive:b
Process 22332 send:c

多线程与多进程

1.多进程的优点是稳定性高,一个子进程崩溃不会影响其他的进程。缺点在于创建进程的开销很大,而且操作系统能同时运行的进程数有限,系统调度会出现问题。
2.多线程比多进程快一点点,缺点在于一个子线程崩可能导致整个进程崩溃。Windows下,多线程比多进程效率要高。
3.无论多进程多线程都要由一定限度,数量一多效率肯定不高。(会消耗掉系统所有资源)
4.计算密集型——利用C语言编写好,Python的效率太低了。
IO密集型(网络、磁盘IO的任务)——利用python开发就好了,因为消耗CPU很少。利用C语言开发无法提高效率。

上一篇 下一篇

猜你喜欢

热点阅读