继上篇 RabbitMQ实战2.消息轮询、响应、持久化

本篇沿用上篇的代码，请先阅读上篇文章

轮询不能保证执行时间的效率最大化

用轮询分配消息的方式只能在消息条数上保证公平，并没有兼顾每个消息的执行时间。这样就可能导致这样的情况：

消费者A正在执行一个非常耗时的任务，预估耗时1小时；消费者B执行的任务只要0.1s就完成，由于是按照任务数分配，就会造成消费者A有一堆任务在后面排队，而消费都B却闲得心里有点慌...

生产者投递一个长时间的任务以及五个短时间的任务

☁  rabbitMq [master] ⚡ python new_task.py long time.......................
 [x] Sent 'long time.......................'
☁  rabbitMq [master] ⚡ python new_task.py one .
 [x] Sent 'one .'
☁  rabbitMq [master] ⚡ python new_task.py two .
 [x] Sent 'two .'
☁  rabbitMq [master] ⚡ python new_task.py three .
 [x] Sent 'three .'
☁  rabbitMq [master] ⚡ python new_task.py four .
 [x] Sent 'four .'
☁  rabbitMq [master] ⚡ python new_task.py five .
 [x] Sent 'five .'

消费者1对任务的接受情况：

☁  rabbitMq [master] ⚡ python worker.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'long time.......................'
 [x] Done
 [x] Received b'two .'
 [x] Done
 [x] Received b'four .'
 [x] Done

在第一个长时间任务未完成之前，后面的两个短时间任务只能一直在等待...

排队等待

消费者2对任务的接受情况:

☁  rabbitMq [master] ⚡ python worker.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'one .'
 [x] Done
 [x] Received b'three .'
 [x] Done
 [x] Received b'five .'
 [x] Done

消费者2很快就完成了分配的任务，然后就无所事事地看着消费者1忙着狗...

设置消费者的预读取数

这样的分配明显不公平，数目上的绝对公平忽视了处理效率的区别。更好的方案应该是在分配任务时，根据哪个消费者空闲，就优先分配给此消费者

具体的配置:

worker.py

channel.basic_qos(prefetch_count=1)  # 同一时刻，不要发送超过一条消息给一个消费者
channel.basic_consume(callback,
                      queue='task_queue')

执行情况：

消费者1: 专注于处理耗时的任务

☁  rabbitMq [master] ⚡ python worker.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'long time.......................'
 [x] Done

消费者2: 将零散的小任务都处理了

☁  rabbitMq [master] ⚡ python worker.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'one .'
 [x] Done
 [x] Received b'two .'
 [x] Done
 [x] Received b'three .'
 [x] Done
 [x] Received b'four .'
 [x] Done
 [x] Received b'five .'
 [x] Done

如此调整，才能实现在总的执行时间最优化

参数文档

• tutorial-two-python