Celery #3 并发

Celery的任务，默认是并发执行（开启Pool）的。但有并发数量限制。
并发数达到上限前，celery会不停接收任务。达到限制后，celery将任务储存到排队序列中，待并发池空出位置，就立刻接收一个新的任务。

比如：
将任务耗时变长一些:

@app.task
def add(x, y):
    time.sleep(1)
    return x,y

循环调用:

for i in range(10):
    add.delay(5,5)

特别说明：这里调用的代码并不会阻塞，而是立刻执行完毕。不能立刻执行的工作会储存在celery中。
可以看到celery的打印信息：

[2018-05-20 14:41:49,684: INFO/MainProcess] Received task: tasks.wrapper[ec5b21b7-d8cb-4a04-aa4b-51f0546a400d]
[2018-05-20 14:41:49,701: INFO/MainProcess] Received task: tasks.wrapper[4662fd8a-ecdd-4536-8f0b-834f84c27898]
[2018-05-20 14:41:49,724: INFO/MainProcess] Received task: tasks.wrapper[a8ca6d67-b833-498b-8ee0-c608e9fbe25c]
[2018-05-20 14:41:49,740: INFO/MainProcess] Received task: tasks.wrapper[581e4f85-0196-46f1-88b9-2b103f40aa02]
[2018-05-20 14:41:49,763: INFO/MainProcess] Received task: tasks.wrapper[165c3a33-2310-48e1-87eb-6f1167673d85]
[2018-05-20 14:41:50,720: INFO/MainProcess] Task tasks.wrapper[4662fd8a-ecdd-4536-8f0b-834f84c27898] succeeded in 1.0149999996647239s: 10
[2018-05-20 14:41:50,728: INFO/MainProcess] Task tasks.wrapper[ec5b21b7-d8cb-4a04-aa4b-51f0546a400d] succeeded in 1.0470000002533197s: 10
[2018-05-20 14:41:50,769: INFO/MainProcess] Received task: tasks.wrapper[a3e1b518-42f0-4de1-8dd4-39d48dc300a8]
[2018-05-20 14:41:50,773: INFO/MainProcess] Task tasks.wrapper[a8ca6d67-b833-498b-8ee0-c608e9fbe25c] succeeded in 1.0469999997876585s: 10
[2018-05-20 14:41:50,780: INFO/MainProcess] Task tasks.wrapper[581e4f85-0196-46f1-88b9-2b103f40aa02] succeeded in 1.0309999999590218s: 10
[2018-05-20 14:41:50,788: INFO/MainProcess] Received task: tasks.wrapper[3d6e1808-37dd-48cb-b669-dace4e551350]
[2018-05-20 14:41:50,827: INFO/MainProcess] Received task: tasks.wrapper[b38c610d-6c26-448b-9cab-61daf52fbb38]
[2018-05-20 14:41:50,849: INFO/MainProcess] Received task: tasks.wrapper[f7c5d7b3-1d4c-45db-98f7-c119f8ab43c6]
[2018-05-20 14:41:51,733: INFO/MainProcess] Task tasks.wrapper[165c3a33-2310-48e1-87eb-6f1167673d85] succeeded in 1.016000000294298s: 10
[2018-05-20 14:41:51,759: INFO/MainProcess] Received task: tasks.wrapper[06354c66-7c78-4e99-9765-a4cb8a69c347]
[2018-05-20 14:41:51,781: INFO/MainProcess] Task tasks.wrapper[a3e1b518-42f0-4de1-8dd4-39d48dc300a8] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:41:51,808: INFO/MainProcess] Task tasks.wrapper[3d6e1808-37dd-48cb-b669-dace4e551350] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:41:51,840: INFO/MainProcess] Task tasks.wrapper[b38c610d-6c26-448b-9cab-61daf52fbb38] succeeded in 1.0149999996647239s: 10
[2018-05-20 14:41:52,744: INFO/MainProcess] Task tasks.wrapper[f7c5d7b3-1d4c-45db-98f7-c119f8ab43c6] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:41:52,791: INFO/MainProcess] Task tasks.wrapper[06354c66-7c78-4e99-9765-a4cb8a69c347] succeeded in 1.0150000001303852s: 10

可见在这次测试中，Celery并发数为5。

重新启动celery，并追加启动参数-c 10，设定并发限制数为10。
这次的打印信息：

[2018-05-20 14:43:34,822: INFO/MainProcess] Received task: tasks.wrapper[aae5ab8d-1b85-420e-8419-925b092ea329]
[2018-05-20 14:43:34,843: INFO/MainProcess] Received task: tasks.wrapper[35f05284-e1b9-4191-98ec-28e37c01d397]
[2018-05-20 14:43:34,893: INFO/MainProcess] Received task: tasks.wrapper[ce7b31ab-7cc5-4089-9e82-b4e1d21851e9]
[2018-05-20 14:43:34,934: INFO/MainProcess] Received task: tasks.wrapper[a57a65fa-080c-4ac6-b1d1-58eebee6633d]
[2018-05-20 14:43:34,980: INFO/MainProcess] Received task: tasks.wrapper[badf2c84-f6b5-40e8-b91f-381764e65961]
[2018-05-20 14:43:34,995: INFO/MainProcess] Received task: tasks.wrapper[b42c1e2a-debc-45d5-8022-a604ff882b6d]
[2018-05-20 14:43:35,033: INFO/MainProcess] Received task: tasks.wrapper[6d1e29fd-85eb-4b24-a23b-ee8b7179372c]
[2018-05-20 14:43:35,047: INFO/MainProcess] Received task: tasks.wrapper[d21a4b94-e489-4a0e-a031-58d4ecfc5e2a]
[2018-05-20 14:43:35,074: INFO/MainProcess] Received task: tasks.wrapper[383c52d7-b5a1-4c33-ba02-42ee7eff1506]
[2018-05-20 14:43:35,088: INFO/MainProcess] Received task: tasks.wrapper[810a89e9-2458-45f1-a296-1d481c52e23f]
[2018-05-20 14:43:35,859: INFO/MainProcess] Task tasks.wrapper[aae5ab8d-1b85-420e-8419-925b092ea329] succeeded in 1.0309999999590218s: 10
[2018-05-20 14:43:35,878: INFO/MainProcess] Task tasks.wrapper[35f05284-e1b9-4191-98ec-28e37c01d397] succeeded in 1.0320000001229346s: 10
[2018-05-20 14:43:35,908: INFO/MainProcess] Task tasks.wrapper[ce7b31ab-7cc5-4089-9e82-b4e1d21851e9] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:43:35,948: INFO/MainProcess] Task tasks.wrapper[a57a65fa-080c-4ac6-b1d1-58eebee6633d] succeeded in 1.016000000294298s: 10
[2018-05-20 14:43:35,994: INFO/MainProcess] Task tasks.wrapper[badf2c84-f6b5-40e8-b91f-381764e65961] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:43:36,020: INFO/MainProcess] Task tasks.wrapper[b42c1e2a-debc-45d5-8022-a604ff882b6d] succeeded in 1.0s: 10
[2018-05-20 14:43:36,046: INFO/MainProcess] Task tasks.wrapper[6d1e29fd-85eb-4b24-a23b-ee8b7179372c] succeeded in 1.0150000001303852s: 10
[2018-05-20 14:43:36,060: INFO/MainProcess] Task tasks.wrapper[d21a4b94-e489-4a0e-a031-58d4ecfc5e2a] succeeded in 1.0159999998286366s: 10
[2018-05-20 14:43:36,087: INFO/MainProcess] Task tasks.wrapper[383c52d7-b5a1-4c33-ba02-42ee7eff1506] succeeded in 1.0149999996647239s: 10
[2018-05-20 14:43:36,101: INFO/MainProcess] Task tasks.wrapper[810a89e9-2458-45f1-a296-1d481c52e23f] succeeded in 1.016000000294298s: 10