與Celery組或鏈一起使用時，Redis失去連接-拋出ERROR / MainProcess]與Redis的連接丟失：現在重試（0/20）

Question

我的問題是當我使用芹菜任務啟動我的工人時，redis在終端中顯示以下錯誤。

我很幸運能夠在芹菜文檔中使用經過修改的和弦版本重現該錯誤。 似乎問題出現在和弦或我有太多並行運行的組的情況下，即chord(add.s(i, i) for i in range(1, num))(list_add.s())或group(add.s(i, i) for i in range(1, num))()

下面是我的代碼示例

@task
def add(x, y):
    return [x, y, x + y, "Next"]

@task
def list_add(nums):
    numbers = []
    count = 1
    for i in nums:
        print("{}). {}".format(count, i))
        numbers.extend(i)
        count += 1
    print(numbers)
    return numbers

@task
def foo(num):
    return chord(add.s(i, i) for i in range(1, num))(list_add.s())

以下是帶有追溯功能的終端輸出的一部分。

    [2015-11-04 20:36:14,912: INFO/MainProcess] Received task: b2b.tasks.add[b87fdc44-e759-4224-bce4-11f9468d12b3]
[2015-11-04 20:36:14,913: INFO/MainProcess] Received task: b2b.tasks.add[120f5bf2-b962-4424-894b-d6f0ca56102b]
[2015-11-04 20:36:14,914: INFO/MainProcess] Task b2b.tasks.bar[9dc93c75-6404-4db3-a685-ff91460e1adb] succeeded in 1.00830382s: <AsyncResult: c891df3e-aa5c-4c9f-ad3f-30abe3b3ccc1>

    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (0/20) now.
    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (1/20) in 1.00 second.
    ...
    /Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py:365: RuntimeWarning: Exception raised outside body: ConnectionError('Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.',):
    Traceback (most recent call last):
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py", line 235, in trace_task
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 256, in store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 490, in _store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 160, in set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 149, in ensure
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/kombu/utils/__init__.py", line 243, in retry_over_time
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 169, in _set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2593, in execute
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2447, in _execute_transaction
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 532, in send_packed_command
      File "/Users/Michael/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 436, in connect
    ConnectionError: Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.

這是我用來在終端芹菜中啟動我的工人的命令-刮板工人-P eventlet -c 1000 -l信息

當foo（）傳遞20時，它可以正常工作，但是當1000或更大時，將顯示錯誤。

>>> a = foo.delay(20)     # works 
>>> a = foo.delay(1000)   # fails

請提出建議，如果您有想法並預先感謝，可以如何解決。

Answer 1

我終於在IRC和Redis用戶組的 Greg的幫助下找到了答案

通過將Mac OSX Yosemite上已打開文件的限制更改為65536，解決了該問題。

我希望有一天能對某人有所幫助。