Celery 任務無法通過 channel_layer.send 調用 Channels 消費者

Question

我一直在為一個相當簡單的應用程序設置一個 websocket。 我編寫了一個 JsonWebsocketConsumer 以及一些 celery 任務，以便執行一些長時間運行的任務（數據庫訪問）。 一切正常，但我的消費者從來沒有接到電話。 有任何想法嗎？

消費者.py

class NetworkCompConsumer(JsonWebsocketConsumer):
    def connect(self):
        self.accept()

    def disconnect(self, close_code):
        pass

    def receive_json(self, content):
        include = content['include']

        # Reduce Dimensions of Inputs to 2, Just in Case User uploads 3D
        # Geojson
        wkb_w = WKBWriter()
        wkb_w.outdim = 2

        include = GEOSGeometry(json.dumps(include))
        include = GEOSGeometry(wkb_w.write_hex(include))

        print("firing tasks")
        genPolySize.delay(include.json, None, self.channel_name)
        genBuildingCount.delay(include.json, None, self.channel_name)
        # This gets sent
        self.send_json({
            "hello world": "Hi!!!"
        })

    def polygon_area(self, event):
        print("poly area consumer called!!")
        # This does not get sent
        self.send_json(
            {"area": event['value']}
        )

    def building_count(self, event):
        print("building count consumer called!!")
        # This does not get sent
        self.send_json(
            {"buildingCount": event['value']}
        )

任務.py

def sync_send(channelName, consumer, value):
    channel_layer = get_channel_layer()
    print(channel_layer)
    async_to_sync(channel_layer.send)(
        channelName,
        {
            "type": consumer,
            "value": value,
        }
    )


@shared_task
def genBuildingCount(include, exclude, channelName):
    query_skeleton = building_count_skeleton
    query_skeleton = getQueryTemplate(query_skeleton, exclude is not None, False)
    with connections['gis_data'].cursor() as cursor:
        cursor.execute(
            query_skeleton, [include, exclude] if exclude is not None else [include])
        results = cursor.fetchone()
        buildingCount = results[0]
        sync_send(channelName, "building.count", buildingCount)
        print("successfully completed genBuildingCount")


@shared_task
def genPolySize(include, exclude, channelName):
    query_skeleton = poly_size_skeleton
    if exclude is not None:
        query_skeleton = poly_size_skeleton_exclude
    with connections['gis_data'].cursor() as cursor:
        query_arguments = [
            include,
            exclude
        ] if exclude is not None else [include]
        cursor.execute(query_skeleton, query_arguments)
        results = cursor.fetchone()
        polygonArea = squaredMetersToMiles(results[0])
        sync_send(channelName, "polygon.area", polygonArea)
        print("successfully completed genPolySize")

日志

django-app_1  | WebSocket HANDSHAKING /ws/network-comparison/ [172.18.0.1:60460]
django-app_1  | WebSocket CONNECT /ws/network-comparison/ [172.18.0.1:60460]
celery_1      | [2020-11-10 18:00:31,916: INFO/MainProcess] Received task: NetworkComparison.tasks.genPolySize[7867638b-76db-4385-b6f6-451970b2f7f3]  
celery_1      | [2020-11-10 18:00:31,921: INFO/MainProcess] Received task: NetworkComparison.tasks.genBuildingCount[875826c0-4c73-4597-9a61-5b6240a4bcfa]  
celery_1      | [2020-11-10 18:00:31,949: WARNING/ForkPoolWorker-3] <channels.layers.InMemoryChannelLayer object at 0x7f0a002c59a0>
celery_1      | [2020-11-10 18:00:31,952: WARNING/ForkPoolWorker-3] successfully completed genPolySize
celery_1      | [2020-11-10 18:00:31,955: INFO/ForkPoolWorker-3] Task NetworkComparison.tasks.genPolySize[7867638b-76db-4385-b6f6-451970b2f7f3] succeeded in 0.03706242493353784s: None
celery_1      | [2020-11-10 18:00:32,050: WARNING/ForkPoolWorker-5] <channels.layers.InMemoryChannelLayer object at 0x7f0a002c9400>
celery_1      | [2020-11-10 18:00:32,053: WARNING/ForkPoolWorker-5] successfully completed genBuildingCount
celery_1      | [2020-11-10 18:00:32,056: INFO/ForkPoolWorker-5] Task NetworkComparison.tasks.genBuildingCount[875826c0-4c73-4597-9a61-5b6240a4bcfa] succeeded in 0.13415803597308695s: None

celery 任務完成，但似乎從未調用過polygon_area和building_count 。 我是否應該擔心 2 個印刷通道層位於不同的內存位置？

謝謝！

Answer 1

我的問題是我使用的是內存通道層而不是 Redis： https ://channels.readthedocs.io/en/stable/topics/channel_layers.html ? highlight = get_channel_layer#in-memory-channel-layer

切換到 Redis 通道層解決了我的問題： https ://channels.readthedocs.io/en/stable/topics/channel_layers.html?highlight=get_channel_layer#redis-channel-layer

Celery Worker 和 Django Worker 位於不同的 docker 容器中，所以我猜內存通道層永遠不會在這種情況下工作。