Pika connection lost Error: pika.exceptions.StreamLostError: Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')

Question

Traceback (most recent call last):

File "download_image_from_queue.py", line 44, in

channel.start_consuming()

File "/home/justdial/miniconda3/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 1866, in start_consuming

self._process_data_events(time_limit=None)

File "/home/justdial/miniconda3/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 2027, in _process_data_events

self.connection.process_data_events(time_limit=time_limit)

File "/home/justdial/miniconda3/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 825, in process_data_events

self._flush_output(common_terminator)

File "/home/justdial/miniconda3/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 522, in _flush_output

raise self._closed_result.value.error

pika.exceptions.StreamLostError: Stream connection lost: ConnectionResetError(104, 'Connection reset by peer'

file "download_image_from_queue.py":

def callback(ch, method, properties, body):

    default_flag = True

    try:
        data = json.loads(body.decode("utf-8"))['DATA']

        url = data['url']
        dest_name = data['dest_name']

        default_flag = False
    except:
        print ("\n INCORRECT DATA FORMAT INSERTED INTO THE QUEUE... \n")
        pass

    if default_flag == False:
        os.system("cd /home/images_pred/ && wget -L --timeout=3 --tries=2 {} -O {}".format(url, dest_name))

    print ('\n Waiting for the queue to be filled...       PRESS CTRL+C TO STOP \n')


credentials = pika.PlainCredentials('abcd', 'BCD')
parameters = pika.ConnectionParameters('0.0.0.1', 5672, '/', credentials )


connection = pika.BlockingConnection(parameters)
channel = connection.channel()
#channel.basic_qos(prefetch_count = 1)

channel.basic_consume(queue = 'IMG_DOWNLOAD', auto_ack = False, on_message_callback = callback)
#channel.basic_ack()

print ('\n Waiting for the queue to be filled...       PRESS CTRL+C TO STOP \n')

channel.start_consuming()

once it start consuming, it starts download but sooner it gives this error "pika.exceptions.StreamLostError: Stream connection lost: ConnectionResetError(104, 'Connection reset by peer')"

can anyone help me with this?

Answer 1

Till the time the job gets processed and your process comes back to send an ACK, there are many heartbeats missed, and thus the connection gone.

Ideal Solution (Recommended)

Use threading and run your code in thread. Check a sample here:

https://github.com/pika/pika/blob/1.0.1/examples/basic_consumer_threaded.py

Quick Workaround [Not Recommended]

Disable heartbeats.

How to disable heartbeats? --> Pass heartbeat=0 while creating connection.

pika.ConnectionParameters(
   host=RABBIT_MQ_HOST, credentials=CREDENTIALS, heartbeat=0
)

Answer 2

Try using different connection_attempts parameters while setting up the connection. The default value for connection_attempts is 1. For more information look at the below link. https://pika.readthedocs.io/en/stable/modules/parameters.html