multiprocessing.Queue weird python behavior

Question

I stuck with understanding quite simple case. Please, can someone explain or show the direction to understand the following:

import multiprocessing as mp

if __name__ == '__main__':
    input_queue = mp.Queue()
    for i in range(5):
        input_queue.put([i]*5)
    print(input_queue.qsize())
    while not input_queue.empty():
        o = input_queue.get()
        print(o)

Output:

5
[0, 0, 0, 0, 0]
[1, 1, 1, 1, 1]
[2, 2, 2, 2, 2]
[3, 3, 3, 3, 3]
[4, 4, 4, 4, 4]

But:

import multiprocessing as mp

if __name__ == '__main__':
    input_queue = mp.Queue()
    for i in range(5):
        input_queue.put([i]*5)
    # print(input_queue.qsize())
    while not input_queue.empty():
        o = input_queue.get()
        print(o)

Output nothing

Update:

import multiprocessing as mp

if __name__ == '__main__':
    input_queue = mp.Queue()
    for i in range(5):
        input_queue.put([i]*5)
    for _ in range(5):
        o = input_queue.get()
        print(o)

Print expected output. So probably issue in.empty() method.

python --version
Python 3.6.9 :: Anaconda, Inc.

Answer 1

You could be hitting this

After putting an object on an empty queue there may be an infinitesimal delay before the queue's empty() method returns False

Can you try adding a sleep on the line before while not input_queue.empty(): ?

import multiprocessing as mp
import time

if __name__ == '__main__':
    input_queue = mp.Queue()
    for i in range(5):
        input_queue.put([i]*5)
    # print(input_queue.qsize())
    time.sleep(1)
    while not input_queue.empty():
        o = input_queue.get()
        print(o)

If the above works, then your print(input_queue.qsize()) call in the first example is what is giving the queue enough time to pickle the objects and start returning False on the empty() call.