Python logging strange issue

Question

I have a strange issue where logs are not written to log file. I can't post all the code here, but what I'm doing is the following:

I have a logger with file handler and then I execute some script (which I get as input). All the logs that I'm doing after executing the script are disappearing from the log file

logging.info('Before execution')
# execute user script
popen(...)
logging.info('After execution')

The 'After execution' message is not written to the log The strange thing is that when I run tail -f <log> int the terminal, then I can see the 'After execution' message. But when I open the log file to see its content, I don't see this message.

When I close the file handler before logging 'After execution' then the message is written to the log as expected.

What could be the issue? I guess it's something with the user script that I'm executing in popen(...) , but I have no idea what this script is doing. When I run some simple script instead, then everything works as expected. What could be the issue?

Answer 1

Popen does not wait for the script to finish so try to use a wait before the logging call, otherwise it will execute the logging as soon as possible and most likely before the script has time to finish.

logging.info('Before execution')
process = Popen(...)
process.wait()
logging.info('After execution')