How to save real-time predicted results into a csv file?

Question

I am working on an audio classifier project and currently when I run the predictor in realtime, it captures the a small segments of audio then starts the prediction process then spills out the results through the terminal like this:

[INFO] 2020-08-07 14:49:20,695: Predictions: Dog: 1.00
[INFO] 2020-08-07 14:49:20,698: Stop processing.
[INFO] 2020-08-07 14:49:21,707: Start processing.
[INFO] 2020-08-07 14:49:21,847: Predictions: Cat: 0.98
[INFO] 2020-08-07 14:49:21,849: Stop processing.

So, what I want is to have these results saved into a CSV file.

What I did so far:

Here is my script:

def _process_loop(self):

with WavProcessor() as proc:
    self._ask_data.set()
    while True:
        if self._process_buf is None:

            time.sleep(self._processor_sleep_time)                                          #Waiting for data to process
            continue

        logger.info('Start processing.')
        predictions = proc.get_predictions(

            self._sample_rate, self._process_buf)
        logger.info(

            'Predictions: {}'.format(format_predictions(predictions))
        )

        prediction = pd.DataFrame(predictions, columns=['Predicted Class', 'Accuracy']).to_csv('predictions.csv')


        logger.info('Stop processing.')
        self._process_buf = None
        self._ask_data.set()
    

I added this line to my code (prediction = pd.DataFrame(predictions, columns=['Predicted Class', 'Accuracy']).to_csv('predictions.csv')) as you can it in the above code, and started getting something here. The thing that I got was only one result saved inside the csv file. Specifically, it saves only the last detected result.

Predicted Class          Accuracy
Cat                      0.98

So, my current problem is how to save all predicted results inside my CSV file.

I appreciate if anyone can help me to solve this problem.

Answer 1

You are rewriting the file on each iteration. You can specify the write mode of the pandas to_csv function. You can append to an existing file if you set it to a .

Something like this:

pd.DataFrame(predictions, columns=['Predicted Class', 'Accuracy']).to_csv('predictions.csv', mode='a', header=False)

Some nice references:

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html

https://tutorial.eyehunts.com/python/python-file-modes-open-write-append-rrwwx-etc/