Writing Multiple Rows to CSV using Numpy Arrays without Brackets?

Question

I am prepping some data to be thrown into R. The data is coming out of Python being exported to CSV files. R is reading the data as only having two columns because the numpy array is being exported into the CSV with brackets surrounding it like so:

loudness_momentary,"[-83.84079859 -83.67103055 -83.80533271]"

I think the problem is being generated because when I write the data to the CSV, there is a "list within a list" happening like so:

data = [ 
    ['loudness_momentary', loudness_momentary],
    ['loudness_short', loudness_short],
    ['loudness_integrated', loudness_integrated],
    ['lra', lra],
    ['crest_factor_1s', crest_factor_1s],
    ['crest_factor_100ms', crest_factor_100ms],
    ['spectral_centroid', spectral_centroid],
    ['spectral_spread', spectral_spread],
    ['spectral_skewness', spectral_skewness],
    ['spectral_kurtosis', spectral_kurtosis],
    ['zcr', zcr],
    ['spectral_rolloff', spectral_rolloff],
    ['spectral_crest_factor', spectral_crest_factor],
    ['spectral_flatness', spectral_flatness],
    ['spectral_flux', spectral_flux]
    ]

    # Write to CSV File
    with open(filepath + "/" + file_name + ".csv", "wb") as stem_data:
        w = csv.writer(stem_data, delimiter=',')
        print "Writing %s to csv" % file_name
        w.writerows(data)
    stem_data.close()

My goal here was to give each row a "header" to let me know what the data is. If I remove the "list within a list" problem and only write one row, the data writes normally.

So basically, I'm trying to output a CSV file with a "string" header for each row, where each row after that is a numpy array without brackets or quotations surrounding it so I can put it into R nicely.

Any help would be appreciated, Thanks.

Answer 1

Convert data to a list of lists:

data = [[row[0]]+row[1].tolist() for row in data]

For example, if data starts as something like

In [34]: data = [['foo', np.array([1,2,3])], ['bar', np.array([1,2,3])]]

Then

In [35]: data = [[row[0]]+row[1].tolist() for row in data]

converts data to this list of lists:

In [36]: data
Out[36]: [['foo', 1, 2, 3], ['bar', 1, 2, 3]]

---

By the way, it is good that you are using the with-statement to open the file. Note however that Python closes the file automatically when the flow of execution leaves the with-block . So calling stem_data.close() explicitly after leaving the with-block is unnecessary.