在python中保存/读取彩色图像像素数据

Question

I'm trying to write an algorithm that will save the filename and the 3 channel np.array stored in each filename to a csv (or similar filetype), and then be able to read in the csv and reproduce the color image image.

The format of my csv should look like this:

  Filename RGB
0 foo.png  np.array      # the shape is 100*100*3
1 bar.png  np.array
2 ...      ...

As it stands, I'm iterating through each file saved in a directory and appending a list that later gets stored in a pandas.DataFrame:

df1= pandas.DataFrame()
df2= pandas.DataFrame()
directory= r'C:/my Directory'
fileList= os.listdir(directory)
filenameList= []
RGBList= []
for eachFile in fileList:
    filenameList.append(eachFile)
    RGBList.append(cv2.imread(directory + eachFile, 1).tostring())
df1["Filenames"]= filenameList
df2["RGB"]= RGBList
df1.to_csv('df1.csv')
df2.to_csv('df2.csv')

df1 functions as desired. I THINK df2 fuctions as intended. A print statement reveals the correct len of 30,000 for each row of the csv. However, when I read in the csv using pandas.read_csv('df2') and use a print statement to view the len of the first row, I get 110541. I intend to use np.fromstring() and np.reshape() to reshape the flattened np.array generated from np.tostring() , but I get the error:

ValueError: string size must be a multiple of element size

...because the number of elements is mismatched.

My question is:

1. Why is the len so much larger when I read in the csv?
2. Is there a more efficient way to write 3 channel color image pixel data to a csv that can easily be read back in?

Answer 1

If you write a single byte for each 8-bit pixel you will get a line with 1 byte per pixel. So, if your image is 80 pixels wide, you will get 80 bytes per line.

If you write a CSV, in human-readable ASCII, you will need more space. Imagine the first pixel is 186. So, you will write a 1 , an 8 , a 6 and a comma - ie 4 bytes now for the first pixel instead of a single byte in binary, and so on.

That means your file will be around 3-4x bigger, ie 110k instead of 30k, which is what you are seeing.

---

There is no "better way" to write a CSV - the problem is that is a fundamentally inefficient format designed for humans rather than computers. Why did you choose CSV? If it has to be legible for humans, you have no choice.

If it can be illegible to humans, but readily legible to computers, choose a different format such np.save() and np.load() - as you wisely have done already ;-)