多维 Numpy 数组到 Dataframe，错误：引发 ValueError（“数据必须是一维”） ValueError：数据必须是一维

Question

I want to train a neural network and I have the labels (one-hot encoded) and the images both as numpy arrays. I want to add them to a DataFrame to use them as input for the training. I tried to recreate an example, it looks something like this:

import pandas as pd
import numpy as np

label_onehot_example = np.asarray([[0, 0, 0, 0, 1], [0, 0, 0, 1, 0], [1, 0, 0, 0, 0], [1, 0, 0, 0, 0], [1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [1, 0, 0, 0, 0.], [1, 0, 0, 0, 0]])
images_example = np.random.randint(0, 1, (8, 10, 10, 3))

test_df = pd.DataFrame(data={'images': images_example, 'labels' : label_onehot_example})

The error I get is "raise ValueError("Data must be 1-dimensional") ValueError: Data must be 1-dimensional"

I guess it is due to the shape of the image-input (in my example that's (8, 10, 10,3)) but I don't know how to fix it. I thought of looping through the image-array and adding the images and labels one by one to the DataFrame but that seems very inefficient.

Answer 1

The values in your dictionary should be a list in this case, as pandas expects some kind of iterable I think for you column values. You can use the normal 'list()' constructor to change the BWHC numpy array to a list with B elements of shape WHC. (same for the labels)

import pandas as pd
import numpy as np

label_onehot_example = np.asarray([[0, 0, 0, 0, 1], [0, 0, 0, 1, 0], [1, 0, 0, 0, 0], [1, 0, 0, 0, 0], [1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [1, 0, 0, 0, 0.], [1, 0, 0, 0, 0]])
images_example = np.random.randint(0, 1, (8, 10, 10, 3))

test_df = pd.DataFrame(data={'images': list(images_example), 'labels' : list(label_onehot_example)})

print(test_df.head())
>>>                                               images                     labels
0  [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], ...  [0.0, 0.0, 0.0, 0.0, 1.0]
1  [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], ...  [0.0, 0.0, 0.0, 1.0, 0.0]
2  [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], ...  [1.0, 0.0, 0.0, 0.0, 0.0]
3  [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], ...  [1.0, 0.0, 0.0, 0.0, 0.0]
4  [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], ...  [1.0, 0.0, 0.0, 0.0, 0.0]

print(test_df.images[0].shape)
>>> (10, 10, 3)

PS: Which version of pandas did you use? When I first ran you code I got a different error then you reported (mine was "ValueError: If using all scalar values, you must pass an index"). I used pandas 1.1.3