Convert list of NDarrays to dataframe

Question

I'm trying to convert a list ND arrays to a dataframe in order to do a Isomap on it. But this doesn't convert. Anyone how to convert in such that I can do an Isomap on it?

#Creation and filling of list samples*  
samples = list()  
for i in range(72):  
 img =misc.imread('Datasets/ALOI/32/32_r'+str(i*5)+'.png' )  
 samples.append(img)  
...  
df = pd.DataFrame(samples) #This doesn't work gives  
                           #ValueError: Must pass 2-d input*  
...  
iso = manifold.Isomap(n_neighbors=4, n_components=3)  
iso.fit(df) #The end goal of my DataFrame

Answer 1

That is obvious, isn't it? All images are 2D data, rows and columns. Stacking them in a list causes it to gain a third dimension. DataFrames are by nature 2D. Hence the error.

You have 2 possible fixes:

Create a Panel .

wp = pd.Panel.from_dict(zip(samples, [str(i*5) for i in range(72)]))

Stack your arrays one on top of the other, or side by side:

# On top of another:
df = pd.concat([pd.DataFrame(sample) for sample in samples], axis=0,
               keys=[str(i*5) for i in range(72)])

# Side by side:
df = pd.concat([pd.DataFrame(sample) for sample in samples], axis=1,
               keys=[str(i*5) for i in range(72)])

Answer 2

Another way to do it is to convert your 2D arrays (images) to 1D arrays (that are expected by sklearn) using the reshape method on the images:

for i in range(yourRange):  
  img = misc.imread(yourFile)  
  samples.append(img.reshape(-1))
df = pd.DataFrame(samples)

Answer 3

Olivera almost had it.

the problem

When you run misc.imread, the output is a NxM (2D) array. Putting this in a list, makes it 3D. DataFrame expects a 2D input.

the fix

Before it goes in the list, the array should be 'flattened' using ravel:

img =misc.imread('Datasets/ALOI/32/32_r'+str(i*5)+'.png' ).ravel()

why .reshape(-1) doesn't work

Reshaping the array preserves the array's rank. Instead of converting it to an Nx1 array, you want it to be Nx(nothing), which is what ravel() does.