转换具有numpy数组的列，将其转换为以dtype为对象的numpy数组

Question

I have a column in a dataframe having numpy arrays of length 10. My dataframe is like this:

0       [2.0, 1246.0, 82.0, 43.0, 569.0, 46.0, 424.0, ...
1       [395.0, 2052.0, 1388.0, 8326.0, 5257.0, 176.0,...
10      [4.0, 1.0, 13.0, 1409.0, 7742.0, 259.0, 1856.0...
100     [4.0, 87.0, 1595.0, 706.0, 2935.0, 6028.0, 442...
1000    [45.0, 582.0, 124.0, 6530.0, 6548.0, 748.0, 61...
Name: embedding1, dtype: object

When I am converting it into a numpy array of array using this:

input = np.asarray(df.tolist())

It is giving the array like this:

array([array([   2., 1246.,   82.,   43.,  569.,   46.,  424.,  446., 1054., 39.]),

       array([4.0000e+00, 1.0000e+00, 1.3000e+01, 1.4090e+03, 7.7420e+03,
       2.5900e+02, 1.8560e+03, 3.6181e+04, 4.2000e+01, 8.9000e+02]),
       ...,
       array([4.000e+00, 1.000e+00, 1.300e+01, 2.900e+01, 4.930e+02, 2.760e+02,1.100e+01, 6.770e+02, 6.740e+02, 5.806e+03]),], dtype=object)

The type it is giving is object. I want the object as float because it is giving the shape(1000,) but I want shape as (1000,10). I have tried using this:

input1 = np.asarray(df1.tolist(),dtype=np.float)

But it is giving the following error:

ValueError: setting an array element with a sequence.

How to solve this?

PS: All the elements of the dataframe's row numpy array are of float type

Answer 1

First of all, it seems like you have a pd.Series and not a data frame.

Take the setup :

x = [[2.0, 1246.0, 82.0, 43.0, 569.0, 46.0, 424.0],
[395.0, 2052.0, 1388.0, 8326.0, 5257.0, 176.0],
[4.0, 1.0, 13.0, 1409.0, 7742.0, 259.0, 1856.0],
[4.0, 87.0, 1595.0, 706.0, 2935.0, 6028.0, 442],
[45.0, 582.0, 124.0, 6530.0, 6548.0, 748.0, 61]]

s = pd.Series(x)

Which yields

0      [2.0, 1246.0, 82.0, 43.0, 569.0, 46.0, 424.0]
1     [395.0, 2052.0, 1388.0, 8326.0, 5257.0, 176.0]
2    [4.0, 1.0, 13.0, 1409.0, 7742.0, 259.0, 1856.0]
3    [4.0, 87.0, 1595.0, 706.0, 2935.0, 6028.0, 442]
4    [45.0, 582.0, 124.0, 6530.0, 6548.0, 748.0, 61]
dtype: object

You have a pd.Series of arrays. And it seems like you want to flatten it. Using the default constructor in a list of lists yields a data frame where each list is interpreted as a row:

df2 = pd.DataFrame(s.tolist())

    0       1       2       3       4       5       6
0   2.0     1246.0  82.0    43.0    569.0   46.0    424.0
1   395.0   2052.0  1388.0  8326.0  5257.0  176.0   NaN
2   4.0     1.0     13.0    1409.0  7742.0  259.0   1856.0
3   4.0     87.0    1595.0  706.0   2935.0  6028.0  442.0
4   45.0    582.0   124.0   6530.0  6548.0  748.0   61.0

Now you can just get the underlying np.array accessing the data frame .values

df2.values

array([[2.000e+00, 1.246e+03, 8.200e+01, 4.300e+01, 5.690e+02, 4.600e+01,
        4.240e+02],
       [3.950e+02, 2.052e+03, 1.388e+03, 8.326e+03, 5.257e+03, 1.760e+02,
              nan],
       [4.000e+00, 1.000e+00, 1.300e+01, 1.409e+03, 7.742e+03, 2.590e+02,
        1.856e+03],
       [4.000e+00, 8.700e+01, 1.595e+03, 7.060e+02, 2.935e+03, 6.028e+03,
        4.420e+02],
       [4.500e+01, 5.820e+02, 1.240e+02, 6.530e+03, 6.548e+03, 7.480e+02,
        6.100e+01]])