转换数据框中具有值的列作为数字列表的列表

Question

I have a data frame that looks like this:

    date         B  C   D   E
0   04/06/2019  258 994 761 [1, 46, 36, 7457, 456]
1   05/06/2019  748 181 565 [3453, 45]
2   07/06/2019  185 876 107 [4976, 46, 57, 7, 3]
3   08/06/2019  241 386 728 [4, 6457, 4]
4   09/06/2019  516 579 596 [65]

I would like to convert df['E'] as a numeric data type. The reason for that is that my goal is to plot the maximum value of E and the average value over time.

I already tried to convert using:

df['E'].infer_objects()
df['E'].astype(np.int16)

But it did not work...

Answer 1

Try this.

df['E'].apply(lambda x:np.array(x,dtype=np.int32))

For max and mean

df['E_max'].apply(lambda x:x.max())
df['E_mean'].apply(lambda x:x.mean())

Answer 2

infer_objects isn't really for what you think it's for. From the docs :

Attempts soft conversion of object-dtyped columns, leaving non-object and unconvertible columns unchanged. The inference rules are the same as during normal Series/DataFrame construction.

This just checks if an object column can be converted to numeric or some other format recognized by pandas .

---

If you want E to be truly numeric in the eyes of pandas , you need to expand each entry of each list to its own column, so that you are storing actual numeric types and not Python objects.

E = pd.DataFrame(df.E.to_numpy().tolist())

      0       1     2       3      4
0     1    46.0  36.0  7457.0  456.0
1  3453    45.0   NaN     NaN    NaN
2  4976    46.0  57.0     7.0    3.0
3     4  6457.0   4.0     NaN    NaN
4    65     NaN   NaN     NaN    NaN

Now that you have this reference frame, you can use min and max directly on this frame. Using vectorized methods will be much faster than an approach using apply

df.assign(**E.agg(['mean', 'max'], 1))

         date    B    C    D                       E    mean     max
0  04/06/2019  258  994  761  [1, 46, 36, 7457, 456]  1599.2  7457.0
1  05/06/2019  748  181  565              [3453, 45]  1749.0  3453.0
2  07/06/2019  185  876  107    [4976, 46, 57, 7, 3]  1017.8  4976.0
3  08/06/2019  241  386  728            [4, 6457, 4]  2155.0  6457.0
4  09/06/2019  516  579  596                    [65]    65.0    65.0