熊猫-平行化类型函数

Question

I'm dealing with a huge dataset with lots of features. Those features are actually int type,but since they have np.nan values, pandas assigns float64 type to them.

I'm casting those features to float32 by iterating every single column. It takes about 10 minutes to complete. Is there any way to speed up this operation?

The data is read from a csv file. There are object and int64 columns in the data.

for col in float_cols:
    df[col] = df[col].astype(np.float32)

Answer 1

使用dtype与词典参数read_csv ：

df = pd.read_csv(file, dtype=dict.fromkeys(float_cols, np.float32))