熊猫：关于如何处理缺少的小数的建议

Question

I have a scenario where the one of the record in the dataset contains empty value (simplified below for ease of understanding). there are two records in data, one with 0.1 and other with None. When I serialize df1 , I get the response I want ie null for second record.

import pandas as pd
import numpy as np
import decimal


data = [{'A': 0.1}, {'A': None}]
df1 = pd.DataFrame(data)
print(df1.to_json(orient='records'))

prints [{"A":0.1},{"A":null}]

I want to treat A as decimal like below:

df3 = df1.copy()
df3['A'] = df2['A'].apply(lambda x: decimal.Decimal(x))
print(df3.to_json(orient='records')) # this throws exception

prints OverflowError: Invalid Nan value when encoding double

I would like to get the same result as df1 ie null for the missing decimal in json. Note this works if I used float instead of decimal but that's not an option I can use.

Answer 1

df3['A'] = df2['A'].apply(lambda x: (decimal.Decimal(x) if not pd.isnull(x) else None))