Pandas数据框：将混合类型的字符串值转换为浮点数，同时跟踪真实的字符串值

Question

I've scrub a table from a web page. Some columns hold dollar amounts or percentage (as strings), but some entries have some notes, ie '(n)'. Before I can transform the str(numbers) to float, I need to inspect the notes to find out if they should get 0 or NA.

What I need is to output the row and note for each entry with a note, ie (4, '(4)'), (13, '(4)'); # or vectors

Using python: 3.5.4; pandas: 0.22.0

I've reproduced a smaller dataframe:

df = pd.DataFrame({'A':[ '$104.64', '$73.04', '(4)', '$82.95', '$92.45', '$95.09', 
                    '$79.20', '$63.66', '$90.27', '$98.80', '$33.82', '(8)', '$56.74', '$49.22', 
                    '$75.74'], 
               'B':['%28.90', '%73.36', '(3)', '%104.64', '%73.04', '%82.95',  
                    '%79.20', '(9)', '%63.66', '%90.27', '%98.80', '%33.82', '%56.74', '%49.22', 
                    '%75.74']})
df

        A   B
0   $104.64 %28.90
1   $73.04  %73.36
2   (4) (3)
3   $82.95  %104.64
4   $92.45  %73.04
5   $95.09  %82.95
6   $79.20  %79.20
7   $63.66  (9)
8   $90.27  %63.66
9   $98.80  %90.27
10  $33.82  %98.80
11  (8) %33.82
12  $56.74  %56.74
13  $49.22  %49.22
14  $75.74  %75.74

out = df['A'].where( df['A']>='(' )   # 1. how to get rid of the NaN?
out

out = out.astype(dtype=str)           # 2. found that NaN is of type float, 
                                           so now all entries are str
out

to get:

2  '(4)'
11 '(8)'

I tried this, but that's not helping because the note value is changed to True:

df['A'].where( df['A']>='(' ).isna() == False

0     False
1     False
2      True
3     False
4     False
5     False
6     False
7     False
8     False
9     False
10    False
11     True
12    False
13    False
14    False

The closest answer I [found][ Skipping char-string values while comparing to a column values of mixed type to int or float in pandas Dataframe does not help:

pd.to_numeric(df.A.str.strip('$'), errors='coerce')

performs the conversion, but transforms the '(n)' note value to nan.

To sum up: The problem is the mixed types in the columns, due to the notes: I cannot just strip the '$' or '%', then convert to float. I also need to document where these were.

I am probably blind to a simple solution...

Answer 1

Since I don't have enough reputation to comment... Can you use df.loc[] to extract ones with '('?

withnotes = df.loc[df['A'].str.contains('\(')]
output = [(i, row.A) for i, row in withnotes.iterrows()]
output

Above example parses only column A and returns a list of tuples: output = [(2, '(4)'), (11, '(8)')]

Answer 2

You can use .loc accessor for this:

res = df.loc[df['A'].str[0] == '(', 'A']

This results in a series:

2     (4)
11    (8)
Name: A, dtype: object

If you need a dataframe:

res = df.loc[df['A'].str[0] == '(', 'A'].to_frame()