[英]pandas Dataframe: create new columns of labels based on other columns
I have this example pandas.DataFrame
with +20K rows, in the following form: 我有这个带有+ 20K行的
pandas.DataFrame
示例,格式如下:
import pandas as pd
import numpy as np
data = {"first_column": ["A", "B", "B", "B", "C", "A", "A", "A", "D", "B", "A", "A"],
"second_column": [0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0]}
df = pd.DataFrame(data)
>>> df
first_column second_column
0 A 0
1 B 1
2 B 1
3 B 1
4 C 0
5 A 0
6 A 0
7 A 1
8 D 1
9 B 1
10 A 1
11 A 0
....
The column first_column
has for each row A
, B
, C
, and D
. 列
first_column
包含每一行A
, B
, C
和D
In the second column, there is a binary label denoting a group of values. 在第二列中,有一个表示一组值的二进制标签。 All consecutive groupings of 1's are a unique "group", eg rows 1-3 is one group, rows 7-10 is another group.
1的所有连续分组是唯一的“组”,例如,行1-3是一组,行7-10是另一组。
I would like to "label" each one of these groups by being either "AB" (the group is only composed of A or B), "CD" (the group is only composed of C or D), or "mixed" (if there is a mixture, eg all B and one C). 我想通过“AB”(该组仅由A或B组成),“CD”(该组仅由C或D组成)或“混合”(“组合”来标记这些组中的每一组。如果有混合物,例如所有B和一个C)。 It would also be useful to know "how" mixed some of these groupings are with a percentage, ie the percentage of AB's out of total labels.
知道“如何”混合这些分组中的一些百分比,即AB在总标签中的百分比也是有用的。 So, if it is only
A
or B
, the identity should be AB
. 因此,如果它只是
A
或B
,则标识应为AB
。 If it is only C
or D
, the identity should be CD
. 如果它只是
C
或D
,则标识应为CD
。 It is a mixture of A,B,C, and/or D, then it is mixed
. 它是A,B,C和/或D的混合物,然后
mixed
。 The percentage is (# of AB rows)/(# of total rows) 百分比是(AB行数#)/(总行数#)
Here is how the resulting DataFrame
would look: 以下是生成的
DataFrame
外观:
>>> df
first_column second_column identity percent
0 A 0 0 0
1 B 1 AB 1.0
2 B 1 AB 1.0
3 B 1 AB 1.0
4 C 0 0 0
5 A 0 0 0
6 A 0 0 0
7 A 1 mixed 0.75 # 3/4, 3-AB, 4-total
8 D 1 mixed 0.75
9 B 1 mixed 0.75
10 A 1 mixed 0.75
11 A 0 0 0
....
My initial thought would be to first try to use df.loc()
with 我最初的想法是首先尝试使用
df.loc()
if (df.first_column == "A" | df.first_column == "B"):
df.loc[df.second_column == 1, "identity"] = "AB"
if (df.first_column == "C" | df.first_column == "D"):
df.loc[df.second_column == 1, "identity"] = "CD"
but this doesn't take into account mixtures, nor does it work with isolated groupings. 但这并未考虑混合物,也不适用于孤立的分组。
Here is one way to do that. 这是一种方法。
Code: 码:
import pandas as pd
from collections import Counter
a_b = set('AB')
c_d = set('CD')
def get_id_percent(group):
present = Counter(group['first_column'])
present_set = set(present.keys())
if group['second_column'].iloc[0] == 0:
ret_val = 0, 0
elif present_set.issubset(a_b) and len(present_set) == 1:
ret_val = 'AB', 0
elif present_set.issubset(c_d) and len(present_set) == 1:
ret_val = 'CD', 0
else:
ret_val = 'mixed', \
float(present['A'] + present['B']) / len(group)
return pd.DataFrame(
[ret_val] * len(group), columns=['identity', 'percent'])
Test Code: 测试代码:
data = {"first_column": ["A", "B", "B", "B", "C", "A", "A",
"A", "D", "B", "A", "A"],
"second_column": [0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0]}
df = pd.DataFrame(data)
groupby = df.groupby((df.second_column != df.second_column.shift()).cumsum())
results = groupby.apply(get_id_percent).reset_index()
results = results.drop(['second_column', 'level_1'], axis=1)
df = pd.concat([df, results], axis=1)
print(df)
Results: 结果:
first_column second_column identity percent
0 A 0 0 0.00
1 B 1 AB 0.00
2 B 1 AB 0.00
3 B 1 AB 0.00
4 C 0 0 0.00
5 A 0 0 0.00
6 A 0 0 0.00
7 A 1 mixed 0.75
8 D 1 mixed 0.75
9 B 1 mixed 0.75
10 A 1 mixed 0.75
11 A 0 0 0.00
Here's one approach: 这是一种方法:
import pandas as pd
# generate example data
data = {"first_column": ["A", "B", "B", "B", "C", "A", "A", "A", "D", "B", "A", "A"],
"second_column": [0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0]}
df = pd.DataFrame(data)
# these are intermediary groups for computation
df['group_type'] = None
df['ct'] = 0
def find_border(x, ct):
''' finds and labels lettered groups '''
ix = x.name
# does second_column == 1?
if x.second_column:
# if it's the start of a group...
if (not ix) | (not df.group_type[ix-1]):
df.ix[ix,'group_type'] = x.first_column
df.ix[ix,'ct'] += 1
return
# if it's the end of a group
elif (not df.second_column[ix+1]):
df.ix[ix,'group_type'] = df.group_type[ix-1] + x.first_column
df.ix[ix,'ct'] = df.ct[ix-1] + 1
for i in range(df.ct[ix-1]+1):
df.ix[ix-i,'group_type'] = df.ix[ix,'group_type']
df.ix[ix,'ct'] = 0
return
# if it's the middle of a group
else:
df.ix[ix,'ct'] = df.ct[ix-1] + 1
df.ix[ix,'group_type'] = df.group_type[ix-1] + x.first_column
return
return
# compute group membership
_=df.apply(find_border, axis='columns', args=(0,))
def determine_id(x):
if not x:
return '0'
if list(set(x)) in [['A'],['B'],['A','B']]:
return 'AB'
elif list(set(x)) in [['C'],['D'],['C','D']]:
return 'CD'
else:
return 'mixed'
def determine_pct(x):
if not x:
return 0
return sum([1 for letter in x if letter in ['A','B']]) / float(len(x))
# determine row identity
df['identity'] = df.group_type.apply(determine_id)
# determine % of A or B in group
df['percent'] = df.group_type.apply(determine_pct)
Output: 输出:
first_column second_column identity percent
0 A 0 0 0.00
1 B 1 AB 1.00
2 B 1 AB 1.00
3 B 1 AB 1.00
4 C 0 0 0.00
5 A 0 0 0.00
6 A 0 0 0.00
7 A 1 mixed 0.75
8 D 1 mixed 0.75
9 B 1 mixed 0.75
10 A 1 mixed 0.75
11 A 0 0 0.00
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.