[英]In Pandas, how to create a unique ID based on the combination of many columns?
I have a very large dataset, that looks like我有一个非常大的数据集,看起来像
df = pd.DataFrame({'B': ['john smith', 'john doe', 'adam smith', 'john doe', np.nan], 'C': ['indiana jones', 'duck mc duck', 'batman','duck mc duck',np.nan]})
df
Out[173]:
B C
0 john smith indiana jones
1 john doe duck mc duck
2 adam smith batman
3 john doe duck mc duck
4 NaN NaN
I need to create a ID variable, that is unique for every BC combination.我需要创建一个 ID 变量,它对于每个 BC 组合都是唯一的。 That is, the output should be也就是说,output应该是
B C ID
0 john smith indiana jones 1
1 john doe duck mc duck 2
2 adam smith batman 3
3 john doe duck mc duck 2
4 NaN NaN 0
I actually dont care about whether the index starts at zero or not, and whether the value for the missing columns is 0 or any other number.我实际上不关心索引是否从零开始,以及缺失列的值是 0 还是任何其他数字。 I just want something fast, that does not take a lot of memory and can be sorted quickly.我只是想要一些快速的东西,不需要很多 memory 并且可以快速排序。 I use:我用:
df['combined_id']=(df.B+df.C).rank(method='dense')
but the output is float64
and takes a lot of memory. Can we do better?但是 output 是float64
并且需要很多 memory。我们可以做得更好吗? Thanks!谢谢!
Making jezrael's answer a little more general (what if the columns were not string?), you can use this compact function:使 jezrael 的答案更笼统一些(如果列不是字符串怎么办?),您可以使用此紧凑函数:
def make_identifier(df):
str_id = df.apply(lambda x: '_'.join(map(str, x)), axis=1)
return pd.factorize(str_id)[0]
df['combined_id'] = make_identifier(df[['B','C']])
I have a very large dataset, that looks like我有一个非常大的数据集,看起来像
df = pd.DataFrame({'B': ['john smith', 'john doe', 'adam smith', 'john doe', np.nan], 'C': ['indiana jones', 'duck mc duck', 'batman','duck mc duck',np.nan]})
df
Out[173]:
B C
0 john smith indiana jones
1 john doe duck mc duck
2 adam smith batman
3 john doe duck mc duck
4 NaN NaN
I need to create a ID variable, that is unique for every BC combination.我需要创建一个ID变量,该变量对于每个BC组合都是唯一的。 That is, the output should be也就是说,输出应为
B C ID
0 john smith indiana jones 1
1 john doe duck mc duck 2
2 adam smith batman 3
3 john doe duck mc duck 2
4 NaN NaN 0
I actually dont care about whether the index starts at zero or not, and whether the value for the missing columns is 0 or any other number.我实际上并不在乎索引是否从零开始,以及缺失列的值是0还是任何其他数字。 I just want something fast, that does not take a lot of memory and can be sorted quickly.我只想要快速的东西,不需要太多的内存并且可以快速排序。 I use:我用:
df['combined_id']=(df.B+df.C).rank(method='dense')
but the output is float64
and takes a lot of memory.但输出为float64
并占用大量内存。 Can we do better?我们可以做得更好吗? Thanks!谢谢!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.