[英]Pandas transformation really slow
I been trying to transform python pandas to from one representation to another but is really slow:我一直在尝试将 python pandas 从一种表示转换为另一种表示,但速度真的很慢:
My current dataframe is like:我目前的 dataframe 是这样的:
Column0, Column 1, Column 2, Column 3, Column 10...
p1 1 2 8 3
p2 2 4 9 6
Column0, sample, Exp
p1 Column 1 1
p1 Column 2 2
p1 Column 3 8
p1 Column 10 3
p2 Column 1 2
. . .
. . .
I use the iterrows
inserting into a new dataframe but is really slow.我使用iterrows
插入一个新的 dataframe 但真的很慢。 The Column 1,2 is not fixed but I have a collection with all the names.第 1,2 列不是固定的,但我有一个包含所有名称的集合。
IIUC, you can use melt
, IIUC,你可以使用melt
,
generally looping is discouraged in pandas unless there is no other option or you have a justifiable usecase.通常在 pandas 中不鼓励循环,除非没有其他选择或者您有合理的用例。
d = """Column0, Column 1, Column 2, Column 3, Column 10
p1, 1, 2, 8, 3
p2, 2, 4, 9, 6"""
from io import StringIO
df = pd.read_csv(StringIO(d),sep=',')
df2 = pd.melt(df,id_vars=['Column0'],var_name='Sample',value_name='Exp')
print(df2)
Column0 Sample Exp
0 p1 Column 1 1
1 p2 Column 1 2
2 p1 Column 2 2
3 p2 Column 2 4
4 p1 Column 3 8
5 p2 Column 3 9
6 p1 Column 10 3
7 p2 Column 10 6
pd.melt(df,id_vars=['Column0'],var_name='Sample',value_name='Exp').rename(
columns = {'Column0' : 'NewCol',...})
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.