[英]I wish to replicate conditional column from Power Query M function in Python using Pandas
I've been working with Power Query for a bit to process some data and I was impressed at how flexible the framework is.我一直在使用 Power Query 处理一些数据,框架的灵活性给我留下了深刻的印象。 Currently, I am looking to replicate a conditional column step in Pandas as I would like to include it in an automated data cleaning script pipeline.目前,我希望在 Pandas 中复制条件列步骤,因为我想将其包含在自动数据清理脚本管道中。
In this case, Power Query creates a new column called acc_col, looks at each column in the dataset (Tags.1, Tags.2, etc) and if the string in that column matches the beginning of the value (Acceleration-) then it outputs that value into the new column, else if no match is found it outputs Unknown Acc.在这种情况下,Power Query 创建一个名为 acc_col 的新列,查看数据集中的每一列(Tags.1、Tags.2 等),如果该列中的字符串与值的开头(Acceleration-)匹配,那么它将该值输出到新列中,否则如果找不到匹配项,则输出 Unknown Acc。 This is how it looks from the Editor这是编辑器的外观
#"Added Conditional Column" = Table.AddColumn(#"Replaced Value", "acc_col", each if Text.StartsWith([Tags.1], "Acceleration-") then [Tags.1] else if Text.StartsWith([Tags.2], "Acceleration-") then [Tags.2] else if Text.StartsWith([Tags.3], "Acceleration-") then [Tags.3] else if Text.StartsWith([Tags.4], "Acceleration-") then [Tags.4] else if Text.StartsWith([Tags.5], "Acceleration-") then [Tags.5] else "Unknown Acc")
I have tried some things with Pandas but my knowledge is a little bit limited.我用 Pandas 尝试了一些东西,但我的知识有点有限。 I managed to read one of the Tag columns using the following我设法使用以下方法阅读了标签列之一
Tags0标签0 | Tags1标签1 | Tags2标签2 |
---|---|---|
Alumni-2017,Acceleration-2016 2017年校友,2016年加速 | None没有任何 | None没有任何 |
Alumni校友 | Acceleration-2017加速-2017 | None没有任何 |
Acceleration-2015加速-2015 | None没有任何 | None没有任何 |
Alumni-2017 2017年校友 | Acceleration-2015加速-2015 | None没有任何 |
Alumni-2017 2017年校友 | Acceleration-2014加速-2014 | None没有任何 |
df['acc_col'] = df['Tags0'].where(df['Tags0'].str.contains('Acceleration', na=False), )
Tags0标签0 | Tags1标签1 | Tags2标签2 | acc_col acc_col |
---|---|---|---|
Alumni-2017,Acceleration-2016 2017年校友,2016年加速 | None没有任何 | None没有任何 | Acceleration-2016加速-2016 |
Alumni校友 | Acceleration-2017加速-2017 | None没有任何 | None没有任何 |
Acceleration-2015加速-2015 | None没有任何 | None没有任何 | Acceleration-2015加速-2015 |
Alumni-2017 2017年校友 | Acceleration-2015加速-2015 | None没有任何 | None没有任何 |
Alumni-2017 2017年校友 | Acceleration-2014加速-2014 | None没有任何 | None没有任何 |
I see that the output took all those that contained the keyword but if I wish to do the same with the other columns it overwrites the previous results.我看到 output 获取了所有包含关键字的内容,但如果我希望对其他列执行相同操作,它会覆盖以前的结果。 I need them all to be on the same column as it reads through one by one.我需要它们都在同一列上,因为它一一阅读。
df['acc_col'] = df['Tags1'].where(df['Tags1'].str.contains('Acceleration', na=False), )
Tags0标签0 | Tags1标签1 | Tags2标签2 | acc_col acc_col |
---|---|---|---|
Alumni-2017,Acceleration-2016 2017年校友,2016年加速 | None没有任何 | None没有任何 | None没有任何 |
Alumni校友 | Acceleration-2017加速-2017 | None没有任何 | Acceleration-2017加速-2017 |
Acceleration-2015加速-2015 | None没有任何 | None没有任何 | None没有任何 |
Alumni-2017 2017年校友 | Acceleration-2015加速-2015 | None没有任何 | Acceleration-2015加速-2015 |
Alumni-2017 2017年校友 | Acceleration-2014加速-2014 | None没有任何 | Acceleration-2014加速-2014 |
I feel like I'm close but I just need a bit more assistance.我觉得我已经很接近了,但我只需要更多帮助。
I think I managed to answer my own question.我想我设法回答了我自己的问题。 I simply had to add the outcome of the condition to another pd.where
statement and continue until I have scanned all columns.我只需要将条件的结果添加到另一个pd.where
语句并继续,直到我扫描了所有列。
df['Acceleration'] = df['Tags0'].where(df['Tags0'].str.contains('Acceleration', na=False),
df['Tags1'].where(df['Tags1'].str.contains('Acceleration', na=False),
df['Tags2'].where(df['Tags2'].str.contains('Acceleration', na=False),
df['Tags3'].where(df['Tags3'].str.contains('Acceleration', na=False),
df['Tags4'].where(df['Tags4'].str.contains('Acceleration', na=False),'Unknown')))))
Try this and see if it is what you are after:试试这个,看看它是否是你所追求的:
Assumption here is that all columns are strings;这里的假设是所有列都是字符串; if not, you can do some modification by selecting dtypes first:如果没有,您可以先选择 dtypes 进行一些修改:
condition = df.transform(lambda x: x.str.contains("Acceleration", na=False))
Create the new column using the where function on the entire dataframe, and forward filling on the columns
axis to get your desired column:在整个 dataframe 上使用where function 创建新列,并在columns
轴上向前填充以获得所需的列:
df.assign(acc_col=df.where(condition).ffill(axis = 'columns').iloc[:, -1])
Tags0 Tags1 Tags2 acc_col
0 Alumni-2017,Acceleration-2016 None None Alumni-2017,Acceleration-2016
1 Alumni Acceleration-2017 None Acceleration-2017
2 Acceleration-2015 None None Acceleration-2015
3 Alumni-2017 Acceleration-2015 None Acceleration-2015
4 Alumni-2017 Acceleration-2014 None Acceleration-2014
If there are any nulls in acc_col
, you can use the replace
function or fillna
to fill it with your preferred value ('Unknown').如果acc_col
中有任何空值,您可以使用replace
function 或fillna
用您的首选值(“未知”)填充它。
In Power Query, I think you could possibly convert the columns into a list, and run the search/extraction within the list, (possibly use List.Contains) combined with a single if statements, rather than individual if/else checks.在 Power Query 中,我认为您可以将列转换为列表,并在列表中运行搜索/提取,(可能使用 List.Contains)结合单个 if 语句,而不是单独的 if/else 检查。 This may require you to write M code.这可能需要您编写 M 代码。 My power query skills has not been called into action in a while, so I may be wrong on the suggestion.我的能量查询技能已经有一段时间没有发挥作用了,所以我的建议可能是错误的。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.