[英]Python: How to delete rows ending in certain characters?
I have a large data file and I need to delete rows that end in certain letters. 我有一个大型数据文件,我需要删除以某些字母结尾的行。
Here is an example of the file I'm using: 这是我正在使用的文件的示例:
User Name DN
MB212DA CN=MB212DA,CN=Users,DC=prod,DC=trovp,DC=net
MB423DA CN=MB423DA,OU=Generic Mailbox,DC=prod,DC=trovp,DC=net
MB424PL CN=MB424PL,CN=Users,DC=prod,DC=trovp,DC=net
MBDA423 CN=MBDA423,OU=DNA,DC=prod,DC=trovp,DC=net
MB2ADA4 CN=MB2ADA4,OU=DNA,DC=prod,DC=trovp,DC=netenter code here
Code I am using: 我正在使用的代码:
from pandas import DataFrame, read_csv
import pandas as pd
f = pd.read_csv('test1.csv', sep=',',encoding='latin1')
df = f.loc[~(~pd.isnull(f['User Name']) & f['UserName'].str.contains("DA|PL",))]
How do I use regular expression syntax to delete the words that end in "DA" and "PL" but make sure I do not delete the other rows because they contain "DA" or "PL" inside of them? 如何使用正则表达式语法删除以“DA”和“PL”结尾的单词,但请确保我不删除其他行,因为它们中包含“DA”或“PL”?
It should delete the rows and I end up with a file like this: 它应该删除行,我最终得到一个这样的文件:
User Name DN
MBDA423 CN=MBDA423,OU=DNA,DC=prod,DC=trovp,DC=net
MB2ADA4 CN=MB2ADA4,OU=DNA,DC=prod,DC=trovp,DC=net
First 3 rows are deleted because they ended in DA and PL. 删除前3行因为它们以DA和PL结束。
You could use this expression 你可以使用这个表达式
df = df[~df['User Name'].str.contains('(?:DA|PL)$')]
It will return all rows that don't end in either DA or PL. 它将返回所有不以DA或PL结尾的行。
The ?:
is so that the brackets would not capture anything. ?:
是这样括号不会捕获任何东西。 Otherwise, you'd see pandas returning the following (harmless) warning: 否则,您会看到pandas返回以下(无害)警告:
UserWarning: This pattern has match groups. To actually get the groups, use str.extract.
Alternatively, using endswith()
and without regular expressions, the same filtering could be achieved by using the following expression: 或者,使用
endswith()
并且没有正则表达式,可以使用以下表达式实现相同的过滤:
df = df[~df['User Name'].str.endswith(('DA', 'PL'))]
As expected, the version without regular expression will be faster. 正如所料,没有正则表达式的版本会更快。 A simple test, consisting of
big_df
, which consists of 10001 copies of your original df
: 一个简单的测试,由
big_df
组成,由原始df
的10001个副本组成:
# Create a larger DF to get better timing results
big_df = df.copy()
for i in range(10000):
big_df = big_df.append(df)
print(big_df.shape)
>> (50005, 2)
# Without regular expressions
%%timeit
big_df[~big_df['User Name'].str.endswith(('DA', 'PL'))]
>> 10 loops, best of 3: 22.3 ms per loop
# With regular expressions
%%timeit
big_df[~big_df['User Name'].str.contains('(?:DA|PL)$')]
>> 10 loops, best of 3: 61.8 ms per loop
You can use a boolean mask whereby you check if the last two characters of User_Name
are in not ( ~
) in a set of two character endings: 你可以使用一个布尔掩码来检查
User_Name
的最后两个字符是否在一组两个字符结尾的非( ~
)中:
>>> df[~df.User_Name.str[-2:].isin(['DA', 'PA'])]
User_Name DN
2 MB424PL CN=MB424PL, CN=Users, DC=prod, DC=trovp, DC=net
3 MBDA423 CN=MBDA423, OU=DNA, DC=prod, DC=trovp, DC=net
4 MB2ADA4 CN=MB2ADA4, OU=DNA, DC=prod, DC=trovp, DC=nete...
Instead of regular expressions
, you can use the endswith()
method to check if a string ends with a specific pattern. 您可以使用
endswith()
方法检查字符串是否以特定模式结束,而不是regular expressions
。
Ie: 即:
for row in rows:
if row.endswith('DA') or row.endswith('PL'):
#doSomething
You should create another df using the filtered data, and then use pd.to_csv()
to save a clean version of your file. 您应该使用筛选的数据创建另一个df,然后使用
pd.to_csv()
来保存文件的干净版本。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.