[英]Fuzzy matching with pyspark or python
我正在尝试使用 pyspark 或 python 进行模糊匹配,其中我有 2 个列表。
一世。 城市标准值表
Clarksburg
Fremont
San Leandro
Albuquerque
Columbus
San Jose
Martinez
New York
Alhambra
Unknown
Las Vegas
Dublin
Niagara Falls
ii. 拼写错误的城市名称列表
Clarksburg
Closed 10/97
Fre,Nont
Fremong
San L:Eandro
Albuquerue
Clmbs
Sanjse
Martinz
New Yrk
Alambra
00011
L Vegas
Vegas
Ssan jose
Nw Yrk
Colmbus
Klarkburg
Alburque
Dublin
Niegara F
现在我想将拼写错误的城市名称与标准值列表匹配,并创建另一个具有适当匹配的列表。 我在下面找 output
Clarksburg - Clarksburg
Closed 10/97 - Unknown
Fre,Nont - Fremont
Fremong - Fremont
San L:Eandro - San Leandro
Albuquerue - Albuquerque
Clmbs -Columbus
Sanjse - San Jose
Martinz - Martinez
New Yrk - New York
Alambra - Alhambra
00011 - Unknown
L Vegas - Las Vegas
Vegas - Las Vegas
Ssan jose - San Jose
Nw Yrk - New York
Colmbus - Columbus
Klarkburg - Clarksburg
Alburque - Albuquerque
Dublin - Dublin
Niegara F - Niagara Falls
任何帮助都会真正帮助我。 提前致谢。
使用fuzzywuzzy
,并更改threshold
以满足您的要求:
from fuzzywuzzy import process
threshold = 40
matchlist = [x for x in """
Clarksburg
Fremont
San Leandro
Albuquerque
Columbus
San Jose
Martinez
New York
Alhambra
Unknown
Las Vegas
Dublin
Niagara Falls
""".splitlines() if x]
checklist = [x for x in """
Clarksburg
Closed 10/97
Fre,Nont
Fremong
San L:Eandro
Albuquerue
Clmbs
Sanjse
Martinz
New Yrk
Alambra
00011
L Vegas
Vegas
Ssan jose
Nw Yrk
Colmbus
Klarkburg
Alburque
Dublin
Niegara F
""".splitlines() if x]
for check in checklist:
match = process.extractOne(check, matchlist)
print(f"{check} - {match[0] if match[1] > threshold else 'Unknown'}")
这给了我:
Clarksburg - Clarksburg
Closed 10/97 - Unknown
Fre,Nont - Fremont
Fremong - Fremont
San L:Eandro - San Leandro
Albuquerue - Albuquerque
Clmbs - Columbus
Sanjse - San Jose
Martinz - Martinez
New Yrk - New York
Alambra - Alhambra
00011 - Unknown
L Vegas - Las Vegas
Vegas - Las Vegas
Ssan jose - San Jose
Nw Yrk - New York
Colmbus - Columbus
Klarkburg - Clarksburg
Alburque - Albuquerque
Dublin - Dublin
Niegara F - Niagara Falls
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.