如何使用agrep獲得模糊字符串匹配的精確常用“max.distance”值？
[英]How can I get the precise common “max.distance” value for fuzzy string matching using agrep?
問題描述
我試圖找出使用agrep在兩個字符串名稱之間進行模糊字符串匹配的最佳精度。
但是，我需要選擇一個精度“max.distance”來在我想要匹配的所有字符串中應用相同的字符串，因為字符串的數量很大。 我不能為我想要匹配的每個字符串選擇最佳精度值“max.distance”。
例如，假設每個“BANK OF AMERICA CORP”和“1st Capital Bank”使用精度“max.distance”為“0.2”，“0.1”和“0.05”。
首先，下面是“BANK OF AMERICA CORP”的“max.distance”為“0.2”，“0.1”和“0.05”：
> agrep("BANK OF AMERICA CORP",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.2)
[1] "BANK OF AMERICA/PRIVATE BANK WEST" "BANK OF AMERICA SECURITIES"
[3] "BANK OF AMERICA SEC LLC" "BANK OF AMERICA SECURITIES LLC"
[5] "BANK OF AMERICA NT & SA" "BANK OF AMERICA CORP"
[7] "ALLIANZ OF AMERICA CORP" "Bank of America Securities/Vice Pre"
[9] "Bank of America Securities/Investme" "Bank of America/President"
[11] "Bank of America Securities LLC/Prin" "Bank of America Securities LLC/Mana"
[13] "Bank of America Securities LLC/Inve" "Bank of America Securities/Principa"
[15] "Bank of America Securities LLC/Bank" "Bank of America Sec/Investment Bank"
[17] "Bank Of America Securities/Managing" "Bank of America/Chairman--Midwest A"
[19] "Bank of America Securities LLC/Vice" "Bank of America Corporation/Sales C"
[21] "Bank of America Securities/Broker" "Bank of America Corporation/Banker"
[23] "Bank of America Corporation/Senior" "Bank of America Securities/Equity R"
[25] "Bank of America Corporation/Vice Ch" "BANK OF AMERICA CORPORATION"
[27] "BANK OF AMERICA HEADQUARTERS" "BANK OF AMERICA ADMINISTRATION"
[29] "BANK OF AMERICA N A" "Bank of America/Commercial Banking"
[31] "Bank of America Sec./Investment Ban"
>
> agrep("BANK OF AMERICA CORP",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.1)
[1] "BANK OF AMERICA CORP" "ALLIANZ OF AMERICA CORP"
[3] "Bank of America Corporation/Sales C" "Bank of America Corporation/Banker"
[5] "Bank of America Corporation/Senior" "Bank of America Corporation/Vice Ch"
[7] "BANK OF AMERICA CORPORATION"
>
> agrep("BANK OF AMERICA CORP",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.05)
[1] "BANK OF AMERICA CORP" "Bank of America Corporation/Sales C"
[3] "Bank of America Corporation/Banker" "Bank of America Corporation/Senior"
[5] "Bank of America Corporation/Vice Ch" "BANK OF AMERICA CORPORATION"
然后下面是“第一資本銀行”，“max.distance”為“0.2”，“0.1”和“0.05”：
> agrep("1st Capital Bank",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.2)
[1] "HURST CAPITAL PARTNERS"
[2] "SOY CAPITAL BANK"
[3] "FIRST CAPITOL BANK OF VICTOR"
[4] "OSTERWEIS CAPITAL MANAGEMENT"
[5] "1ST NATIONAL BANK"
[6] "FIRST CAPITAL BANK"
[7] "SEATTLE 1ST NAT'L BANK"
[8] "FIELD POINT CAPITAL MANAGEMENT"
[9] "SUMMERSET CAPITAL MANAGEMENT"
[10] "AMERIQUEST CAPITAL ASSOC"
[11] "BB&T CAPITAL MARKETS"
[12] "HUGHES CAPITAL MANAGEMENT"
[13] "WELLS CAPITAL MANAGEMENT"
[14] "SUPERIOR ST CAPITAL ADVISORS"
[15] "ORMES CAPITAL MARKETS INC"
[16] "1ST NAT'L BANK OF IL"
[17] "ADVENT CAPITAL MANAGEMENT"
[18] "1ST CAPITOL BANK"
[19] "BIONDI REISS CAPITAL MANAGEMENT"
[20] "CCYBYS CAPITAL MARKETS"
[21] "SEACOAST CAPITAL PARTNERS"
[22] "DOUGLAS CAPITAL MANAGEMENT"
[23] "HIGHFIELDS CAPITAL MANAGEMENT"
[24] "PRECEPT CAPITAL MANAGEMENT LP"
[25] "AUGUST CAPITAL MANAGEMENT"
[26] "SAKSA CAPITAL MANAGEMENT"
[27] "IMS CAPITAL MANAGEMENT"
[28] "TRENT CAPITAL MANAGEMENT"
[29] "Ormes Capital Management"
[30] "GARNET CAPITAL MANAGEMENT LLC"
[31] "INTERFASE CAPITAL MANAGERS"
[32] "RJS CAPITAL MANAGEMENT INC"
[33] "1ST NATIONAL BANK OF DE KALB"
[34] "1ST NAT'L BANK OF PHILLIPS CO"
[35] "1ST NAT'L BANK OF OKLAHOMA"
[36] "PROGRESS CAPITAL MANAGEMENT INC"
[37] "CAPITAL BANK & TRUST"
[38] "1ST NATL BANK"
[39] "ASB Capital Management/Real Estate"
[40] "Sears Capital Management"
[41] "Osterweis Capital Management/Invest"
[42] "Cerberus Capital Management LP/Asse"
[43] "LVS Capital Management/President"
[44] "1st Central Bank/Banker"
[45] "Summit Capital Management"
[46] "Orwes Capital Markets/Stockbroker"
[47] "Ormes Capital Management/Investment"
[48] "Nevis Capital Management/Investment"
[49] "Duncan Hurst Capital Management"
[50] "Progress Capital Management/Preside"
[51] "Cerberus Capital Management LP"
[52] "Wit Capital/Banker"
[53] "Ormes Capital Markets Inc."
[54] "Ormes Capital Markets/President & C"
[55] "Berents & Hess Capital Management"
[56] "Progress Capital Management/Venture"
[57] "First Capital Bank of KY"
[58] "Foothill Capital/Banker"
[59] "Pequot Capital Management/Equity Re"
[60] "First Dominion Capital/Banking"
[61] "Greenwhich Capital/Banker"
[62] "Veritas Capital Management/Banker"
[63] "Veritas Capital Management/Investme"
[64] "Lesese Capital Management/Investmen"
[65] "Douglas Capital Management/Investme"
[66] "FIRST NATINAL BANK OF AMARILLO"
[67] "NEVIS CAPITAL MANAGEMENT"
[68] "VERITAS CAPITAL MANAGEMENT"
[69] "SIEBERT CAPITAL MARKETS"
[70] "HOURGLASS CAPITAL MANAGEMENT"
[71] "1ST NATIONAL BANK DALHART"
[72] "TEXAS CAPITAL BANK"
[73] "NICHOLAS CAPITAL MANAGEMENT"
[74] "CERBUS CAPITAL MANAGEMENT"
[75] "CROESUS CAPITAL MANAGEMENT"
[76] "EAST WEST CAPITAL ASSOCIATES INC"
[77] "PRENDERGAST CAPITAL MANAGEMENT"
[78] "NANTUCKET CAPITAL MANAGEMENT"
[79] "1ST NATIONAL BANK TEMPLE"
[80] "ENTRUST CAPITAL INC"
[81] "1ST NATIONAL BANK OF IL"
[82] "SIMMS CAPITAL MANAGEMENT"
[83] "FIRST CAPITAL ADVISORS"
[84] "FIRST CAPITAL MANAGEMENT LTD"
[85] "1ST NATIONAL BANK & TRUST"
[86] "PENTECOST CAPITAL MANAGEMENT INC"
[87] "EAST-WEST CAPITAL ASSOCIATES"
[88] "1ST NAT'L BANK OF JOLIET"
[89] "FIRST CAPITOL BANK OF VICTO"
[90] "FIRST CAPITAL FINANCIAL"
[91] "PACIFIC COAST CAPITAL PARTNERS"
[92] "FIRST CAPITOL BANK"
[93] "FIRST CAPITAL ENGINEERING"
[94] "MIDWEST CAPITOL MANAGEMENT"
[95] "PEQUOT CAPITAL MANAGEMENT"
[96] "AGGOTT CAPITAL MANAGEMENT"
[97] "SIMMS CAPITAL MANAGEMENT INC"
[98] "PHILLIPS CAPITAL MANAGEMENT LLC"
[99] "1ST NATIONAL BANK OF COLD SP"
[100] "SOY CAPITOL BANK"
>
> agrep("1st Capital Bank",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.1)
[1] "FIRST CAPITOL BANK OF VICTOR" "FIRST CAPITAL BANK"
[3] "1ST CAPITOL BANK" "First Capital Bank of KY"
[5] "TEXAS CAPITAL BANK" "FIRST CAPITOL BANK OF VICTO"
[7] "FIRST CAPITOL BANK"
>
> agrep("1st Capital Bank",C1999_0[,2],ignore.case = TRUE, value = TRUE,fixed = TRUE,max.distance =0.05)
[1] "FIRST CAPITAL BANK" "1ST CAPITOL BANK"
[3] "First Capital Bank of KY"
正如您所看到的，很難找到“max.distance”的公共精度值來申請每個字符串，例如“BANK OF AMERICA CORP”和“1st Capital Bank”。 除了這兩個之外，我還有更多的公司名稱，這就是為什么我很難找到模糊字符串匹配的公共精度值和命令。
C1999_0的原始數據文件太大而無法附加，所以我認為只使用上面顯示的輸出值就足以復制了。
我知道有幾個子類別可以操作，例如成本，替換，插入等，但它們與僅更改“max.distance”值本身沒有太大區別。
如果我能得到幫助，我將非常感激！
2 個解決方案
解決方案1
3 2018-09-22 19:16:54
有一個問題agrep是，它像grep如中記錄help("grep")
由於有人不經意地閱讀了描述，甚至提交了錯誤報告，請注意這與
x的每個元素的子串（就像grep一樣）而不是整個元素。 另請參閱包utils中的adist，它可選擇返回匹配的子字符串的偏移量。
這似乎是后一個例子中的問題，因為你有許多包含“Capital”或“Bank”或兩者的名字。 我要做的是，用來計算Levenshtein距離 （這是agrep所做的或廣義版本， 僅適用於子串 ）並采用距離最短的距離。 例如，
C1999 <- c("HURST CAPITAL PARTNERS", "SOY CAPITAL BANK", "FIRST CAPITOL BANK OF VICTOR", "OSTERWEIS CAPITAL MANAGEMENT", "1ST NATIONAL BANK", "FIRST CAPITAL BANK", "SEATTLE 1ST NAT'L BANK", "FIELD POINT CAPITAL MANAGEMENT", "SUMMERSET CAPITAL MANAGEMENT", "AMERIQUEST CAPITAL ASSOC", "BB&T CAPITAL MARKETS", "HUGHES CAPITAL MANAGEMENT", "WELLS CAPITAL MANAGEMENT", "SUPERIOR ST CAPITAL ADVISORS", "ORMES CAPITAL MARKETS INC", "1ST NAT'L BANK OF IL", "ADVENT CAPITAL MANAGEMENT", "1ST CAPITOL BANK", "BIONDI REISS CAPITAL MANAGEMENT", "CCYBYS CAPITAL MARKETS", "SEACOAST CAPITAL PARTNERS", "DOUGLAS CAPITAL MANAGEMENT", "HIGHFIELDS CAPITAL MANAGEMENT", "PRECEPT CAPITAL MANAGEMENT LP", "AUGUST CAPITAL MANAGEMENT", "SAKSA CAPITAL MANAGEMENT", "IMS CAPITAL MANAGEMENT", "TRENT CAPITAL MANAGEMENT", "Ormes Capital Management", "GARNET CAPITAL MANAGEMENT LLC", "INTERFASE CAPITAL MANAGERS", "RJS CAPITAL MANAGEMENT INC", "1ST NATIONAL BANK OF DE KALB", "1ST NAT'L BANK OF PHILLIPS CO", "1ST NAT'L BANK OF OKLAHOMA", "PROGRESS CAPITAL MANAGEMENT INC", "CAPITAL BANK & TRUST", "1ST NATL BANK", "ASB Capital Management/Real Estate", "Sears Capital Management", "Osterweis Capital Management/Invest", "Cerberus Capital Management LP/Asse", "LVS Capital Management/President", "1st Central Bank/Banker", "Summit Capital Management", "Orwes Capital Markets/Stockbroker", "Ormes Capital Management/Investment", "Nevis Capital Management/Investment", "Duncan Hurst Capital Management", "Progress Capital Management/Preside", "Cerberus Capital Management LP", "Wit Capital/Banker", "Ormes Capital Markets Inc.", "Ormes Capital Markets/President & C", "Berents & Hess Capital Management", "Progress Capital Management/Venture", "First Capital Bank of KY", "Foothill Capital/Banker", "Pequot Capital Management/Equity Re", "First Dominion Capital/Banking", "Greenwhich Capital/Banker", "Veritas Capital Management/Banker", "Veritas Capital Management/Investme", "Lesese Capital Management/Investmen", "Douglas Capital Management/Investme", "FIRST NATINAL BANK OF AMARILLO", "NEVIS CAPITAL MANAGEMENT", "VERITAS CAPITAL MANAGEMENT", "SIEBERT CAPITAL MARKETS", "HOURGLASS CAPITAL MANAGEMENT", "1ST NATIONAL BANK DALHART", "TEXAS CAPITAL BANK", "NICHOLAS CAPITAL MANAGEMENT", "CERBUS CAPITAL MANAGEMENT", "CROESUS CAPITAL MANAGEMENT", "EAST WEST CAPITAL ASSOCIATES INC", "PRENDERGAST CAPITAL MANAGEMENT", "NANTUCKET CAPITAL MANAGEMENT", "1ST NATIONAL BANK TEMPLE", "ENTRUST CAPITAL INC", "1ST NATIONAL BANK OF IL", "SIMMS CAPITAL MANAGEMENT", "FIRST CAPITAL ADVISORS", "FIRST CAPITAL MANAGEMENT LTD", "1ST NATIONAL BANK & TRUST", "PENTECOST CAPITAL MANAGEMENT INC", "EAST-WEST CAPITAL ASSOCIATES", "1ST NAT'L BANK OF JOLIET", "FIRST CAPITOL BANK OF VICTO", "FIRST CAPITAL FINANCIAL", "PACIFIC COAST CAPITAL PARTNERS", "FIRST CAPITOL BANK", "FIRST CAPITAL ENGINEERING", "MIDWEST CAPITOL MANAGEMENT", "PEQUOT CAPITAL MANAGEMENT", "AGGOTT CAPITAL MANAGEMENT", "SIMMS CAPITAL MANAGEMENT INC", "PHILLIPS CAPITAL MANAGEMENT LLC", "1ST NATIONAL BANK OF COLD SP", "SOY CAPITOL BANK")
func <- function(x, y, tol = 0L){
require(stringdist)
dista <- stringdist::stringdist(x, y, method = "lv")
min_dista <- min(dista)
y[dista <= min_dista + tol]
}
func("1st Capital Bank", C1999)
#R [1] "Wit Capital/Banker"
func("1st Capital Bank", C1999, 4L)
#R [1] "Wit Capital/Banker" "First Capital Bank of KY"
func("1st Capital Bank", C1999, 10L)
#R [1] "SOY CAPITAL BANK" "1ST NATIONAL BANK"
#R [3] "FIRST CAPITAL BANK" "1ST CAPITOL BANK"
#R [5] "Ormes Capital Management" "1ST NATL BANK"
#R [7] "Sears Capital Management" "1st Central Bank/Banker"
#R [9] "Summit Capital Management" "Wit Capital/Banker"
#R [11] "Ormes Capital Markets Inc." "First Capital Bank of KY"
#R [13] "Foothill Capital/Banker" "Greenwhich Capital/Banker"
#R [15] "TEXAS CAPITAL BANK" "FIRST CAPITOL BANK"
#R [17] "SOY CAPITOL BANK"
# ignoring cases
func <- function(x, y, tol = 0L){
require(stringdist)
dista <- stringdist::stringdist(tolower(x), tolower(y), method = "lv")
min_dista <- min(dista)
y[dista <= min_dista + tol]
}
func("1st Capital Bank", C1999, 0L)
#R [1] "1ST CAPITOL BANK"
該tol在參數func如果要包括作為例子控制tol從最小Levenshtein距離漸行漸遠。 我看到我沒有准確回答你的要求（ 如何使用agrep獲得模糊字符串匹配的精確常用“max.distance”值？ ）但我認為我的答案可能就是你要找的。
我使用stringdist::stringdist而不是adist因為前者似乎更快。 它仍然可能有點慢，我希望在那里你可以設置最大距離的R包，但我沒有遇到過這樣的包。 這可以使得（然后加蓋的）Levenshtein距離的計算更快。
解決方案2
2 已采納 2018-09-26 02:44:50
這似乎是一個無法解決的問題，如上所述，沒有一個max.distance可以很好地適用於所有輸入字符串。
可能值得嘗試像tf-idf這樣的方法來識別字符串的異常，並將你的max.distance縮放到那個。 因此，“Ziggurat Mutual”可能比“First Bank National”更具通用性。
您也可以考慮使用fuzzyjoin包，它提供了一些嘗試不同選項的快速方法。 例如，您可以嘗試：
df <- c("HURST CAPITAL PARTNERS", "SOY CAPITAL BANK", "FIRST CAPITOL BANK OF VICTOR", "OSTERWEIS CAPITAL MANAGEMENT", "1ST NATIONAL BANK", "FIRST CAPITAL BANK", "SEATTLE 1ST NAT'L BANK", "FIELD POINT CAPITAL MANAGEMENT", "SUMMERSET CAPITAL MANAGEMENT", "AMERIQUEST CAPITAL ASSOC", "BB&T CAPITAL MARKETS", "HUGHES CAPITAL MANAGEMENT", "WELLS CAPITAL MANAGEMENT", "SUPERIOR ST CAPITAL ADVISORS", "ORMES CAPITAL MARKETS INC", "1ST NAT'L BANK OF IL", "ADVENT CAPITAL MANAGEMENT", "1ST CAPITOL BANK", "BIONDI REISS CAPITAL MANAGEMENT", "CCYBYS CAPITAL MARKETS", "SEACOAST CAPITAL PARTNERS", "DOUGLAS CAPITAL MANAGEMENT", "HIGHFIELDS CAPITAL MANAGEMENT", "PRECEPT CAPITAL MANAGEMENT LP", "AUGUST CAPITAL MANAGEMENT", "SAKSA CAPITAL MANAGEMENT", "IMS CAPITAL MANAGEMENT", "TRENT CAPITAL MANAGEMENT", "Ormes Capital Management", "GARNET CAPITAL MANAGEMENT LLC", "INTERFASE CAPITAL MANAGERS", "RJS CAPITAL MANAGEMENT INC", "1ST NATIONAL BANK OF DE KALB", "1ST NAT'L BANK OF PHILLIPS CO", "1ST NAT'L BANK OF OKLAHOMA", "PROGRESS CAPITAL MANAGEMENT INC", "CAPITAL BANK & TRUST", "1ST NATL BANK", "ASB Capital Management/Real Estate", "Sears Capital Management", "Osterweis Capital Management/Invest", "Cerberus Capital Management LP/Asse", "LVS Capital Management/President", "1st Central Bank/Banker", "Summit Capital Management", "Orwes Capital Markets/Stockbroker", "Ormes Capital Management/Investment", "Nevis Capital Management/Investment", "Duncan Hurst Capital Management", "Progress Capital Management/Preside", "Cerberus Capital Management LP", "Wit Capital/Banker", "Ormes Capital Markets Inc.", "Ormes Capital Markets/President & C", "Berents & Hess Capital Management", "Progress Capital Management/Venture", "First Capital Bank of KY", "Foothill Capital/Banker", "Pequot Capital Management/Equity Re", "First Dominion Capital/Banking", "Greenwhich Capital/Banker", "Veritas Capital Management/Banker", "Veritas Capital Management/Investme", "Lesese Capital Management/Investmen", "Douglas Capital Management/Investme", "FIRST NATINAL BANK OF AMARILLO", "NEVIS CAPITAL MANAGEMENT", "VERITAS CAPITAL MANAGEMENT", "SIEBERT CAPITAL MARKETS", "HOURGLASS CAPITAL MANAGEMENT", "1ST NATIONAL BANK DALHART", "TEXAS CAPITAL BANK", "NICHOLAS CAPITAL MANAGEMENT", "CERBUS CAPITAL MANAGEMENT", "CROESUS CAPITAL MANAGEMENT", "EAST WEST CAPITAL ASSOCIATES INC", "PRENDERGAST CAPITAL MANAGEMENT", "NANTUCKET CAPITAL MANAGEMENT", "1ST NATIONAL BANK TEMPLE", "ENTRUST CAPITAL INC", "1ST NATIONAL BANK OF IL", "SIMMS CAPITAL MANAGEMENT", "FIRST CAPITAL ADVISORS", "FIRST CAPITAL MANAGEMENT LTD", "1ST NATIONAL BANK & TRUST", "PENTECOST CAPITAL MANAGEMENT INC", "EAST-WEST CAPITAL ASSOCIATES", "1ST NAT'L BANK OF JOLIET", "FIRST CAPITOL BANK OF VICTO", "FIRST CAPITAL FINANCIAL", "PACIFIC COAST CAPITAL PARTNERS", "FIRST CAPITOL BANK", "FIRST CAPITAL ENGINEERING", "MIDWEST CAPITOL MANAGEMENT", "PEQUOT CAPITAL MANAGEMENT", "AGGOTT CAPITAL MANAGEMENT", "SIMMS CAPITAL MANAGEMENT INC", "PHILLIPS CAPITAL MANAGEMENT LLC", "1ST NATIONAL BANK OF COLD SP", "SOY CAPITOL BANK")
library(dplyr); library(fuzzyjoin)
df <- df %>% as_data_frame()
df %>%
# Allowable methods include osa, lv, dl, hamming, lcs, qgram,
# cosine, jaccard, jw, soundex
fuzzyjoin::stringdist_inner_join(df, method = "lv", distance_col = "distance", max_dist = 4) %>%
filter(distance > 0)
Joining by: "value"
# A tibble: 70 x 3
value.x value.y distance
<chr> <chr> <dbl>
1 SOY CAPITAL BANK 1ST CAPITOL BANK 4
2 SOY CAPITAL BANK SOY CAPITOL BANK 1
3 FIRST CAPITOL BANK OF VICTOR FIRST CAPITOL BANK OF VICTO 1
4 1ST NATIONAL BANK 1ST NATL BANK 4
5 FIRST CAPITAL BANK 1ST CAPITOL BANK 4
6 FIRST CAPITAL BANK FIRST CAPITOL BANK 1
7 HUGHES CAPITAL MANAGEMENT DOUGLAS CAPITAL MANAGEMENT 4
8 HUGHES CAPITAL MANAGEMENT AUGUST CAPITAL MANAGEMENT 4
9 WELLS CAPITAL MANAGEMENT IMS CAPITAL MANAGEMENT 4
10 WELLS CAPITAL MANAGEMENT NEVIS CAPITAL MANAGEMENT 3
...在您的示例列表中嘗試潛在的非完全匹配。
使用agrep（）模糊名稱匹配更快的R代碼用於多種模式......？
[英]Faster R code for fuzzy name matching using agrep() for multiple patterns…?
聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.