無法使用 Logstash 將數據從 CSV 加載到 Elasticsearch

Question

我正在使用 Logstash 2.3.3 將數據從 csv 文件加載到 Elasticsearch 2.3.3。

這是我的 Logstash 配置文件：

input {
  #stdin {
   # type => "stdin-type"
  #}

  file {
    path => "/Users/hieupm/export.txt"
    start_position => "beginning"
    codec => plain {
      charset => "CP1252"
    }
  }
}

filter {
  mutate {
    gsub => [
      "message", "\"", " ",
      "message", " +", " ",
      "message", " ;", ";",
      "message", "; ", ";"
    ]
    lowercase => ["message"]
  }
  csv {
    columns => ["siren","raison_sociale","enseigne","cp_ville","naf","nature_juridique","code_tranche"] 
    separator => ","
  }
}

output {
  #stdout {
  #}

  elasticsearch {
    hosts => "127.0.0.1:9200"
    index => "ereputation"
    document_id => "%{siren}"
  }
}

執行日志（在終端中）沒有給出任何錯誤，但它只是顯示如下並且從未完成

Settings: Default pipeline workers: 8
Pipeline main started

我的樣本數據：

siren,raison_sociale,enseigne,cp_ville,naf,nature_juridique,code_tranche 
5420021,ETABLISSEMENTS BIQUEZ,,80100 ABBEVILLE,4669B,5710,11
5420120,STE DES SUCRERIES DU MARQUENTERRE,,62140 MARCONNELLE,1081Z,5599,11
5480082,STE BRETONNE DE TRAVAUX PUBLICS,,44600 ST NAZAIRE,4312A,5599,21
5480546,SOCIETE DE DISTRIBUTION DE PAPIER,,44350 GUERANDE,1712Z,5710,21
5520176,CARTONNAGE HERNAS,,80210 FEUQUIERES EN VIMEU,1721A,5710,21
5520242,PROSPA,,80510 LONGPRE LES CORPS SAINTS,2030Z,5710,12
5520325,ETABLISSEMENTS ADRIEN RIQUIER,,80570 DARGNIES,4674B,5710,12
5540380,SNC COMTE ET CIE,PHARMACIE COMTE,04000 DIGNE LES BAINS,4773Z,5202,11
5541552,ETABLISSEMENTS BONNET FRERES,,04370 BEAUVEZER,5610C,5499,03

感謝您的任何建議！

Answer 1

我是elasticsearch_loader的作者
如果您只想將 CSV 數據加載到 elasticsearch 中，您可以使用 elasticsearch_loader
我為這個確切的問題寫了 ESL

你可以用pip下載它：

pip install elasticsearch-loader

然后您將能夠通過發出以下命令將 csv/json/parquet 文件加載到 elasticsearch 中：

elasticsearch_loader --index ereputation --type incident csv input1.csv input2.csv