[英]Cannot parse CSV file with Logstash
我在使用logstash將CSV文件導入ElasticSearch中時遇到問題,以便在Kibana中進行進一步處理。
這是我的logstash配置文件:
input {
file {
path => ["/absolute_path_to_file/export.csv"]
start_position => beginning
ignore_older => 0
sincedb_path => "/dev/null"
}
}
#filter {
# csv {
# columns => [
# "id",
# "cislo_smlouvy",
# "zdroj",
# "produkt",
# "sjednani",
# "datum_odeslani",
# "cas_odeslani",
# "pojistovna",
# "tarif",
# "pojistnik",
# "telefon",
# "predmet_pojisteni",
# "rz",
# "pocatek_pojisteni",
# "rocni_pojistne",
# "urgence",
# "stav"
# ]
# separator => ";"
# remove_field => ["message"]
# }
#}
output {
# elasticsearch {
# hosts => "localhost:9200"
# index => "smlouvy"
# }
stdout {
codec => rubydebug
}
}
摘自我的CSV文件:
"id";"číslo smlouvy";"zdroj";"produkt";"sjednání";"datum odeslaní";"čas odeslání";"pojišťovna";"tarif";"pojistník";"pojistnik telefon";"předmět pojištění";"rz";"počátek";"roční pojistné";"urgence";"stav"
"114951";"6132681255";"SRO";"POV";;"1.6.2016";"12:28";"csob";"csob-2";"BB TEST";"721666333";"Škoda Favorit";"NENÍ";"2.6.2016 00:00";"4657,00";;"TEST"
"114950";;"POV";"POV";"VO Bukvicova";"1.6.2016";"12:16";"csob";"csob-2";"BB BB";"721000111";"BMW X3";"NENÍ";"3.6.2016 00:00";"5550,00";;"TEST"
我正在調用此命令:
sudo logstash -f /absolute_path_to_file/logstash.conf --vebrose
具有以下輸出:
starting agent {:level=>:info}
starting pipeline {:id=>"main", :level=>:info}
Settings: Default pipeline workers: 2
Registering file input {:path=>["/absolute_path_to_file/export.csv"], :level=>:info}
Starting pipeline {:id=>"main", :pipeline_workers=>2, :batch_size=>125, :batch_delay=>5, :max_inflight=>250, :level=>:info}
Pipeline main started
一會兒什么都不做之后,我關閉了它:
^CSIGINT received. Shutting down the agent. {:level=>:warn}
stopping pipeline {:id=>"main"}
Closing inputs {:level=>:info}
Closed inputs {:level=>:info}
Input plugins stopped! Will shutdown filter/output workers. {:level=>:info}
Pipeline main has been shutdown
可能相關的版本信息:
logstash 2.3.2
logstash-input-file (2.2.5)
logstash-filter-csv (2.1.3)
logstash-output-elasticsearch (2.6.2)
logstash-output-stdout (2.0.6)
logstash-codec-rubydebug (2.0.7)
我已經閱讀了所有可以找到的文檔,並嘗試從GitHub復制很多logstash.conf示例,但是沒有運氣。 有什么我想念的幫助嗎?
所以我終於找到了問題所在。 它與輸入的CSV文件相同。
CSV文件的\\r
為新行,logstash的默認值為\\n
。
順便說一句:您不能在logstash文件輸入過濾器配置中將\\r
設置為定界符,因此我不得不使用\\n
將CSV文件轉換為一個
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.