![](/img/trans.png)
[英]How to push data to an existing index(Elastic Search) from Logstash?
[英]How to upload data to an existing index with Logstash?
我正在尝试使用 Logstash 将数据从 a.csv-File 插入到已经存在的索引(已经有数据)。
无论如何,这是我的 logstash_true.config 文件:
input {
file {
path => "pathToFile"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["title", "text", "subject", "date"]
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => "127.0.0.1:9200"
index => "news"
document_type => "true_news"
document_id => "%{id}"
}
}
上传数据时,我可以在命令行中看到文件或数据没有任何问题,并且 document_type true_news确实存在。
但是在尝试获取数据时:
{
"count" : 0,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
数据未加载。
更新
启用调试时出现以下错误:
Could not index event to Elasticsearch. {:status=>400, :action=>
["index", {:_id=>"%{id}", :_index=>"news", :routing=>nil,
:_type=>"true_news"}, #<LogStash::Event:0x7e10d60f>], :response=>
{"index"=>{"_index"=>"news", "_type"=>"true_news", "_id"=>"%{id}",
"status"=>400, "error"=>{"type"=>"illegal_argument_exception",
"reason"=>"Rejecting mapping update to [news] as the final mapping
would have more than 1 type: [fake_news, true_news]"}}}}
由于 Elasticsearch 6.0 版,您的索引中不能有多种类型。
您的索引news
似乎已经有类型为fake_news
的文档或映射,并且您正在尝试插入类型为true_news
的文档,这是不可能的,这就是您收到此错误的原因:
"type"=>"illegal_argument_exception",
"reason"=>"Rejecting mapping update to [news] as the final mapping
would have more than 1 type: [fake_news, true_news]"
由于您只能有 1 种类型并且您希望能够区分true_news
和fake_news
,因此最好重新创建索引以对每个文档使用默认类型doc
,并在文档中添加带有true_news
或fake_news
的标签在您的输入中使用配置add_tag => ["tag"]
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.