[英]Failed to parse @UNIX_MS from Logstash to Elasticsearch
So the data I want to store in Elasticsearch is an csv file. 所以我想存储在Elasticsearch中的数据是一个csv文件。 It contains the following: 它包含以下内容:
1465309033156,84,http://test.com/purchase.php,200,OK,ThreadGroup 1-3,true,6,6,84,testPC,0
1465309033469,176,http://test.com/,200,OK,ThreadGroup 1-7,true,7,7,176,testPC,91
Note that the first line equals the time in UNIX_MS. 请注意,第一行等于UNIX_MS中的时间。
I'm trying to send the data with logstash. 我正在尝试使用Logstash发送数据。 Here is my config file: 这是我的配置文件:
input {
file {
sincedb_path => "NUL"
ignore_older => 0
type => "csv"
path => ["C:/result/piet.jtl"]
start_position => "beginning"
}
}
filter {
csv {
columns => ["@jmeter_timestamp", ...]
separator => ","
}
date {
locale => "en"
match => ["@jmeter_timestamp", "UNIX_MS"]
remove_field => ["@jmeter_timestamp"]
target => "@timestamp"
timezone => "Europe/Amsterdam"
}
}
output {
elasticsearch {
template => "c:/result/piet.json"
template_name => "piet"
hosts => ["192.168.43.51:9200"]
index => "piet-%{+YYYY.MM.dd}"
}
}
Here is part of piet.json: 这是piet.json的一部分:
"mappings": {
"logs": {
"properties": {
"@timestamp": {
"type": "date"
},
Now i'm getting an error running the config file. 现在我在运行配置文件时遇到错误。
"error"=>{
"type"=>"mapper_parsing_exception",
"reason"=>"failed to parse [@timestamp]",
"caused_by"=>{
"type"=>"illegal_argument_exception",
"reason"=>"Invalid format: \"2016-06-07T14:17:34.611Z\" is malformed at \"-06-07T14:17:34.611Z\""
}}}}, :level=>:warn}
My stdout looks like this: 我的标准输出看起来像这样:
I'm just lost. 我只是迷路了。 How can I insert data from the csv file in Elasticsearch.. 如何从Elasticsearch中的csv文件插入数据。
Solved it myself. 我自己解决了。 Elasticsearch is complaining about the date, because it doesn't recognize the date correctly. Elasticsearch在抱怨日期,因为它无法正确识别日期。 So, I removed the mapping completely from Elasticsearch via the Kopf plugin. 因此,我通过Kopf插件从Elasticsearch中完全删除了映射。
I re-added the .json, including the following details. 我重新添加了.json,其中包括以下详细信息。 (Note the format) (注意格式)
"mappings": {
"logs": {
"properties": {
"@timestamp": {
"type": "date",
"format" : "strict_date_optional_time||epoch_millis"
It is working now. 现在正在工作。 Huray 哈雷
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.