[英]ElasticSearch indexing issue ,failed to parse timestamp
我是ELK的新手。 我已經在Elasticsearch中創建了索引
{
"logstash": {
"aliases": {},
"mappings": {
"log": {
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "keyword",
"include_in_all": false
},
"activity": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"beat": {
"properties": {
"hostname": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"version": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
},
"filename": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"host": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"input_type": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"message": {
"type": "text",
"norms": false
},
"offset": {
"type": "long"
},
"source": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"tags": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"timestamp": {
"type": "date",
"include_in_all": false,
"format": "YYYY-MM-DD HH:mm:ss.SSS"
},
"type": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"user": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1488805244467",
"number_of_shards": "1",
"number_of_replicas": "0",
"uuid": "5ijhh193Tr6y_hxaQrW9kg",
"version": {
"created": "5020199"
},
"provided_name": "logstash"
}
}
}
}
下面是我的logstash配置
input{
beats{
port=>5044
}
}filter{
grok{
match=>{"message" => "\[%{TIMESTAMP_ISO8601:timestamp}\] ALL AUDIT: User \[%{GREEDYDATA:user}\] is %{GREEDYDATA:activity} \[%{GREEDYDATA:filename}\] for transfer."}
}
}output{
elasticsearch{
hosts=>"localhost:9200"
index=> "logstash"
}
樣本數據
[2017-03-05 12:37:21.465] ALL AUDIT: User [user1] is opening file [filename1] for transfer.
但是當我通過Elasticsearch中的filebeat> logstash> elasticsearch加載文件時,我得到以下錯誤
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp]
Caused by: java.lang.IllegalArgumentException: Invalid format: "2017-03-05T12:36:33.606" is malformed at "12:36:33.606"
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[joda-time-2.9.5.jar:2.9.5]
請幫忙,我應該配置哪種時間戳格式?
在時間戳映射中,將格式表示為"format": "YYYY-MM-DD HH:mm:ss.SSS"
在這里,您通過拍子發送的格式不相同,請檢查: 2017-03-05T12:36:33.606
這就是Elastic抱怨格式的原因。 您的格式應為: "YYYY-MM-DD'T'HH:mm:ss.SSS"
(注意大寫"YYYY-MM-DD'T'HH:mm:ss.SSS"
T)
請參閱文檔以獲取更多詳細信息: https : //www.elastic.co/guide/zh-CN/elasticsearch/reference/current/mapping-date-format.html
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.