Elasticsearch和Laravel Scout-Elasticsearch-Driver时间戳格式错误

Question

I have successfully configured ES and the babenkoivan/scout-elasticsearch-driver, but run into this error when adding new entries to the DB:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [updated_at.raw]"}],"type":"mapper_parsing_exception","reason":"failed to parse [updated_at.raw]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"2018-07-13 07:52:02\" is malformed at \" 07:52:02\""}},"status":400}

I have set the format in the mapping like this, and according to the ES docs this format should work:

protected $mapping = [
        'properties' => [
           'created_at' => [
                'type' => 'date',
                'format' => 'yyyy-MM-DD HH:mm:ss',
                'fields' => [
                    'raw' => [
                        'type' => 'date',
                        'index' => 'not_analyzed'
                    ]
                ]
            ],
            'updated_at' => [
                'type' => 'date',
                'format' => 'yyyy-MM-DD HH:mm:ss',
                'fields' => [
                    'raw' => [
                        'type' => 'date',
                        'index' => 'not_analyzed'
                    ]
                ]
            ]
        ]
    ];

https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html#multiple-date-formats

Is there something I'm missing here?

Answer 1

In your mapping you defined a custom date format ( yyyy-MM-DD HH:mm:ss ) for created_at and updated_at . The raw fields instead are a date type too, but use the default format (which according the doc is date_optional_time , meaning yyyy-MM-DD'T'HH:mm:ss ).

This means that the former expects 2018-07-13 07:52:02 , while the latter 2018-07-13T07:52:02 , so you indexing can't possibly avoid breaking one of the two.

Now, the use of multi-fields is meant to index values in different ways, but what you are doing is to create a new field raw with basically the same properties of the base value (they are both date types, except for the inconsistency in the format, of course).

So, in my opinion you options are:

1. if you don't have any specific use for raw you can remove it from the mapping. Sorting and matching works well with the base field.

    "created_at": {"type": "date", "format": "yyyy-MM-DD HH:mm:ss"} 

2. if you need to keep the original string format (as "raw" may suggest) you can use a keyword type

    "created_at": {"type": "date", "format": "yyyy-MM-DD HH:mm:ss", "fields": {"raw": {"type": "keyword"}}} 

3. if you really need the raw field as is, you have to specify a format that is consistent with the other one:

    "created_at": {"type": "date", "format": "yyyy-MM-DD HH:mm:ss", "fields": {"raw": {"type": "date", "format": "yyyy-MM-DD HH:mm:ss"}}}