简体   繁体   中英

update mapping property for data going to elasticsearch index

New to elasticsearch here and debugging an existing index. I was creating a dashboard based on a search and found that some fields that were being sent do not appear as an option to filter on. I checked further into this and saw that there are some fields that are not indexed. The person who created the index claims that there is no restriction on what fields are being indexed but I disagree having found the following:

     "customerid": {
              "type": "string",
              "norms": {
                 "enabled": false
              },
              "fields": {
                 "raw": {
                    "type": "string",
                    "index": "not_analyzed",
                    "ignore_above": 256
                 }
              }
           },

It shows the value as not_analyzed. I would like to update this value so that the fields I need are indexed and available for visualizations in the dashboard. I know the index name but the data itself and mapping is under types. So not sure how to do this. When looking in elasticsearch plugin/head I do not see the index.

"customer_index": {
        "dynamic_templates": [
           {
              "string_fields": {
                 "mapping": {
                    "index": "analyzed",
                    "omit_norms": true,
                    "type": "multi_field",
                    "fields": {

except this initial headers for all fields related. Any suggestions or help will be appreciated.

EDIT:

As pointed out correctly by Alain, my misunderstanding of not_analyzed. I am still confused a bit and hope that adding some additional information will help diagnose this problem.

Firstly, this is a view of kibana that shows for the specific index that contains the data, the available fields:

Available Fields
   @timestamp
   _id
   _type
   etc.

Customer ID is not one of them. Now there are different data sources coming to the same index for example :

job records
customer records
project records 

etc.

This is defined by _type field. Now I want to access the customer record object and it has its own properties:

customer_index": {
        "dynamic_templates": [
           {
              "string_fields": {
                 "mapping": {
                    "index": "analyzed",
                    "omit_norms": true,
                    "type": "multi_field",
                    "fields": {
                       "{name}": {
                          "index": "analyzed",
                          "omit_norms": true,
                          "type": "string"
                       },
                       "raw": {
                          "ignore_above": 256,
                          "index": "not_analyzed",
                          "type": "string"
                       }
                    }
                 },
                 "match": "*",
                 "match_mapping_type": "string"
              }
           },
           {
              "message_field": {
                 "mapping": {
                    "index": "analyzed",
                    "omit_norms": true,
                    "type": "string"
                 },
                 "match": "message",
                 "match_mapping_type": "string"
              }
           }
        ],
        "_all": {
           "enabled": true,
           "omit_norms": true
        },
        "properties": {
           "@timestamp": {
              "type": "date",
              "format": "dateOptionalTime"
           },
           "@version": {
              "type": "string",
              "index": "not_analyzed"
           },
           "CCType": {
              "type": "string",
              "norms": {
                 "enabled": false
              },
              "fields": {
                 "raw": {
                    "type": "string",
                    "index": "not_analyzed",
                    "ignore_above": 256
                 }
              }
           },
           "Crawled": {
              "type": "string",
              "norms": {
                 "enabled": false
              },
              "fields": {
                 "raw": {
                    "type": "string",
                    "index": "not_analyzed",
                    "ignore_above": 256
                 }
              }
           },
           "customerid": {
              "type": "string",
              "norms": {
                 "enabled": false
              },
              "fields": {
                 "raw": {
                    "type": "string",
                    "index": "not_analyzed",
                    "ignore_above": 256
                 }
              }
           },

Now I would like to search for these property fields. I found SENSE plugin which I am trying to use to learn the queries and was able to do this:

GET _search
{
"query": {
    "match": {
       "customerid": "11908906"
    }
}

}

This worked great in returning the messages and counts. Now when I try within Kibana in discover tab, I search for the type (customer_index) and filter the field I need and I see all the results. I just wondering how to translate this visually.

"not_analyzed" does not mean "not indexed". It means that elasticsearch is not trying to analyze the string, instead leaving it intact. This is a very common thing in elasticsearch, depending on your content.

For example:

/var/log/messages

will, by default, be split into:

[ "var", "log", "messages" ]

which is not very useful when you want to search on it as a full path. By setting the field to not_analyzed, it will leave it alone. You can still search using the field.

Second, you say that the fields aren't available for filtering. Assuming you're using Kibana, be aware that Kibana will cache the field mapping, meaning that new fields don't automatically show up. Go to Settings->Indices, select your index, and click Reload Field List.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM