How do I not match a bare hyphen in Elasticsearch?

Question

I am querying apache logs stored in Elasticsearch. I want to return log entries from a given hostname that has a hyphen and with a populated auth field.

These strings should be an exact match: "hostname": "example-dev" and not "auth": "-" .

My questions are:

1. How do I correctly remap a type in Elasticsearch to allow a hyphen to be part of the matched string.
2. How do I correctly query a type in Elasticsearch with a bare hyphen.

The hyphen is a reserved character in Elasticsearch, so I understand it takes special effort. However, I'm having what seems like a lot of trouble figuring out how to include it in my query.

I have tried to remap the type to be not_analysed . It looks like the format has recently changed. The old way of defining the index ( "analysed" , "not_analysed" , and "no" ) makes sense to me. The new way ( true or false ) does not. In either case, I cannot seem to get remapping to work.

Here is my attempt at remapping:

DELETE /search
PUT search
{
    "mappings" : {
        "beat" : {
            "properties" : {
                "hostname" : {
                    "type" : "text",
                    "norms" : false,
                    "index" : false
                }
            }
        }
    }
}

I have not included the remapping of the auth field because it only returns a mapper_parsing_exception .

I am using json to query Elasticsearch. Here is my query:

GET _search
{
    "query": {
        "bool": {
            "filter": {
                "bool": {
                    "must": [
                        {
                            "match": {
                                "beat.hostname": "example-dev"
                            }
                        }
                    ],
                    "must_not": [
                        {
                            "match": {
                                "auth.keyword": "-"
                            }
                        }
                    ]
                }
            }
        }
    }
}

I have tried escaping the hyphen with \\\\- but that returns results that match "auth": "-" . The hostname still does not match exactly. The hostname query also matches something like "example-prod".

I have tried using "term" rather than "match"; that returns no results.

I can match a specific string for "auth", for example "must": { "match": { "auth": "foo" } } returns all entries for auth = "foo". That is opposite of what I need, but it does work. The hostname is still not exactly matched if it includes a hyphen.

The log entries are parsed into Elasticsearch using ELK stack, however this will be a report that is generated outside of Kibana for legacy reasons.

I have read the documentation and examples, but there is a lot to dig through. Many of the examples I have found are for older versions of Elasticsearch, which is understandable, but confusing.

I am new to Elasticsearch. It feels like I am just overlooking something, but it the problem might stem from a basic misunderstanding of how Elasticsearch is doing things.

Answer 1

After spending some more time with ElascticSearch queries, I think I have it figured out.

Splitting the hostname string into two separate string and matching for both filters the hostname as expected. Using an empty string for the negative match also seems to work as expected.

Here is the updated query:

{
"query": {
    "bool": {
        "filter": {
            "bool": {
                "must": [
                    {
                        "match": {
                            "beat.hostname": "example"
                        }
                    },
                    {
                        "match": {
                            "beat.hostname": "dev"
                        }
                    }
                ],
                "must_not": [
                    {
                        "match_phrase": {
                            "auth.keyword": ""
                        }
                    }
                ]
            }
        }
    }
}

I will do bit more testing is need to make sure this is actually returning what I need.

I was trying too hard to make ElasticSearch fit what I expected. Instead of working with ElasticSearch, I was trying to fight against it.