How can I get logType and level from Json ElasticSearch by using Nest?

Question

I am trying to get value of loglevel and logType from below json logs of ElasticSearch by using Nest Code. LogLevel should be "ERROR" LogType should be "Response". How can I write this Nest query. I think that Nest is Really difficult.

        var settings = new connectionsettings(new uri("http://localhost:9200")).defaultındex("mylog-2019-07-29");

        var client = new elasticclient(settings);

        var searchresponse = client.search<rootobject>(s => s
                                        .allındices()
                                        .from(0)
                                        .size(10)
                                        .query(q => q
                                             .match(m => m
                                                .field(f => f.logLevel)
                                                .query("Error")
                                             )
                                        ));

        var r3 = client.search<rootobject>(search => search.matchall().ındex("mylog-2019-07-29")).documents;

        // OR 
        var result = client.search<rootobject>(q => q.allındices());
        var result2 = result.hits;

        using (httpclient client = new httpclient())
        {
            var result = await client.getasync("http://localhost:9200");

            var data = jsonconvert.deserializeobject<list<rootobject>>(result.content.readasstringasync().result);

        }

How can I do that with below json?

{
  "took": 72,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2968679,
    "max_score": 1.0,
    "hits": [
      {
        "_index": "slog-2019-07-29",
        "_type": "logmessage",
        "_id": "335af90f-6dab-459d-95f2-df09ffb15380",
        "_score": 1.0,
        "_source": {
          "id": "335af90f-6dab-459d-95f2-df09ffb15380",
          "parentLogId": "3ccf12bb-028e-498d-bc04-1394746c1d6f",
          "logDate": "2019-07-29T11:05:04.0554493Z",
          "logLevel": "Information",
          "logSource": "TourService",
          "logType": "Response",
          "environment": "Production",
          "sessionId": null,
          "traceId": "d6dc979e-4ce6-4c16-8f8a-7ed41c1cd87a",
          "klm": null,
          "data": "{\"isSuccess\":true,\"code\":null,\"message\":null,\"data\":{\"xxxx\":{\"provider\":0,\......\"xxxxxxxx\":0.0}]}]}}}",
          "processCost": 107.6808,
          "urlMethod": "POST",
          "urlPath": "/api/tour/get",
          "urlQueryString": null,
          "responseStatusCode": 200,
          "headers": [
            {
              "key": "Content-Type",
              "value": [
                "application/json; charset=utf-8"
              ]
            },
            {
              "key": "Request-Context",
              "value": [
                "appId=cid-v1:ec6df344-e1df-4097-bef4-69a3cce06eb6"
              ]
            }
          ],
          "additionalData": {}
        }
      },
      {
        "_index": "slog-2019-07-29",
        "_type": "logmessage",

        .....
        ....
        ..
        .

My Index MAP:

{
    "slog-2019-07-29": {
        "mappings": {
            "logmessage": {
                "properties": {
                    "additionalData": {
                        "properties": {
                            "data": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            },
                            "method": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            },
                            "url": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            }
                        }
                    },
                    "caller": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "correlationId": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "data": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "environment": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "headers": {
                        "properties": {
                            "key": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            },
                            "value": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            }
                        }
                    },
                    "id": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "logDate": {
                        "type": "date"
                    },
                    "logLevel": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "logSource": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "logType": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "parentLogId": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "processCost": {
                        "type": "float"
                    },
                    "responseStatusCode": {
                        "type": "long"
                    },
                    "urlMethod": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "urlPath": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "urlQueryString": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    }
                }
            }
        }
    }
}

Answer 1

Because you want to get back documents with exact value in the field, you need to use term query . So this elasticsearch query will work for you:

GET slog-2019-07-29/_search
{
  "query": {
    "term": {
      "logLevel.keyword": {
        "value": "Information"
      }
    }
  }
}

Notice that for the field I used logLevel.keyword instead of logLevel this is because logLevel.keyword field is of type keyword which is very useful in your case because elasticsearch doesn't analyse value in such fields, so Information is saved in elasticsearch as Information . That will allow you to find documents with exact value in specific fields. For logLevel field which is of type text that is saved as information .

We also need to use a term query because we don't want elasticsearch to analyse what we have put into value .

Now, with the NEST you can write such query with the following code:

var searchResponse = await elasticClient
    .SearchAsync<Document>(s => s
        .Query(q => q
            .Term(t => t
                .Field(f => f.LogLevel.Suffix("keyword"))
                .Value("Information"))));

class Document
{
    public string LogLevel { get; set; }
}

LogLevel should be "ERROR" LogType should be "Response".

For this, you would need to use bool query which will allow you to combine multiple conditions into one query.

GET slog-2019-07-29/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "logLevel.keyword": {
              "value": "ERROR"
            }
          }
        },
        {
          "term": {
            "logType.keyword": {
              "value": "Response"
            }
          }
        }
      ]
    }
  }
}

NEST has a really nice section in docs on how to build bool queries , so I'll leave it for you as an exercise.

Also, I encourage you to read Elasticsearch: The Definitive Guide which will give you a great overview of basics and how things work in elasticserch.

Hope that helps.