堆栈内存溢出
  繁体   English   中英

Elastic NEST 客户端如何按嵌套属性排序

[英]Elastic NEST Client how to sort by on a nested property

 原文  2021-05-09 00:43:17  3   1    c#/ elasticsearch/ nest/ asp.net-core-5.0

问题描述

作为 NEST 库的新手，用对我来说完全没有意义的文档来拉扯我的头发。

First Issue: Cannot figure out how to map the results from a query to a strongly type model, despite having captured the json result and pasting to a C# class, what ever I try, unless I use a dynamic type in the query method then the结果返回 null。

Model 派生自查询返回的 json 结果：

public class Rootobject
{
    public int took { get; set; }
    public bool timed_out { get; set; }
    public _Shards _shards { get; set; }
    public Hits hits { get; set; }
}

public class _Shards
{
    public int total { get; set; }
    public int successful { get; set; }
    public int skipped { get; set; }
    public int failed { get; set; }
}

public class Hits
{
    public Total total { get; set; }
    public float max_score { get; set; }
    public Hit[] hits { get; set; }
}

public class Total
{
    public int value { get; set; }
    public string relation { get; set; }
}

public class Hit
{
    public string _index { get; set; }
    public string _type { get; set; }
    public string _id { get; set; }
    public float _score { get; set; }
    public _Source _source { get; set; }
}

public class _Source
{
    public DateTime timestamp { get; set; }
    public string level { get; set; }
    public string messageTemplate { get; set; }
    public string message { get; set; }
    public Fields fields { get; set; }
}

public class Fields
{
    public string LogEventCategory { get; set; }
    public string LogEventType { get; set; }
    public string LogEventSource { get; set; }
    public string LogData { get; set; }
    public string MachineName { get; set; }
    public int MemoryUsage { get; set; }
    public int ProcessId { get; set; }
    public string ProcessName { get; set; }
    public int ThreadId { get; set; }
}

查询返回的示例 JSON 结果：

{
  "took" : 16,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 8,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "webapp-razor-2021.05",
        "_type" : "_doc",
        "_id" : "n2tbTnkBwE4YgJowzRsT",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2021-05-09T00:41:47.2321845+01:00",
          "level" : "Information",
          "messageTemplate" : "{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}",
          "message" : "\"WebApp-RAZOR\"\"Application Startup\"\"System\"\"Application Starting Up\"",
          "fields" : {
            "LogEventCategory" : "WebApp-RAZOR",
            "LogEventType" : "Application Startup",
            "LogEventSource" : "System",
            "LogData" : "Application Starting Up",
            "MachineName" : "DESKTOP-OS52032",
            "MemoryUsage" : 4713408,
            "ProcessId" : 15152,
            "ProcessName" : "WebApp-RAZOR",
            "ThreadId" : 1
          }
        }
      },
      {
        "_index" : "webapp-razor-2021.05",
        "_type" : "_doc",
        "_id" : "oGtdTnkBwE4YgJowuxu_",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2021-05-09T00:43:54.0326968+01:00",
          "level" : "Information",
          "messageTemplate" : "{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}",
          "message" : "\"WebApp-RAZOR\"\"Application Startup\"\"System\"\"Application Starting Up\"",
          "fields" : {
            "LogEventCategory" : "WebApp-RAZOR",
            "LogEventType" : "Application Startup",
            "LogEventSource" : "System",
            "LogData" : "Application Starting Up",
            "MachineName" : "DESKTOP-OS52032",
            "MemoryUsage" : 4656048,
            "ProcessId" : 12504,
            "ProcessName" : "WebApp-RAZOR",
            "ThreadId" : 1
          }
        }
      },
      {
        "_index" : "webapp-razor-2021.05",
        "_type" : "_doc",
        "_id" : "oWtgTnkBwE4YgJownRtc",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2021-05-09T00:47:02.8954368+01:00",
          "level" : "Information",
          "messageTemplate" : "{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}",
          "message" : "\"WebApp-RAZOR\"\"Application Startup\"\"System\"\"Application Starting Up\"",
          "fields" : {
            "LogEventCategory" : "WebApp-RAZOR",
            "LogEventType" : "Application Startup",
            "LogEventSource" : "System",
            "LogData" : "Application Starting Up",
            "MachineName" : "DESKTOP-OS52032",
            "MemoryUsage" : 4717560,
            "ProcessId" : 17952,
            "ProcessName" : "WebApp-RAZOR",
            "ThreadId" : 1
          }
        }
      }
    ]
  }
}

第二个问题：我正在尝试按嵌套在 JSON 中的属性进行排序，例如名为“LogEventCategory”的属性

NEST 客户端方法：

var searchResponse = await _elasticClient.SearchAsync<dynamic>(s => s
   .RequestConfiguration(r => r
        .DisableDirectStreaming()
    )
    //.AllIndices()
   .From(0) // From parameter defines the offset from the first result you want to fetch.
   .Size(3) // Size parameter allows you to configure the maximum amount of hits to be returned.
   .Index("webapp-razor-*")
   //.Index("index-1,index-2")
   .Query(q => q
        .MatchAll()
    )
    .Sort(so => so
         .Field(fs => fs
             //.Field("@timestamp") // this one seems to work
             .Field("logEventCategory")
             .Order(SortOrder.Ascending)
             //.Order(ColumnSortOrder)
          )
    )
);

知道 NEST 高级客户端针对适当的模型设计为 map，我意识到使用动态类型是不正确的，但我只是无法理解为什么使用直接从样本 Z466DEEC76ECDF2456D38571F63 响应创建/映射的 class 响应仍然不起作用并且这样做时返回的结果只为我提供了前几个属性的 null 值。

下面的示例仅看到前几个项目（但仍然为空）

在此处输入图像描述

1 个解决方案

解决方案1
 0 已采纳  2021-05-11 08:10:35

First Issue: Cannot figure out how to map the results from a query to a strongly type model, despite having captured the json result and pasting to a C# class, what ever I try, unless I use a dynamic type in the query method then the结果返回 null。

Model 派生自查询返回的 json 结果：

您不需要为整个 JSON 响应定义模型； 客户端将负责正确反序列化大部分响应，因此唯一需要创建的 POCO 是_source JSON object。 根据提供的示例，这将类似于

public class LogMessage
{
    [PropertyName("@timestamp")]
    public DateTimeOffset Timestamp { get; set; }
    
    public string Level {get;set;}
    
    public string MessageTemplate {get;set;}
    
    public string Message {get;set;}
    
    public Dictionary<string, object> Fields {get;set;}
}

我们可以通过创建一个存根响应并将其传递给客户端进行反序列化来检查它是否正确反序列化

private static void Main()
{
    var defaultIndex = "default_index";
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

    var json = @"{
  ""took"" : 16,
  ""timed_out"" : false,
  ""_shards"" : {
        ""total"" : 1,
    ""successful"" : 1,
    ""skipped"" : 0,
    ""failed"" : 0
  },
  ""hits"" : {
        ""total"" : {
            ""value"" : 8,
      ""relation"" : ""eq""
    
    },
    ""max_score"" : 1.0,
    ""hits"" : [
      {
            ""_index"" : ""webapp-razor-2021.05"",
        ""_type"" : ""_doc"",
        ""_id"" : ""n2tbTnkBwE4YgJowzRsT"",
        ""_score"" : 1.0,
        ""_source"" : {
                ""@timestamp"" : ""2021-05-09T00:41:47.2321845+01:00"",
          ""level"" : ""Information"",
          ""messageTemplate"" : ""{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}"",
          ""message"" : ""\""WebApp-RAZOR\""\""Application Startup\""\""System\""\""Application Starting Up\"""",
          ""fields"" : {
                    ""LogEventCategory"" : ""WebApp-RAZOR"",
            ""LogEventType"" : ""Application Startup"",
            ""LogEventSource"" : ""System"",
            ""LogData"" : ""Application Starting Up"",
            ""MachineName"" : ""DESKTOP-OS52032"",
            ""MemoryUsage"" : 4713408,
            ""ProcessId"" : 15152,
            ""ProcessName"" : ""WebApp-RAZOR"",
            ""ThreadId"" : 1

          }
            }
        },
      {
            ""_index"" : ""webapp-razor-2021.05"",
        ""_type"" : ""_doc"",
        ""_id"" : ""oGtdTnkBwE4YgJowuxu_"",
        ""_score"" : 1.0,
        ""_source"" : {
                ""@timestamp"" : ""2021-05-09T00:43:54.0326968+01:00"",
          ""level"" : ""Information"",
          ""messageTemplate"" : ""{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}"",
          ""message"" : ""\""WebApp-RAZOR\""\""Application Startup\""\""System\""\""Application Starting Up\"""",
          ""fields"" : {
                    ""LogEventCategory"" : ""WebApp-RAZOR"",
            ""LogEventType"" : ""Application Startup"",
            ""LogEventSource"" : ""System"",
            ""LogData"" : ""Application Starting Up"",
            ""MachineName"" : ""DESKTOP-OS52032"",
            ""MemoryUsage"" : 4656048,
            ""ProcessId"" : 12504,
            ""ProcessName"" : ""WebApp-RAZOR"",
            ""ThreadId"" : 1

          }
            }
        },
      {
            ""_index"" : ""webapp-razor-2021.05"",
        ""_type"" : ""_doc"",
        ""_id"" : ""oWtgTnkBwE4YgJownRtc"",
        ""_score"" : 1.0,
        ""_source"" : {
                ""@timestamp"" : ""2021-05-09T00:47:02.8954368+01:00"",
          ""level"" : ""Information"",
          ""messageTemplate"" : ""{@LogEventCategory}{@LogEventType}{@LogEventSource}{@LogData}"",
          ""message"" : ""\""WebApp-RAZOR\""\""Application Startup\""\""System\""\""Application Starting Up\"""",
          ""fields"" : {
                    ""LogEventCategory"" : ""WebApp-RAZOR"",
            ""LogEventType"" : ""Application Startup"",
            ""LogEventSource"" : ""System"",
            ""LogData"" : ""Application Starting Up"",
            ""MachineName"" : ""DESKTOP-OS52032"",
            ""MemoryUsage"" : 4717560,
            ""ProcessId"" : 17952,
            ""ProcessName"" : ""WebApp-RAZOR"",
            ""ThreadId"" : 1

          }
            }
        }
    ]
  }
}";

    // create a connection that always returns the above response
    var connection = new InMemoryConnection(Encoding.UTF8.GetBytes(json));

    var settings = new ConnectionSettings(pool, connection)
        .DefaultIndex(defaultIndex);
        
    var client = new ElasticClient(settings);

    // actually doesn't matter what query we send through as we'll always get the stubbed response
    var searchResponse = client.Search<LogMessage>(s => s);
    
    foreach (var document in searchResponse.Documents)
    {
        Console.WriteLine($"{document.Timestamp} {document.Message}");
    }
}

产生

9/05/2021 12:41:47 AM +01:00 "WebApp-RAZOR""Application Startup""System""Application Starting Up"
9/05/2021 12:43:54 AM +01:00 "WebApp-RAZOR""Application Startup""System""Application Starting Up"
9/05/2021 12:47:02 AM +01:00 "WebApp-RAZOR""Application Startup""System""Application Starting Up"

第二个问题：我正在尝试按嵌套在 JSON 中的属性进行排序，例如名为“LogEventCategory”的属性

在LogMessage POCO 中， fields被映射为Dictionary<string,object>因为我猜键/值可以是动态的。 要对此 object 中的属性进行排序，将是

var searchResponse = client.Search<LogMessage>(s => s
    .Sort(so => so
        .Ascending(f => f.Fields["LogEventCategory"])
    )
);

产生以下请求

POST http://localhost:9200/default_index/_search?pretty=true&typed_keys=true 
{
  "sort": [
    {
      "fields.LogEventCategory": {
        "order": "asc"
      }
    }
  ]
}

如果 Elasticsearch 已经推断出LogEventCategory的映射，即没有为其提供明确的数据类型映射，那么它将被映射为带有keyword 多字段（或子字段）的text字段。 尝试对未启用fielddata （默认禁用）的text字段进行排序将导致错误。 通常应该使用keyword字段进行排序和聚合，因此如果已推断出映射，则可以使用keyword多字段进行排序

var searchResponse = client.Search<LogMessage>(s => s
    .Sort(so => so
        .Ascending(f => f.Fields["LogEventCategory"].Suffix("keyword"))
    )
);

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何在NEST弹性搜索中基于命中分数对结果进行排序 如何在NEST弹性搜寻中按热门成绩排序结果 Elastic NEST 客户端：更新数据 如何使用 NEST 客户端为弹性搜索索引 Json 数据？ Elastic Nest将动态值传递给排序 通过 NEST 客户端在弹性搜索中获取索引的 GetById 弹性搜索 Nest 客户端未应用提升 单元测试 Elastic Search Nest 客户端 - BulkAll 如何使用nest client在elasticsearch中按数组搜索数组属性 如何将我的 DSL 聚合查询转换为 NEST 聚合弹性客户端查询？
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM