logstash拆分路徑，並通過其位置[2]取某個值並將其插入字段

Question

我需要根據其位置從路徑中提取特定值...

示例：假設我使用“ \\”將以下路徑拆分為數組，作為拆分字符E：\\ OUM82 \\ APP \\ Logs \\ UploadManager_20062019.log我應該得到類似以下內容：

[0]=E:
[1]=OUM82
[2]=APP (this value I want to take into a field  )
[3]=logs
[4]=UploadManager_20062019.log

因此，我一直想采用[2]中的任何內容，該如何實現？ 它與紅寶石有關？

編輯：

我嘗試了這種方法：（如建議使用@baudsp），但我仍然收到“ _grokparsefailure”

grok {
              match => { path => "%{GREEDYDATA:pathDriveSign}\\%{GREEDYDATA:RootFolder}\\%{GREEDYDATA:customerFolder}" }
           }

這是std輸出：

{
      "tags" => [
    [0] "beats_input_codec_plain_applied",
    [1] "_grokparsefailure"
],
     "agent" => {
            "type" => "filebeat",
    "ephemeral_id" => "bd6ace26-79cd-4297-bfb5-5add9f4b4217",
              "id" => "83fb6261-5872-4d95-853a-44f2cc41d436",
         "version" => "7.0.0",
        "hostname" => "OctUpload"
},
   "message" => "2019-06-13 17:40:34,591 INFO QueriesParserEngine.Run - GSP queries parser engine end. Total run time duration: 00:02:32.1831164 ",
"@timestamp" => 2019-06-22T16:25:26.204Z,
     "cloud" => {
    "provider" => "az",
     "machine" => {
        "type" => "Standard_DS13_v2"
    },
      "region" => "westeurope",
    "instance" => {
        "name" => "OctUpload",
          "id" => "768097b1-bfb9-4939-b99c-5337aede39ca"
    }
},
 "extractor" => "SQLSERVER",
     "input" => {
    "type" => "log"
},
       "ecs" => {
    "version" => "1.0.0"
},
  "@version" => "1",
    "fields" => {
    "logtype" => "log4net"
},
      "host" => {
              "os" => {
           "build" => "14393.2608",
         "version" => "10.0",
            "name" => "Windows Server 2016 Datacenter",
        "platform" => "windows",
          "kernel" => "10.0.14393.2608 (rs1_release.181024-1742)",
          "family" => "windows"
    },
              "id" => "d79c20df-4184-41a8-b95d-83669c8e3fbe",
            "name" => "OctUpload",
    "architecture" => "x86_64",
        "hostname" => "OctUpload"
},
       "log" => {
      "file" => {
        "path" => "E:\\OUM82\\Micron\\TI_DS_FILES\\SQLSERVER_LOGS\\QueriesParser-SQLS-BOMSSPROD66-2_13062019_173801 - Copy.log"
    },
    "offset" => 927068
}

}

Answer 1

注意 ：我不確定這是最好的過濾器，但它是我使用最多的過濾器，應該可以使用。

如果您只對路徑的APP部分感興趣，則應該可以使用grok過濾器進行檢索。

假設您的路徑位於名為path的字段中：

grok {
   match => {path => "^%{DATA}\\%{DATA}\\%{DATA:value}\\"}
}

過濾器會將值APP放入value字段。

有關grok過濾器的更多信息：

• 官方文件
• Grok模式測試儀
• 格羅模式列表

Answer 2

ELK團隊的Badger另一個更好的解決方案：

ELK團隊的Badger提供了更好的解決方案

由於這個問題 ，您無法使用mutate + split（我通常會建議這樣做），這會影響正則表達式，單引號字符串和雙引號字符串。

如果在logstash.yml上啟用config.support_escapes，則可以使用grok ...信不信由你

 grok { match => { "path" => "^(?<pathDriveSign>\\w{1}):\\\\\\\\(?<RootFolder>[^\\\\\\\\]+)\\\\\\\\(?<customerFolder>[^\\\\\\\\]+)\\\\\\\\." } } 

會得到你

 "RootFolder" => "OUM82", "pathDriveSign" => "E", "customerFolder" => "APP", 

不要要求我解釋為什么代表單個反斜杠需要4個反斜杠。

---

還有一種偷偷摸摸的方法來使用紅寶石。 字符串的末尾不能包含反斜杠，因此我們有一個包含反斜杠並從中提取反斜杠的字符串。

 ruby { code => ' backslash = "\\\\Z"[0] event.set("components", event.get("path").split(backslash)) ' } 

結果是

 "components" => [ [0] "E:", [1] "OUM82", [2] "APP", [3] "Logs", [4] "UploadManager_20062019.log" ]