logstash拆分路径，并通过其位置[2]取某个值并将其插入字段

Question

I need to extract from a path certain value by its position...

example: let's say I split this following path into array using '\\' as split char E:\\OUM82\\APP\\Logs\\UploadManager_20062019.log I should get something like this:

[0]=E:
[1]=OUM82
[2]=APP (this value I want to take into a field  )
[3]=logs
[4]=UploadManager_20062019.log

so, I'm always want to take whatever in [2] how do I implement it? its something with ruby?

Edit:

I Tried this approach: (like @baudsp suggested) but I'm still getting "_grokparsefailure"

grok {
              match => { path => "%{GREEDYDATA:pathDriveSign}\\%{GREEDYDATA:RootFolder}\\%{GREEDYDATA:customerFolder}" }
           }

here is the std output:

{
      "tags" => [
    [0] "beats_input_codec_plain_applied",
    [1] "_grokparsefailure"
],
     "agent" => {
            "type" => "filebeat",
    "ephemeral_id" => "bd6ace26-79cd-4297-bfb5-5add9f4b4217",
              "id" => "83fb6261-5872-4d95-853a-44f2cc41d436",
         "version" => "7.0.0",
        "hostname" => "OctUpload"
},
   "message" => "2019-06-13 17:40:34,591 INFO QueriesParserEngine.Run - GSP queries parser engine end. Total run time duration: 00:02:32.1831164 ",
"@timestamp" => 2019-06-22T16:25:26.204Z,
     "cloud" => {
    "provider" => "az",
     "machine" => {
        "type" => "Standard_DS13_v2"
    },
      "region" => "westeurope",
    "instance" => {
        "name" => "OctUpload",
          "id" => "768097b1-bfb9-4939-b99c-5337aede39ca"
    }
},
 "extractor" => "SQLSERVER",
     "input" => {
    "type" => "log"
},
       "ecs" => {
    "version" => "1.0.0"
},
  "@version" => "1",
    "fields" => {
    "logtype" => "log4net"
},
      "host" => {
              "os" => {
           "build" => "14393.2608",
         "version" => "10.0",
            "name" => "Windows Server 2016 Datacenter",
        "platform" => "windows",
          "kernel" => "10.0.14393.2608 (rs1_release.181024-1742)",
          "family" => "windows"
    },
              "id" => "d79c20df-4184-41a8-b95d-83669c8e3fbe",
            "name" => "OctUpload",
    "architecture" => "x86_64",
        "hostname" => "OctUpload"
},
       "log" => {
      "file" => {
        "path" => "E:\\OUM82\\Micron\\TI_DS_FILES\\SQLSERVER_LOGS\\QueriesParser-SQLS-BOMSSPROD66-2_13062019_173801 - Copy.log"
    },
    "offset" => 927068
}

}

Answer 1

NB : I'm not sure it's the best filter to use here, but it's the one I've used the most and it should work.

If you are only interested in the APP part of your path, you should be able to retrieve it with the grok filter.

Supposing that your path is in a field called path :

grok {
   match => {path => "^%{DATA}\\%{DATA}\\%{DATA:value}\\"}
}

The filter will put the value APP in the value field.

For more information on the grok filter:

• Official documentation
• Grok pattern tester
• Grok patterns list

Answer 2

another better solution by Badger from ELK team:

better solution by Badger from ELK team

You cannot do it with mutate+split (which is what I would normally suggest) due to this issue , which affects regexps, single quoted string, and double quoted strings.

It is possible using grok if you enable config.support_escapes on logstash.yml... Believe it or not

 grok { match => { "path" => "^(?<pathDriveSign>\\w{1}):\\\\\\\\(?<RootFolder>[^\\\\\\\\]+)\\\\\\\\(?<customerFolder>[^\\\\\\\\]+)\\\\\\\\." } } 

will get you

 "RootFolder" => "OUM82", "pathDriveSign" => "E", "customerFolder" => "APP", 

Do not ask me to explain why 4 backslashes are required to represent a single backslash.

---

There is also a sneaky way to do it in ruby. You cannot have a backslash at the end of a string, so we have a string that contains a backslash and extract the backslash from it.

 ruby { code => ' backslash = "\\\\Z"[0] event.set("components", event.get("path").split(backslash)) ' } 

results in

 "components" => [ [0] "E:", [1] "OUM82", [2] "APP", [3] "Logs", [4] "UploadManager_20062019.log" ]