Logstash grok 过滤器 apache 模式

Question

This is a sample Apache Tomcat log:

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2

I would like to capture the following variables

09/Sep/2019:00:08:02 +0200

/en/search-results?p_p_id=portal2....

35S64857F6860FDFC0F60B5B47A97E18

Can you help me with that? I want to index only those and drop the others, is it possible? Thank you

Answer 1

Use this grok pattern:

%{GREEDYDATA:field1} %{IP:ip1} - - \[%{GREEDYDATA:date}] \"%{WORD:method} %{GREEDYDATA:request}" %{WORD:numbers} %{WORD:numbers} %{WORD:numbers} - - %{WORD:last_parameter}

input:

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2

output:

{
  "field1": [
    [
      "portal.portal.some.thing.int:8443"
    ]
  ],
  "ip1": [
    [
      "13.233.220.113"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "13.233.220.113"
    ]
  ],
  "date": [
    [
      "09/Sep/2019:00:08:02 +0200"
    ]
  ],
  "method": [
    [
      "GET"
    ]
  ],
  "request": [
    [
      "/en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1"
    ]
  ],
  "numbers": [
    [
      "230",
      "334734",
      "6261"
    ]
  ],
  "last_parameter": [
    [
      "35S64857F6860FDFC0F60B5B47A97E18"
    ]
  ]
}

fields you want are:

• date
• request
• last_parameter

You can remove other fields using remove field in mutate filter.