Why won't Logstash multiline merge lines based on grok'd field?

Question

I'm trying to get logstash multiline to work with the following test file:

val=abc
123 abc
test

and using the following config for the filter:

filter
{
    if [message] =~ "val"
    {
         match => ["message", "val=%{WORD:calc}"
    }
    multiline
    {
        pattern => [calc]
        what => "next"
    }
}

The output shows up as follows (with the other fields stripped):

"message" => "val=abc"
"calc" => "abc"
...
"message" => "123 abc"

The above lets me know that the grok is matching (hence the "calc" field) but I'm not sure why the multiline isn't merging the the first and 2nd line

Answer 1

Do you mean if the calc field exist, the first line and the second line will merge to a single envet?

If yes, the following answer can help you. Your multiline pattern is incorrect. Please refer to this config:

input {
    stdin{}
}

filter {
    if [message] =~ "val"
    {
        grok {
            match => ["message", "val=%{WORD:calc}"]
        }
    } 
    multiline
    {
        pattern => "(val)"
        what => "next"
    }
}

output {
    stdout {
        codec => "rubydebug"
    }
}

The pattern in multiline is when the message field has the val word, you meet the pattern and it will multiline merge with the second line. In your example you use [cal] that's means when the message field has the cal word, however, there is no any cal in the message field.