fluentd regexp to extract events from a log file

Question

I'm new to fluentd.

I have a log that I want to push to AWS with fluentd but I can't figure out what the regexp should be.

All the log lines, except the multilines, start with a UUID.

Here's a sample log:

6b0815f2-8ff1-4181-a4e6-058148288281 2020-11-03 13:00:05.976366 [DEBUG] switch_core_state_machine.c:611 (some_other_data) State Change CS_REPORTING -> CS_DESTROY

And, I'm trying to get UUID, DateTime, and Message.

With this regex:

/^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>.*) (?<message>[^ ]*)/gm

I'm getting the last word CS_DESTROY .

I tried fluentular and still got:

text:

f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914 2020-11-03 14:32:34.975779 [CRIT] mod_dptools.c:1866 audio3: https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg

regexp:

^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>.*) (?<message>[^ ]*)$

and got:

time    2020/11/03 14:32:34 +0000
UUID    f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914
message https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg

It's missing what's between the datetime and "https".

Answer 1

Try:

^(?<UUID>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) (?<time>[^\[]*) (?<message>\[.*)$

Live at rubular: https://rubular.com/r/JQQXs5VTkr2IxM

Here's the output for both logs:

Match 1
UUID    6b0815f2-8ff1-4181-a4e6-058148288281
time    2020-11-03 13:00:05.976366
message [DEBUG] switch_core_state_machine.c:611 (some_other_data) State Change CS_REPORTING -> CS_DESTROY

Match 2
UUID    f6a6e1ae-e52e-4aba-a8a5-4e3cc7f40914
time    2020-11-03 14:32:34.975779
message [CRIT] mod_dptools.c:1866 audio3: https://mydomain.s3-eu-west-1.amazonaws.com/media/576d06e5-04fc-11eb-a52c-020fd8c14d18/5f9ddf2d5df0f698094395.mpg