简体   繁体   中英

How to create grok/json filter to parse the below json format

I want to parse this JSON to Kibana using Logstash

{
"Format": "IDEA0",
"ID": "2b03eb1f-fc4c-4f67-94e5-31c9fb32dccc",
"DetectTime": "2022-01-31T08:16:12.600470+07:00",
"EventTime": "2022-01-31T01:23:01.637438+00:00",
"Category": ['Intrusion.Botnet'],
"Confidence": 0.03,
"Note": "C&C channel, destination IP: 192.168.1.24 port: 8007/tcp score: 0.9324",
"Source": [{'IP4': ['192.168.1.25'], 'Type': ['CC']}]
}

I want that ID, Detect Time, Event Time, Category, Confidence, Note, Source is a single field so later i can do visualization in kibana.

Here's what I'm already trying to do

input {
        file {
                path => "/home/ubuntu/Downloads/StratosphereLinuxIPS/output/*.json"
                start_position => "beginning"
                sincedb_path => "/dev/null"
        }
}

filter {
        json {
                source => "message"
        }
}

output {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "test-test"
                user => "***"
                password => "***"
        }
        stdout{}
}

But the field is not separated correctly

Kibana 发现

Any help will be meaningful. Thanks.

:::UPDATE:::

I already found the solution (Help by other guys from Elastic forum but not 100% optimize need to tweak it a little more)

Here's the Logstash Conf I'm using if someone needs it in the future

input {
        file {
                path => "/home/ubuntu/Downloads/StratosphereLinuxIPS/output/alerts.json"
                start_position => "beginning"
                sincedb_path => "/dev/null"
                codec => multiline { pattern => "^{$" negate => "true" what => "previous" }
        }
}

filter {
        mutate {
                gsub => ["message", "'", '"']
        }
        json {
                source => "message"
        }
}

output {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "test-keempat"
                user => "xxx"
                password => "xxx"
        }
        stdout{ codec => rubydebug }
}

Thanks !

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM