How to find pattern starts and ends with in a file in python

Question

How to write a Regex to find lines starting with TCP 0.0.0.0 , ending with ongoing text from a log file.

Log file sample:

ensePo#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 100.1.1.1 0 15 Regular "policy100" ongoing 127410 120891 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0023-00005CD33064
    Line 1896: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 101.1.1.1 0 15 Regular "policy101" ongoing 127259 120740 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0024-00005CD33064
    Line 1897: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy102" ongoing 130656 124136 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0025-00005CD33064
    Line 1898: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy103" ongoing 130504 123984 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0026-00005CD33064
    Line 1899: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy104" ongoing 130351 123831 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0027-00005CD33064
    Line 1900: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy105" ongoing 130350 123830 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0028-00005CD33064
    Line 1901: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy106" ongoing 130196 123678 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0029-00005CD33064
    Line 1902: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy107" ongoing 130044 123524 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002A-00005CD33064
    Line 1903: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy108" ongoing 129890 123371 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002B-00005CD33064
    Line 1904: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy109" ongoing 129739 123220 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002C-00005CD33064
    Line 1905: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 11.1.1.1 0 15 Regular "policy11" ongoing 137160 130641 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002D-00

I want print those lines and count them . New to python please help

Answer 1

Try this code:

import re

f = open("sample_log.txt", "r")
counter = 0
print("="*20)
for line in f:
    match = re.search("(TCP 0\.0\.0\.0) (.*) (ongoing)", line)
    if match:
        counter += 1
        print("-"*10)

        # If you want to print the whole line
        print("Count {}:[F] {}".format(counter, line.rstrip()))

        # if you want to print just the matched section
        # print("Count {}:[M] {}".format(counter, match.groups()[1].rstrip()))

print("="*20)
print("Total Found: {}".format(counter))
f.close()

The output will look like:

----------
Count 1:[F] ensePo#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 100.1.1.1 0 15 Regular "policy100" ongoing 127410 120891 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0023-00005CD33064
----------
Count 2:[F]     Line 1896: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 101.1.1.1 0 15 Regular "policy101" ongoing 127259 120740 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0024-00005CD33064
----------
Count 3:[F]     Line 1897: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy102" ongoing 130656 124136 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0025-00005CD33064
----------
Count 4:[F]     Line 1898: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy103" ongoing 130504 123984 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0026-00005CD33064
----------
Count 5:[F]     Line 1899: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy104" ongoing 130351 123831 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0027-00005CD33064
----------
Count 6:[F]     Line 1900: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy105" ongoing 130350 123830 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0028-00005CD33064
----------
Count 7:[F]     Line 1901: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy106" ongoing 130196 123678 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0029-00005CD33064
----------
Count 8:[F]     Line 1902: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy107" ongoing 130044 123524 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002A-00005CD33064
----------
Count 9:[F]     Line 1903: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy108" ongoing 129890 123371 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002B-00005CD33064
----------
Count 10:[F]     Line 1904: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy109" ongoing 129739 123220 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002C-00005CD33064
----------
Count 11:[F]     Line 1905: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 11.1.1.1 0 15 Regular "policy11" ongoing 137160 130641 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002D-00
====================
Total Found: 11

Given the sample_log.txt being:

asdadsfgdsgsd-- SOME JUNK -- fgdfsgdfsg asdfaskdjlfh alkjsdflkahjs jsdfs
ensePo#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 100.1.1.1 0 15 Regular "policy100" ongoing 127410 120891 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0023-00005CD33064
    Line 1896: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 101.1.1.1 0 15 Regular "policy101" ongoing 127259 120740 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0024-00005CD33064
asdadsfgdsgsdfgdfsgdfsg asdfaskdjlfh alkjsdflkahjs jsdfs
    Line 1897: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy102" ongoing 130656 124136 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0025-00005CD33064
    Line 1898: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy103" ongoing 130504 123984 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0026-00005CD33064
    Line 1899: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy104" ongoing 130351 123831 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0027-00005CD33064
    Line 1900: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy105" ongoing 130350 123830 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0028-00005CD33064
    Line 1901: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy106" ongoing 130196 123678 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-0029-00005CD33064
    Line 1902: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy107" ongoing 130044 123524 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002A-00005CD33064
    Line 1903: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy108" ongoing 129890 123371 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002B-00005CD33064
    Line 1904: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 0.0.0.0 0 15 Regular "policy109" ongoing 129739 123220 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002C-00005CD33064
    Line 1905: DefensePro#08-05-2019 15:47:55 WARNING 240 Anomalies "TCP handshake violation, first packet not syn" TCP 0.0.0.0 0 11.1.1.1 0 15 Regular "policy11" ongoing 137160 130641 N/A 0 N/A low drop FFFFFFFF-FFFF-FFFF-002D-00