从文件中读取字符串的特定区域

Question

I'm trying to figure out how to do some stuff with python. I got a text file, which containes strings ie :

M, 1, 14/08/2019 11:39, 4, xxxx, name, “Initialization of the system, and loading
M, 1, 14/08/2019 11:40, 100, xxxx, name, “Open Connection”
M, 1, 14/08/2019 11:40, 100, xxxx, name, “Close Connection, and reboot”
S, 1, 14/08/2019 11:40, 6, xxxx, name, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User tal logged in
M, 1, 14/08/2019 11:39, 4, xxxx, name, “Initialization of the system”
S, 1, 14/08/2019 11:40, 6, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, User logged in, User tal logged in
S, 1, 14/08/2019 11:42, 3, User logged in, User tal logged in
M, 2, 14/08/2019 11:43, 100, yyy, yura, 12345, Message

What I'm trying to do is getting into the file, and if its the first time there is M,1 I should print some text, same if its S,1 or M,2 or S,1. I also have to print only selected rows from the file (didnt make it yet but I will with a row counter). What I also have to do is to print only selected columns, what I mean by columns is there is a seperator ',' between columns, ie if I want to print 3 and 4 colums of rows 1 and 2 I should print only 14/08/2019 11:39 , 4 and 14/08/2019 11:40 , 100. I figured out already how to split the strings with re.split, But I have no idea how to continue. Thanks.

import re
import string
filename = '11.txt'
def infile(filename):
    m1 = m2 = s1 = s2 = 0
    linecounter = 1
    lines = [1,2,3]
    colums = [2,4]
    i=0
    fin = open(filename, 'r')
    if fin.closed:
        print ('file is closed')
    lines = fin.readlines()
    for line in lines:
        if(line[0] == 'M' and line[3] == '1' and m1 == 0):
            print('---M, 1, Datetime, Error Level, DeviceId, UserId, Message---\n')
            m1 = 1
        elif (line[0] == 'M' and line[3] == '2' and m2 == 0):
            print('---M, 2, Datetime, Error Level, DeviceId, UserId, MobileId, Message---\n')
            m2 = 1
        elif (line[0] == 'S' and line[3] == '1' and s1 == 0):
            print('---S, 1, Datetime, Error Level, DeviceId, Action, Message---\n')
            s1 = 1
        elif (line[0] == 'S' and line[3] == '2' and s2 == 0):
            print('---S, 2, Datetime, Error Level, DeviceId, IP, Action, Message---\n')
            s2 = 1
        for p in re.split(",",line): // thats a check of spliting, nothing else
            print("piece="+p)
        print(line)

infile(filename)

Answer 1

From re.split(",",line) , which returns a vector, you can access you desired values using eg:

slit_str=re.split(",",line)
split_str[2] #Returns the dates
split_str[3] #Returns the number in the column after the date

To speed up, you can also break the loop if m1,m2,s1 and s1 == 1, use break

Answer 2

I have created a function below select_columns that will take an array of int's (for the columns) and then split the line by the , delimeter and return a string of the collated values.

Hope this helps

import re
import string
filename = '11.txt'
column_list = [3, 4] #Index 1 not index 0
def infile(filename, column_list):
    m1 = m2 = s1 = s2 = 0
    linecounter = 1
    lines = [1,2,3]
    colums = [2,4]
    i=0
    fin = open(filename, 'r')
    if fin.closed:
        print ('file is closed')
    lines = fin.readlines()
    for line in lines:
        if(line[0] == 'M' and line[3] == '1' and m1 == 0):
            print('---M, 1, Datetime, Error Level, DeviceId, UserId, Message---\n')
            print(select_columns(row = line, column_list = column_list))
            m1 = 1
        elif (line[0] == 'M' and line[3] == '2' and m2 == 0):
            print('---M, 2, Datetime, Error Level, DeviceId, UserId, MobileId, Message---\n')
            print(select_columns(row = line, column_list = column_list))
            m2 = 1
        elif (line[0] == 'S' and line[3] == '1' and s1 == 0):
            print('---S, 1, Datetime, Error Level, DeviceId, Action, Message---\n')
            print(select_columns(row = line, column_list = column_list))
            s1 = 1
        elif (line[0] == 'S' and line[3] == '2' and s2 == 0):
            print('---S, 2, Datetime, Error Level, DeviceId, IP, Action, Message---\n')
            print(select_columns(row = line, column_list = column_list))
            s2 = 1
        for p in re.split(",",line): # thats a check of spliting, nothing else
            print("piece="+p)
        print(line)

def select_columns(row, column_list):
    column_split = row.split(',')
    return_string = ''
    for column in column_list:
        return_string = '{0},{1}'.format(return_string, column_split[column - 1])
    return return_string[1:] # retruns the string trimming the first comma


infile(filename, column_list)

Answer 3

A simpler way to do this would be to load the file into a dataframe and then Filter rows based on column values

-->To Load as Dataframe:

data = pd.read_csv('output_list.txt', sep=" ", header=None)
data.columns = ["a", "b", "c", "etc."]

Load data from txt with pandas

to filter rows based on column values: pandas: filter rows of DataFrame with operator chaining https://cmdlinetips.com/2018/02/how-to-subset-pandas-dataframe-based-on-values-of-a-column/

Answer 4

you can split line and print column 2,3 by replacing the for loop using below code:

splittedLine = line.split(",")
print(splittedLine[2],splittedLine[3])

this will print:

14/08/2019 11:39  4
and so on.....

Answer 5

You can use a dictionary to maintain information regarding the first occurrence of each line's prefix, and then use the dictionary to print information accordingly.

Additionally, maintaining a mapping for each type ("M, 1 ", "M, 2" etc.) with its header will make it easier to print the end result.

import json
from pprint import pprint
input_string = """M, 1, 14/08/2019 11:39, 4, xxxx, name, “Initialization of the system, and loading
M, 1, 14/08/2019 11:40, 100, xxxx, name, “Open Connection”
M, 1, 14/08/2019 11:40, 100, xxxx, name, “Close Connection, and reboot”
S, 1, 14/08/2019 11:40, 6, xxxx, name, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, xxxx, User logged in, User tal logged in
M, 1, 14/08/2019 11:39, 4, xxxx, name, “Initialization of the system”
S, 1, 14/08/2019 11:40, 6, New User, We created the user in the systems
S, 1, 14/08/2019 11:41, 3, User logged in, User tal logged in
S, 1, 14/08/2019 11:42, 3, User logged in, User tal logged in
M, 2, 14/08/2019 11:43, 100, yyy, yura, 12345, Message"""


# Maintain mapping between the type of line, and the header corresponding to it
header_mapping = {"M, 1": ["Datetime", "Error Level", "DeviceId", "UserId", "Message"], 
    "M, 2":  ["Datetime", "Error Level", "DeviceId", "UserId", "MobileId", "Message"],
    "S, 1": ["Datetime", "Error Level", "DeviceId", "Action", "Message"],
    "S, 2": ["Datetime", "Error Level", "DeviceId", "IP", "Action", "Message"]
}
mapping = dict()

# Split the string into lines
lines = input_string.splitlines() 

for line in lines:
    split_line = line.split(", ") # Split each line using ", "
    key = split_line[0] + ", " + split_line[1] # First two elements of the split list form your key
    # Check if the key already exists. This is to ensure that our mapping dictionary contains only the first occurrence of each type.
    if not mapping.get(key, None):
        header = header_mapping[key]
        line_info = dict(zip(header, split_line[2:])) # Create dictionary with header-value mapping
        mapping[key] = line_info # Enter dictionary entry with type-values mapping

pprint(mapping)
"""
{'M, 1': {'Datetime': '14/08/2019 11:39',
          'DeviceId': 'xxxx',
          'Error Level': '4',
          'Message': '“Initialization of the system',
          'UserId': 'name'},
 'M, 2': {'Datetime': '14/08/2019 11:43',
          'DeviceId': 'yyy',
          'Error Level': '100',
          'Message': 'Message',
          'MobileId': '12345',
          'UserId': 'yura'},
 'S, 1': {'Action': 'name',
          'Datetime': '14/08/2019 11:40',
          'DeviceId': 'xxxx',
          'Error Level': '6',
          'Message': 'We created the user in the systems'}}

"""