how to convert txt file into json using python?

Question

I'm writing a log file using python and the log file is like this :

MessageName:mouse left down | TimeStamp:2019-02-13 15:43:31.664 |  Window:13500784 | Position:(483, 587) | Wheel:0
MessageName:mouse left up | TimeStamp:2019-02-13 15:43:31.873 | Window:13500784 | Position:(483, 587) | Wheel:0

I would like to convert this log into json format.

This is the code that i have tried :

import json

def convert() :
f = open("log_events.log", "r")
content = f.read()
splitcontent = content.splitlines()

for line in splitcontent :
    pipesplit = line.split(' | ')
    print(pipesplit)
    with open("json_log.json", 'a') as fout:
        json.dump(pipesplit, fout, indent=4)

And the Output i need is like this :

[
{
    "MessageName" : "mouse left down",
    "TimeStamp" : "2019-02-13 15:43:31.664",
    "Window" : "13500784",
    "Position" : "(483, 587)",
    "Wheel" : 0"
},
{
    "MessageName" : "mouse left up",
    "TimeStamp" : "2019-02-13 15:43:31.873",
    "Window" : "13500784",
    "Position" : "(483, 587)",
    "Wheel" : "0"
},

]

But with the above given code, my output is

[
    "MessageName" : "mouse left down",
    "TimeStamp" : "2019-02-13 15:43:31.664",
    "Window" : "13500784",
    "Position" : "(483, 587)",
    "Wheel" : 0"
][
    "MessageName" : "mouse left up",
    "TimeStamp" : "2019-02-13 15:43:31.873",
    "Window" : "13500784",
    "Position" : "(483, 587)",
    "Wheel" : "0"
]

How to convert into proper JSON format ?

Answer 1

You could use simple split ting and parsing like the below :)

$ cat file.txt
MessageName:mouse left down | TimeStamp:2019-02-13 15:43:31.664 |  Window:13500784 | Position:(483, 587) | Wheel:0
MessageName:mouse left up | TimeStamp:2019-02-13 15:43:31.873 | Window:13500784 | Position:(483, 587) | Wheel:0

# cat mkdict.py
import json

with open('file.txt') as file, open('dump.json', 'w') as json_file:
    items = []
    for line in file:
        if not line.strip():
            continue
        d = {}
        data = line.split('|')
        for val in data:
            key, sep, value = val.partition(':')
            d[key.strip()] = value.strip()
        items.append(d)
    json.dump(items, json_file)

print(items)


$ python mkdict.py
[{'Wheel': '0', 'TimeStamp': '2019-02-13 15:43:31.664', 'Window': '13500784', 'Position': '(483, 587)', 'MessageName': 'mouse left down'}, {'Wheel': '0', 'TimeStamp': '2019-02-13 15:43:31.873', 'Window': '13500784', 'Position': '(483, 587)', 'MessageName': 'mouse left up'}]

$ cat dump.json
[{"Wheel": "0", "TimeStamp": "2019-02-13 15:43:31.664", "Window": "13500784", "Position": "(483, 587)", "MessageName": "mouse left down"}, {"Wheel": "0", "TimeStamp": "2019-02-13 15:43:31.873", "Window": "13500784", "Position": "(483, 587)", "MessageName": "mouse left up"}]

Answer 2

You need to add extra handling for each line, to convert the string into a key:value pair, then add that pair to a dictionary.

Also, you only need to open the JSON file once and write the entire data structure:

import json

def line_to_dict(split_Line):
    # Assumes that the first ':' in a line
    # is always the key:value separator

    line_dict = {}
    for part in split_Line:
        key, value = part.split(":", maxsplit=1)
        line_dict[key] = value

    return line_dict

def convert() :
    f = open("log_events.log", "r")
    content = f.read()
    splitcontent = content.splitlines()

    # Split each line by pipe
    lines = [line.split(' | ') for line in splitcontent]

    # Convert each line to dict
    lines = [line_to_dict(l) for l in lines]

    # Output JSON 
    with open("json_log.json", 'w') as fout:
        json.dump(lines, fout, indent=4)

Answer 3

The following code should work with use of list and dictionary

import json

logfile = open('log_events.log')

#initialising a list to append all the log lines formatted as json
log_list = []

for line in logfile:
    # splitting on '|'
    pipe_split = [ele.strip() for ele in line.split("|")]

    # initialising dictionary to fill the line splitted data in key-value pairs
    line_dict = dict()

    for ele in pipe_split:
        # splitting on first occurrence of ':' 
        key,val = ele.split(":",1)
        line_dict[key] = val

    # appending the key-value data of each line to a list
    log_list.append(line_dict)

with open('json_log.json','w') as f:
    json.dump(log_list,f,indent=4)

Gives output to the file in following format,

[
{
    "MessageName": "mouse left down",
    "TimeStamp": "2019-02-13 15:43:31.664",
    "Window": "13500784",
    "Position": "(483, 587)",
    "Wheel": "0"
},
{
    "MessageName": "mouse left up",
    "TimeStamp": "2019-02-13 15:43:31.873",
    "Window": "13500784",
    "Position": "(483, 587)",
    "Wheel": "0"
}
]

Answer 4

You where mostly correct. What you didn't do is generate dict s.

1. split on the first occurrence of :
2. add each dict to a list
3. dump them into your .json .

The magic is in this line:

d.append(dict(s.split(':',1) for s in l))

In words: Split each element on the first occurrence of : for every element in l then make them to a dict and append them to the list d

Working example:

import json

f = open("log.file", "r")
content = f.read()
splitcontent = content.splitlines()

d = []
for v in splitcontent:
    l = v.split(' | ')
    d.append(dict(s.split(':',1) for s in l))


with open("json_log.json", 'w') as file:
    file.write((json.dumps(d, indent=4, sort_keys= False)))

Output:

[
    {
        "MessageName": "mouse left down",
        "TimeStamp": "2019-02-13 15:43:31.664",
        " Window": "13500784",
        "Position": "(483, 587)",
        "Wheel": "0"
    },
    {
        "MessageName": "mouse left up",
        "TimeStamp": "2019-02-13 15:43:31.873",
        "Window": "13500784",
        "Position": "(483, 587)",
        "Wheel": "0"
    }
]