Easiest way to export dictionary (or current output text file) to .csv file instead of a .txt file?

Question

I created a parser and extractor for a log file and wanted to see an example of a quick way to either:

1. Take current output written onto a .txt file and convert it into a new .csv file (possibly with pandas ), or

2. Use the .csv module to change the write method sequence into a csv.writer and then using csv.DictReader .

What is most efficient in terms of practicality and resource consumption? My current exported .txt file and relevant code is posted below.

Exported data:

Request ID : bf710010
Username   : kadaniel
ECID       : 6ca4862b-14d1-4a7f-8158-5e6cac363144-001477ac
Start Time : 2019-06-12T09:14:54.947
End Time   : 2019-06-12T09:14:55.22

Request ID : bf710020
Username   : kadaniel
ECID       : 6ca4862b-14d1-4a7f-8158-5e6cac363144-001477ac
Start Time : 2019-06-12T09:14:55.343
End Time   : 2019-06-12T09:14:55.514

Code:

process_records = {}

with open(log_file_path, "r") as file:

    for line in file:
        m = pattern.match(line)
        if m is not None:        # If there is a match with pattern
            (timestamp, ecid, requestid, username) = m.groups()
            if requestid not in process_records:
                process_records[requestid] = (timestamp, username, ecid, None)
            else:
                process_records[requestid] = process_records[requestid][:3] + (timestamp,)

    for requestid, (start, username, ecid, end) in process_records.items():
        print("Request ID: {}\nUsername: {}\nECID: {}\nStart Time: {}\nEnd Time: {}\n\n".format(
                requestid,
                username,
                ecid,
                start,
                end,
            ))

file.close()

with open(export_file, 'w+') as file:

    file.write("EXPORTED DATA:\n\n")

    if pattern != None:
        for requestid, (start, username, ecid, end) in process_records.items():
                file.write(("Request ID : {}\nUsername   : {}\nECID       : {}\nStart Time : {}\nEnd Time   : {}\n\n".format(
                    requestid,
                    username,
                    ecid,
                    start,
                    end,
                )))

file.close()

I currently have the data in a dictionary, process_records . Each key ( requestid ) is associated with 4 elements in a tuple. I want the key and each element thereafter to represent its own column.

Answer 1

The ideal way to do this in my opinion would be to use the builtin csv library.

First, import the library.

import csv

Then use the following snippet for writing -

with open(export_file, 'w+') as file_handler:
    csv_writer = csv.writer(fileobj=file_handler, delimiter=',')
    for requestid, (start, username, ecid, end) in process_records.items():
        csv_writer.writerow([requestid, username, ecid, start, end,])

Answer 2

CSV is coma separated. You do not need pandas for this. Only change rule o writing record

with open(export_file, 'w+') as file:

    file.write("Request ID,Username,ECID,Start Time,End Time\n") # header

    if pattern != None:

        for requestid, (start, username, ecid, end) in process_records.items():
                file.write(("{},{},{},{},{}\n".format(
                    requestid,
                    username,
                    ecid,
                    start,
                    end,
                )))  # record

file.close()