how to delete the annoying data in the csv file

Question

i want to delete some string("Description" "This is a simulation") in my csv file, and also i want to delete some"=" in the data and ", " at the end of the data. the file looks like the following

"time","student","items"

="09:00:00","Tim","apple",

="09:00:10","Jason","orange",

"09:10:10","Emily","grape",

"09:22:10","Ivy","kiwi",

"Description"

"This is a simulation"

i have tried .pop(). it didn't work

ff= []

import csv

with open('file.csv') as f:

    for row in csv.DictReader(f):

        row.replace(',','')

        ff.append(row)

i want to get like this:

"time","student","items"

"09:00:00","Tim","apple"

"09:00:10","Jason","orange"

"09:10:10","Emily","grape"

"09:22:10","Ivy","kiwi"

Answer 1

You probably want to read the file as raw text file rather than csv so that it will be easier for you to perform string manipulation with it.

Edit: I assume that tmp is the path to the CSV file and the <list data> is a list of dictionary generated by csv.DictReader . Then you can write the convert(tmp) by performing 2 main steps. One is to reformatted the file and it to a temporary file and the other is to read the temporary file into a list of dictionary data using csv.DictReader . After you're done reading the data, the temporary file will be deleted using the os module:

import csv
import os

def convert(tmp):
    new_lines = []
    temp_file = tmp + '.tmp'
    with open(tmp) as fd:
        for line in fd:
            # remove new line characters
            line = line.replace('\n', '').replace('\r', '')

            # delete string
            line = line.replace('=', '').replace('"Description"', '').replace('"This is a simulation"', '')

            # don't add empty string
            if line.strip() == '':
                continue

            # remove last line commas
            if line[-1] == ',':
                line = line[:-1]

            new_lines.append(line)

    # write formatted data to temporary csv file
    with open(temp_file, 'w') as fd:
        fd.write('\n'.join(new_lines))

    # get list data
    ff = None
    with open(temp_file) as f:
        ff = list(csv.DictReader(f))

    # delete temporary file
    os.remove(temp_file)

    return ff

print convert('./file.csv')

Answer 2

Mostly leveraging built-in str methods, with an assumption that the first row is always a valid header row.

ff = []

with open('file.csv') as f:

    for row in f:
        # strip empty lines, and head/tail = ,
        line = row.strip().strip('=').strip(',')

        # skip empty lines
        if not line:
            continue

        # assume first row is always a valid header row
        # split by comma to see if it matches header row
        if not len(ff) or (len(line.split(',')) == len(ff[0].split(','))):
            ff.append(line)