Checking a CSV for the existence of a similar value in Python

Question

Consider the following CSV:

date,description,amount
14/02/2020,march contract,-99.00
15/02/2020,april contract,340.00
16/02/2020,march contract,150.00
17/02/2020,april contract,-100.00

What I'd like to do is:

• Iterate through all of the rows
• Total the amount s of lines which have the same description
• Return the last line which has that newly-calculated amount

Applied to the above example, the CSV would look like this:

16/02/2020,march contract,51.00
17/02/2020,april contract,240.00

So far, I've tried nesting csv.reader() s inside of each other and I'm not getting the result I am wanting.

I'd like to achieve this without any libraries and/or modules.

Here is the code I have so far, where first_row is each row in the CSV and second_row is the iteration of looking for matching descriptions:

csv_reader = csv.reader(report_file)
        for first_row in csv_reader:
            description_index = 5
            amount_index = 13
            print(first_row)
            for second_row in csv_reader:
                if second_row is not first_row:
                    print(first_row[description_index] == second_row[description_index])
                        if first_row[description_index] == second_row[description_index]:
                            first_row[amount_index] = float(first_row[amount_index]) + float(second_row[amount_index])

Answer 1

This will work:

import csv
uniques = {}  # dictionary to store key/value pairs


with open(report_file, newline='') as f:
    reader = csv.reader(f, delimiter=',')
    next(reader, None)  # skip header row
    for data in reader:
        date = data[0]
        description = data[1]
        if description in uniques:
            cumulative_total = uniques[description][0]
            uniques[description] = [cumulative_total+float(data[2]), date]
        else:
            uniques[description] = [float(data[2]), date]

# print output
for desc, val in uniques.items():
    print(f'{val[0]}, {desc}, {val[1]}')

I know that you've asked for a solution without pandas, but you'll save yourself a lot of time if you use it:

df = pd.read_csv(report_file)

totals = df.groupby(df['description']).sum()
print(totals)

Answer 2

I suggest you should use pandas , it'll be efficient.

or if you still want to go with your way then this will help.

import csv

with open('mycsv.csv') as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    value_dict = {}
    line_no = 0
    for row in csv_reader:
        if line_no == 0:
            line_no += 1
            continue
        cur_date = row[0]
        cur_mon = row[1]
        cur_val = float(row[2])
        if row[1] not in value_dict.keys():
            value_dict[cur_mon] = [cur_date, cur_val]
        else:
            old_date, old_val = value_dict[cur_mon]
            value_dict[cur_mon] = [cur_date, (old_val + cur_val)]
        line_no += 1

for key, val_list in value_dict.items():
    print(f"{val_list[0]},{key},{val_list[1]}")

Output:

16/02/2020,march contract,51.0
17/02/2020,april contract,240.0

Mark this as answer if it helps you.

Answer 3

working with dictionary makes it easy to access values

import csv
from datetime import datetime

_dict = {}
with open("test.csv", "r") as f:
    reader = csv.reader(f, delimiter=",")

    for i, line in enumerate(reader):
        if i==0:
            headings = [line]
        else:
            if _dict.get(line[1],None) is None:
                _dict[line[1]] = {
                                    'date':line[0], 
                                    'amount':float(line[2])
                                 }
            else:
                if datetime.strptime(_dict.get(line[1]).get('date'),'%d/%m/%Y') < datetime.strptime(line[0],'%d/%m/%Y'):
                    _dict[line[1]]['date'] = line[0]

                _dict[line[1]]['amount'] = _dict[line[1]]['amount'] + float(line[2])

Here your _dict will contain unique description and values

>>> print(_dict)
{'march contract': {'date': '16/02/2020', 'amount': 51.0},  
'april contract': {'date': '17/02/2020', 'amount': 240.0}}

convert to list and add headings

headings.extend([[value['date'],key,value['amount']] for key,value in _dict.items()])

>>>print(headings)
[['date', 'description', 'amount'],['16/02/2020', 'march contract', 51.0], ['17/02/2020', 'april contract', 240.0]]

save list to csv

with open("out.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerows(headings)

Answer 4

You can also use itertools.groupby and sum() for this if you don't mind outputting in sorted form.

from datetime import datetime
from itertools import groupby
import csv

with open(report_file, 'r') as f:
    reader = csv.reader(f)
    lst = list(reader)[1:]

    sorted_input = sorted(lst, key=lambda x : (x[1], datetime.strptime(x[0],'%d/%m/%Y')))  #sort by description and date
    groups = groupby(sorted_input, key=lambda x : x[1])
    for k,g in groups:
        rows = list(g) 
        total = sum(float(row[2]) for row in rows)
        print(f'{rows[-1][0]},{k},{total}')  #print last date, description, total

Output:

17/02/2020,april contract,240.0
16/02/2020,march contract,51.0