data accumulating from csv file using python

Question

---

out_gate,useless_column,in_gate,num_connect
a,u,b,1
a,s,b,3
b,e,a,2
b,l,c,4
c,e,a,5
c,s,b,5
c,s,b,3
c,c,a,4
d,o,c,2
d,l,c,3
d,u,a,1
d,m,b,2

shown above is a given, sample csv file. First of all, My final goal is to get the answer as a form of csv file like below:

 ,a,b,c,d 
a,0,4,0,0 
b,2,0,4,0 
c,9,8,0,0 
d,1,2,5,0

I am trying to match this each data (a,b,c,d) one by one to the in_gate so, for example when out_gate 'c'-> in_gate 'b', number of connections is 8 and 'c'->'a' becomes 9.

I want to solve it with lists(or tuple, Dictionary, set) or collections. defaultdict WITHOUT USING PANDAS OR NUMPY, and I want a solution that can be applied to many gates (around 10 to 40) as well.

I understand there is a similar question and It helped a lot, but I still have some troubles in compiling. Lastly, Is there any way with using lists of columns and for loop?

((ex) list1=[a,b,c,d],list2=[b,b,a,c,a,b,b,a,c,c,a,b])

what if there are some useless columns that are not related to the data but the final goal remains same?

thanks

Answer 1

I'd use a Counter for this task. To keep the code simple, I'll read the data from a string. And I'll let you figure out how to produce the output as a CSV file in the format of your choice.

import csv
from collections import Counter

data = '''\
out_gate,in_gate,num_connect
a,b,1
a,b,3
b,a,2
b,c,4
c,a,5
c,b,5
c,b,3
c,a,4
d,c,2
d,c,3
d,a,1
d,b,2
'''.splitlines()

reader = csv.reader(data)
#skip header
next(reader)
# A Counter to accumulate the data
counts = Counter()

# Accumulate the data
for ogate, igate, num in reader:
    counts[ogate, igate] += int(num)

# We could grab the keys from the data, but it's easier to hard-code them
keys = 'abcd'

# Display the accumulated data
for ogate in keys:
    print(ogate, [counts[ogate, igate] for igate in keys])

output

a [0, 4, 0, 0]
b [2, 0, 4, 0]
c [9, 8, 0, 0]
d [1, 2, 5, 0]

Answer 2

If I understand your problem correctly, you could try and using a nested collections.defaultdict for this:

import csv
from collections import defaultdict

d = defaultdict(lambda : defaultdict(int))

with open('gates.csv') as in_file:
    csv_reader = csv.reader(in_file)
    next(csv_reader)
    for row in csv_reader:
        outs, ins, connect = row
        d[outs][ins] += int(connect)

gates = sorted(d)
for outs in gates:
    print(outs, [d[outs][ins] for ins in gates])

Which Outputs:

a [0, 4, 0, 0]
b [2, 0, 4, 0]
c [9, 8, 0, 0]
d [1, 2, 5, 0]