How to extract values of one field of csv file based on other fields?

Question

I have a csv file with 4 fields; student_id , date_of_exam , subject and marks . I want to store the values in marks field in some list based on every different student_id and subject so that I can perform some operation on that list later (ex: get the average marks etc).

I can do this if I have a student_id and subject beforehand; I can check them against all the values in the csv file and store marks corresponding to that particular student_id and subject (as shown in the code snippet below). But how do I do it for every student? This is the part I can't seem to figure out.

import csv

with open('results_file.csv', 'r') as f:
    reader = csv.reader(f)

    # next(reader)

    marks = []
    for line in reader:
        if line[0] == student_id and line[2] == subject:
            values.append(float(line[3]))
    print("Maximum: {}, Minimum: {}, Average: {}, Count: {}".format(max(values), min(values), sum(values) / len(values), len(values)))

The csv file looks something like this:

student_id,date_of_exam,subject,marks

a1,2012-05-21,Maths,45

a2,2012-05-24,Physics,48

a2,2012--5-27,Chemistry,42

a1,2012-05-15,Language,35

a2,2012-05-21,Maths,49

a3,2012-05-15,Language,47

Answer 1

You can use dictionaries:

grades_per_student = {}
grades_per_subject = {}

with open('results_file.csv', 'r') as f:
    reader = csv.reader(f)
    for line in reader:
        if line[0] in grades_per_student.keys():
            grades_per_student[line[0]].append(line[-1])
        else:
            grades_per_student[line[0]] = [line[-1]]
        if line[2] in grades_per_subject.keys():
            grades_per_subject[line[2]].append(line[-1])
        else:
            grades_per_subject[line[2]] = [line[-1]]

Result:

grades_per_student = {'a1': [45, 35], 'a2': [48, 42,49], 'a3': [47]}
grades_per_subjects = {'Maths': [45, 49], 'Physics': [48], 'Chemistry': [42], 'Language': [35, 47]}

Answer 2

You can use collections.defaultdict to store marks for every student/subject:

import csv
from collections import defaultdict

with open('out.csv', 'r') as f:
    reader = csv.reader(f)

    next(reader)    # skip header

    marks = defaultdict(list)
    grades = defaultdict(dict)
    subjects = set()
    for (student_id, date_of_exam, subject, mark) in reader:
        marks[student_id].append(int(mark))
        grades[student_id][subject] = int(mark)
        subjects.add(subject)

    subjects = sorted(subjects)

    print('{: ^10}{: ^10}{: ^10}{: ^10}{: ^5}'.format('student_id', 'maximum', 'minimum', 'average', 'count'))
    for student, marks in marks.items():
        print('{: ^10}{: ^10}{: ^10}{: ^10.2f}{: ^5}'.format(student, max(marks), min(marks), sum(marks) / len(marks), len(marks) ))

    print()

    print('{: ^15}'.format('student\subject'), end='')
    for s in subjects:
        print('{: ^15}'.format(s), end='')

    print()

    for student_id, student_subjects in grades.items():
        print('{: ^15}'.format(student_id), end='')
        for s in subjects:
            if s in student_subjects:
                print('{: ^15}'.format(student_subjects[s]), end='')
            else:
                print('{: ^15}'.format('-'), end='')
        print()

Prints:

student_id maximum   minimum   average  count
    a1        45        35      40.00     2  
    a2        49        42      46.33     3  
    a3        47        47      47.00     1  

student\subject   Chemistry      Language         Maths         Physics    
      a1              -             35             45              -       
      a2             42              -             49             48       
      a3              -             47              -              -       

Answer 3

I recommend that you use pandas library :

Read your data into a dataframe using pandas.read_csv function.
Passing the argument names , you can load only the columns of the csv that you want

import pandas as pd

df = pd.read_csv('results_file.csv', names=['student_id', 'subject', 'marks'])