Searching a .CSV file

Question

I've been trying to create a program in python whereby you can access, search and input(append) the data in a CSV file. I am able to append and print the file, however I am having trouble searching the external file.

This is what my code looks like so far:

def a():
    # knows to use the file (getting something that's made)
    import csv

    myfile = open("student01.csv", "a")

    student_num = input("Enter the student number: ")
    name = input("Enter student's name: ")
    tutor_group = input("Enter the tutor group: ")
    gender = input("Enter M or F: ")

    new_record =student_num+","+name+","+tutor_group+","+gender

    myfile.write(str(new_record))
    myfile.write("\n")

    myfile.close()

def d():
    # knows to use the file (getting something that's made)
    import csv

    myfile = open("student01.csv", "rb")
    reader = csv.reader(myfile)
    for row in myfile:
        print(row)

    myfile.close()

def menu():
    print("Welcome to Student.csv\nWhat would you like to do?")
    print()
    print("1. Add to the file")
    print("2. Display all the data from the file")
    print("3. Search for particular data")
    enter = input("Enter 1, 2 or 3: ")
    enter = enter.upper()
    if enter == "1":
        a()
    elif enter == "2":
        d()
   else:
        s()

    menu()

I'm just not sure what to do now for my def s(): so that I am able to search for a student in the file by their name, number or tutor group.

Does anyone have any ideas?

Lotz

Answer 1

There is a DictReader in the csv module, perfect for what you want to do (assuming some fields are unique, like student_num probably)

just for info I see some problems in your code

1. don't import module in function if not necessary, and don't repeat yourself(DRY), just put one import statement at the top outside any function, and if you import something, make use of it ;)
2. use function's parameters instead of harcoding literals constant inside the body for file name
3. this : student_num+","+name+","+tutor_group+","+gender should be removed, use ', '.join(your_iterable) instead
4. when you want to open a file and do work on it before close it, use a context manager : `with open('myfile.txt') as flux:', it defines a block where the file is opened, the file will be closed automatically, even if a problem occurs and an exception is raised

Answer 2

import csv

def create_record(number_of_students):

    while number_of_students:
        student_num = raw_input("Enter the student number: ")
        name = raw_input("Enter student's name: ")
        tutor_group = raw_input("Enter the tutor group: ")
        gender = raw_input("Enter M or F: ")

        if student_num and name and tutor_group and gender:
            record_list = [student_num, name, tutor_group, gender]

            with open("student_record.csv", "ab") as wf:
                 writer = csv.writer(wf)
                 writer.writerow(record_list)
        number_of_students -= 1

def display_record(option):

    with open("student_record.csv", "r") as rf:
        reader = csv.reader(rf)
        if option == 2:
            for row in reader:
                print "  ".join(row)
        elif option == 3:
            search_feild = raw_input("Search by student name, number, tutor_group, gender :")
            for row in reader:
                if search_feild in row:
                    print "  ".join(row)


def main():
    print("1. Add to the file")
    print("2. Display all the data from the file")
    print("3. Search for particular data")
    print("0. To Exit")

    choice = True
    while choice:
        your_choice = input("Enter your choice:")
        if your_choice == 1:
            number_of_students = input("Enter number of records you want to enter:")    
            create_record(number_of_students)
        if your_choice == 2:
            display_record(2)
        if your_choice == 3:
            display_record(3)
        if your_choice == 0:
            choice = False

if __name__ == "__main__":
    main()    

Output:
1. Add to the file
2. Display all the data from the file
3. Search for particular data
0. To Exit
Enter your choice:1
Enter number of records you want to enter:3
Enter the student number: 1
Enter student's name: Jhon
Enter the tutor group: Python
Enter M or F: M
Enter the student number: 2
Enter student's name: Lina
Enter the tutor group: Django
Enter M or F: F
Enter the student number: 3
Enter student's name: Max
Enter the tutor group: Python
Enter M or F: M
Enter your choice:2
1  Jhon  Python  M
2  Lina  Django  F
3  Max  Python  M
Enter your choice:3
Search by student name, number, tutor_group, gender :Python
1  Jhon  Python  M
3  Max  Python  M
Enter your choice:3
Search by student name, number, tutor_group, gender :F
2  Lina  Django  F
Enter your choice:0

You should even write your data in csv file using csv writer method.

Searching is easy as while reading a csv it gives a list in which we can search any element by using in keyword.

Answer 3

Searching through a set of records is not a trivial problem. I hope this won't make your like super complicated but you'll need a couple of things:

1. Indices for each field you would like to search on (a data structure that copies information for fast lookup)
2. A clever way of guessing whether the user is entering a name, tutor group or number (or just asking them what they want to search by)

You can make your own stuff here: maybe start by compiling the users' input into a regular expression and search subsets of the CSV file (like the name column) by doing binary comparisons between the entry and your user's query.

You can also make an index: just pick a data type (or make your own of course) and make sure it's super efficient for checking through itself in a reasonable amount of time for the number of records you have (as low a time complexity as possible).

Then again, a lot of people have been working on this problem for a long time and have TONS of code out there to help you. Check out Information Retrieval and get a feel for the type of problem you are looking at.

You can use something like Xapian for search but there are a lot of alternatives.

Here are some SO questions that might help you: the one without a chosen answer , the one that uses the any function, and the one about getting the right look at columns