How to convert text from a file into a list in Python?

Question

I am currently making a program for my school work that asks the user 3 sets of 10 randomly generated questions, and then stores the students results in one of three text files, depending on which class they are in. The program should then be able to load all the data from the text file, so that when the next student takes the quiz, their results are appended to the rest. It is stored in a list called "class1", "class2" or "class3", depending on what class the user is in. The data can then be sorted in different ways. In this post, we will assume the user is in Class 1.

The program tracks the users name, highest score, average score and their three attempts at the quiz and stores of it into the list "class1". Below is the list "class1".

[('Albert', 6, 2, [6, 0, 0]), ('Bob', 6, 2.33, [6, 1, 0]), ('Cameron', 5, 4, [5, 2, 5])].

If the program has never run before, then the program will simply create a new text file called "Class 1 Data.txt". If this file already exists, however, then this following section of code runs:

f = open("C:/Users/Custom/Desktop/Class " + str(classNumber) + " Data.txt", "r")
lines = f.readlines()
oldData = lines[3]

The variable "oldData" is now just a long string containing " [('Albert', 6, 2, [6, 0, 0])... ". Now lets assume the program is turned off and then loaded by another student called Zara, who then takes the quiz. Since "oldData" is just a string and not a list, when I try to append it back to the list "class1", this is what happens:

[('Zara', 9, 6, [3, 9, 6]), "[('Albert', 6, 2, [6, 0, 0]), ('Bob', 6, 2.33, [6, 1, 0]), ('Cameron', 5, 4, [5, 2, 5])]\n"]

How would I go about trying to load text from a file and then storing it as a list? I have tried using various methods, however none of them worked. Any help would be greatly appreciated.

Answer 1

It seems like you've just copied and pasted the textual representation (within Python) of your data into a text file, and now you want to take that text and convert it (from text) back into a Python data structure. This is an overly complicated way of doing that, I think. Parsers and the grammar involved in parsing are fairly complicated, and it would be very difficult for you to reproduce that. Plus I just don't think it's a valuable use of your time.

That leaves you with two choices:

1. If you actually want to serialize your python types to disk, I would strongly recommend trying to use the pickle library. This is a little complicated, but once you get the hang of it you should just be able to pickle and pickle objects of totally arbitrary complexity, rather than trying to parse them from plain old text.

2. If you don't do that, you should find a better, more predictable, more easily-parsed way of saving the data to text.

3. Within your program itself, you should create classes to more easily encapsulate your data - you have a list of tuples of strings and integers and lists of integers. It's a little much to walk through absent any object-oriented structure.

For example, if you were to use a different textual representation that's not tied to the way python types look:

name:Zara highscore:9 averagescore:6 attempt1:3 attempt2:9 attempt3:6
name:Albert highscore:6 averagescore:2 attempt1:6 attempt2:0 attempt3:0

Or if you were to use XML, you could save your document something like this:

<users>
    <user name="yrName" highscore="yrScore" averagescore="yrAverage" 
          attempt1="1" attempt2="2" attempt3="3">       
    </user>
    <user>
        ...
</users>

And you could use xml.etree.ElementTree to walk through the nodes and pick out each piece of information you needed.

I guess the biggest question about this, though, isn't why you're storing data the way you are but why you're storing a lot of it in the first place. 40% of your data - all high scores and average scores - has no reason at all to be stored. These figures are trivially calculated if you have access to the three attempts, and create so, so much more work for you than just using (one + two + three) / 3 or min([one, two, three]).