Python 3: Pickling and UnPickling class instances returning “no persistent load” error

Question

I am trying to make a program that collects together lots of data about when certain Players in a band are available for busking this Christmas, and I'm struggling to get the pickle function to do what I want... The data is stored in class instances of the class below, Player :

import pickle

class Player():
    def __init__(self, name, instrument, availability):
        self.Name=name
        self.Instrument=instrument
        self.Availability=availability

The list of players, PlayerList is defined as an empty list at first, and I have defined a function, AddPlayer that will initialise a class instance with the player's details stored as attributes...

PlayerList=[]

def AddPlayer(PlayerList, name, instrument, availability):
    NewPlayer = Player(name, instrument, availability)
    PlayerList.append(NewPlayer)
    print("Player "+name+" has been added.\n\n")

I then have the function that stores the list of players when the user quits the program...

def StartProgram(PlayerList):
    while True:
        choice=input("Would you like to:\n1 Add a Player?\n2 Quit?\n")

        if choice=="1":                
        ## Adds the details of the Player using the above function
            AddPlayer(PlayerList, "Test Player", "Instrument", ["1st Dec AM"])
            StartProgram(PlayerList)

        elif choice=="2":
            file=open("BuskingList.txt", "wb")
            file=open("BuskingList.txt", "ab")            
            def AddToList(PlayerList):
                print("PlayerList: "+str(PlayerList))
                HalfPlayerList=PlayerList[:5]
                ## For some reason, pickle doesn't like me trying to dump a list with more than
                ## 5 values in it, any reason for that?

                for Player in HalfPlayerList:
                    print("Player: "+str(Player))
                    PlayerList.remove(Player)
                    ## Each player removed from original list so it's only added once.

                print("HalfPlayerList: "+str(HalfPlayerList))
                pickle.dump(HalfPlayerList, file)
                if len(PlayerList) !=0:
                    AddToList(PlayerList)
                    ## Recursive function call while there are still players not dumped
            AddToList(PlayerList)
            file.close()
            quit()

        else:
            print("Enter the number 1, 2, or 3.\n")
            StartProgram(PlayerList)

And last the function run at the start of the program to load all the player's information...

def Start():
    file=open("BuskingList.txt", "rb")
    print("File contains: "+str(file.readlines()))
    PlayerList=[]
    CheckForPlayers=file.read()
    if CheckForPlayers!="":
        file=open("BuskingList.txt", "rb")
        ListOfLists=[]
        for line in file:
            ToAppend=pickle.load(file)
            ListOfLists.append(ToAppend)
        for ListOfPlayers in ListOfLists:
            for Player in ListOfPlayers:
                PlayerList.append(Player)
        StartProgram(PlayerList)


print("When entering dates, enter in the form 'XXth Month AM/PM'\n")
Start()

When the program is first run (provided BuskingList.txt exists), the program is run fine, adding a name works and pickling it and dumping it on quitting apparently works. However, when the program is restarted, it fails to read the stored data with the error below...

File contains: [b'\x80\x03]q\x00c__main__\n', b'Player\n', b'q\x01)\x81q\x02}q\x03(X\x04\x00\x00\x00Nameq\x04X\x0b\x00\x00\x00Test Playerq\x05X\n', b'\x00\x00\x00Instrumentq\x06h\x06X\x0c\x00\x00\x00Availabilityq\x07]q\x08X\n', b'\x00\x00\x001st Dec AMq\tauba.']
Traceback (most recent call last):
  File "I:/Busking/Problem.py", line 63, in <module>
    Start()
  File "I:/Busking/Problem.py", line 54, in Start
    ToAppend=pickle.load(file)
_pickle.UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.

I have done a bit of research and found that this persistent id malarkey shouldn't be an issue, so why has it come up here? Also, why the five value limit on lists when pickling? Any help would be appreciated.

Answer 1

You first read the list with .readlines() :

print("File contains: "+str(file.readlines()))

then try to read it again:

CheckForPlayers=file.read()

This won't work; the file pointer is now at the end of the file. Rewind or reopen the file:

file.seek(0)  # rewind to start

Not that you need to check for the file contents here; let pickle do that for you.

Next you read the file line by line:

for line in file:
    ToAppend=pickle.load(file)

This doesn't work; the pickle format is binary , not line oriented, and you are reading by using iteration, then reading again by passing in the file object.

Leave reading the file to the pickle module altogether:

with open("BuskingList.txt", "rb") as infile:
    ToAppend=pickle.load(file)

You also mention in your code:

## For some reason, pickle doesn't like me trying to dump a list with more than
## 5 values in it, any reason for that?

Pickle has no problems with any list size, there is no reason to break up your players list into chunks of five. You didn't state what problem you encountered, but the number of items in the list cannot have been the cause:

>>> import pickle
>>> with open('/tmp/test.pickle', 'wb') as testfile:
...     pickle.dump(range(1000), testfile)
... 
>>> with open('/tmp/test.pickle', 'rb') as testfile:
...     print len(pickle.load(testfile))
... 
1000

This stored and re-loaded a list of 1000 integers.