Cannot load file containing pickled data - Python .npy I/O

Question

I am trying to save a dataframe and a matrix as.npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 1

I am trying to save a dataframe and a matrix as .npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 2

I am trying to save a dataframe and a matrix as .npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 3

I am trying to save a dataframe and a matrix as .npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 4

I am trying to save a dataframe and a matrix as .npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 5

I am trying to save a dataframe and a matrix as .npy files with np.save() and then read them using np.load() but I get the following error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 457, in load
    raise ValueError("Cannot load file containing pickled data "

ValueError: Cannot load file containing pickled data when allow_pickle=False

Even if I write allow_pickle=True I get an error:

  File "/Users/sofiafarina/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 463, in load
    "Failed to interpret file %s as a pickle" % repr(file))

OSError: Failed to interpret file 'finaldf_p_85_12.npy' as a pickle

So how could I save a df from a python script and then load it in another one? Should I use other functions? Thank you!

Answer 6

只要确保文件没有损坏。

Answer 7

I had the same issue. Try np.loadtxt instead.

Answer 8

I was dealing with the problem long time. I have tried all of the solutions which are listed here however they all didn't work. I have tried different versions of python such as 3.7, 2.7, 3.9 and the result was same.

Finally I have noticed that the file with the extension .npy is corrupted so it gives out this error. Here is the line giving the error.

npyFile = np.load('file1.npy')

So whoever come accross the same problem first of all it would be better to check the .npy file.

Answer 9

I uploaded my documents to drive and I uploaded the documents from the drive. It is solved.

from google.colab import drive
drive.mount("/content/drive")
label = np.load("path/labels.npy") 

Answer 10

my hypothesis is when I am picking npz file, some other task is writing in that npz and in that case np.load(file_path,mmap_mode='r') gives 'Cannot load file containing pickled data when allow_pickle=False'.

So reading it after some fixed it.