简体   繁体   中英

Saving a dictionary of numpy arrays in human-readable format

This is not a duplicate question. I looked around a lot and found this question , but the savez and pickle utilities render the file unreadable by a human. I want to save it in a .txt file which can be loaded back into a python script. So I wanted to know whether there are some utilities in python which can facilitate this task and keep the written file readable by a human.

The dictionary of numpy arrays contains 2D arrays.

EDIT:
According to Craig's answer , I tried the following :

import numpy as np 

W = np.arange(10).reshape(2,5)
b = np.arange(12).reshape(3,4)
d = {'W':W, 'b':b}
with open('out.txt', 'w') as outfile:
    outfile.write(repr(d))

f = open('out.txt', 'r')
d = eval(f.readline())

print(d) 

This gave the following error: SyntaxError: unexpected EOF while parsing .
But the out.txt did contain the dictionary as expected. How can I load it correctly?

EDIT 2: Ran into a problem : Craig's answer truncates the array if the size is large. The out.txt shows first few elements, replaces the middle elements by ... and shows the last few elements.

Convert the dict to a string using repr() and write that to the text file.

import numpy as np

d = {'a':np.zeros(10), 'b':np.ones(10)}
with open('out.txt', 'w') as outfile:
    outfile.write(repr(d))

You can read it back in and convert to a dictionary with eval() :

import numpy as np

f = open('out.txt', 'r')
data = f.read()
data = data.replace('array', 'np.array')
d = eval(data)

Or, you can directly import array from numpy :

from numpy import array

f = open('out.txt', 'r')
data = f.read()
d = eval(data)

H/T: How can a string representation of a NumPy array be converted to a NumPy array?

Handling large arrays

By default, numpy summarizes arrays longer than 1000 elements. You can change this behavior by calling numpy.set_printoptions(threshold=S) where S is larger than the size of the arrays. For example:

import numpy as np 

W = np.arange(10).reshape(2,5)
b = np.arange(12).reshape(3,4)
d = {'W':W, 'b':b}

largest = max(np.prod(a.shape) for a in d.values()) #get the size of the largest array
np.set_printoptions(threshold=largest) #set threshold to largest to avoid summarizing

with open('out.txt', 'w') as outfile:
    outfile.write(repr(d))    

np.set_printoptions(threshold=1000) #recommended, but not necessary

H/T: Ellipses when converting list of numpy arrays to string in python 3

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM