How to read a txt file containing numpy.ndarray in python

Question

I wonder what is the right syntax to read the following test.txt file that contains the following values:

(ps test.txt has the type of numpy.ndarray)

[0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.
 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.
 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.
 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.
 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.51  0.47  0.45
 0.42  0.42  0.4   0.385 0.375 0.41  0.415 0.375 0.355 0.36  0.41  0.4
 0.39  0.38  0.375 0.375 0.375 0.38  0.39  0.395 0.385 0.38  0.375 0.375
 0.37  0.365 0.36  0.355 0.35  0.35  0.345 0.345 0.35  0.36  0.355 0.355
 0.35  0.35  0.355 0.355 0.35  0.35  0.35  0.345 0.34  0.335 0.325 0.325
 0.325 0.33  0.345 0.325 0.32  0.315 0.315 0.315 0.31  0.31  0.31  0.305
 0.305 0.3   0.3   0.29  0.29  0.3   0.295 0.29  0.29  0.29  0.29  0.29]

I've tried to read the file using the following code:

data_test = np.genfromtxt('test.txt')

But I'm getting error massage saying:

ValueError: Some errors were detected !
    Line #43 (got 8 columns instead of 12)

Any help on how to read this kind of data that is separated by space/columns would be appreciated a lot!

Answer 1

use numpy.fromstring

with open('test.txt') as file:
    data = file.read()
data = data.replace('\n', '')
arr = np.fromstring(data[1:-1], sep=' ', dtype=np.float32)

Answer 2

Since the file can be viewed as a bunch of floats embedded in non-decimal junk, a regular expression can pull them out. Just find all of the substrings consisting of decimals and the period.

>>> import numpy as np
>>> import re
>>> with open('foo.txt') as fileobj:
...     arr = np.array([float(val) for val in re.findall(r"[\d\.]+",
...             fileobj.read(), flags=re.MULTILINE)])
... 
>>> arr
array([0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   , 0.   ,
       0.   , 0.   , 0.   , 0.51 , 0.47 , 0.45 , 0.42 , 0.42 , 0.4  ,
       0.385, 0.375, 0.41 , 0.415, 0.375, 0.355, 0.36 , 0.41 , 0.4  ,
       0.39 , 0.38 , 0.375, 0.375, 0.375, 0.38 , 0.39 , 0.395, 0.385,
       0.38 , 0.375, 0.375, 0.37 , 0.365, 0.36 , 0.355, 0.35 , 0.35 ,
       0.345, 0.345, 0.35 , 0.36 , 0.355, 0.355, 0.35 , 0.35 , 0.355,
       0.355, 0.35 , 0.35 , 0.35 , 0.345, 0.34 , 0.335, 0.325, 0.325,
       0.325, 0.33 , 0.345, 0.325, 0.32 , 0.315, 0.315, 0.315, 0.31 ,
       0.31 , 0.31 , 0.305, 0.305, 0.3  , 0.3  , 0.29 , 0.29 , 0.3  ,
       0.295, 0.29 , 0.29 , 0.29 , 0.29 , 0.29 ])