readlines from file and convert to numpy

Question

I'm trying to read a file and convert it to a N by 5 array. I can't use loadtxt because the file is too complex. I have to read line by line; however, I want the format to be just like that obtained when using np.loadtxt; Eg

[[ 8.00000e+00  4.00000e+00  3.82920e+00 -3.71733e-02  2.14022e+01]
 [ 5.00000e+00  4.00000e+00  9.14763e+00  3.06177e+00  2.12681e+01]
 [ 3.00000e+00  4.00000e+00  2.49941e+01  2.73711e+01  3.07871e+00]
 [ 1.00000e+01  4.00000e+00  2.47697e+01  3.04612e+01  7.01204e+00]
 [ 1.50000e+01  4.00000e+00  2.18943e+01  4.54100e+01  1.86717e+01]]

I'm doing the following:

   f = open(filename, 'r+')
   lines = f.readlines()
   f.close()
   Rlines = list(reversed(lines)) # Need to read the file backwards
   yarray = []
   skip = 0
   for i in range(0, len(Rlines)):
      if skip:
         skip -=1
      elif "ITEM" in Rlines[i]:  # Need to skip 8 lines every occurrence of ITEM
         skip = 8
      else:
         yvalue = Rlines[i].split()
         yarray.append(yvalue)
   yarray = np.array(yarray)
   return yarray

and I get:

[array(['15', '4', '-14.339', '11.4973', '37.7431'], dtype='<U7'),
 array(['7', '4', '65.9747', '-5.05116', '-36.7654'], dtype='<U8'),
 array(['10', '4', '13.7083', '-40.7658', '-6.79697'], dtype='<U8'),
 array(['3', '4', '5.10123', '51.4941', '-93.8115'], dtype='<U8'),
 array(['16', '4', '-17.8979', '-1.56454', '-22.2222'], dtype='<U8'),
 array(['4', '4', '10.9422', '3.34997', '-5.03861'], dtype='<U8')]

Thanks for any help.

Answer 1

Use

yvalue = np.fromstring(Rlines[i], sep=' ')

instead of

yvalue = Rlines[i].split()

to convert lines into numpy arrays.

https://numpy.org/doc/stable/reference/generated/numpy.fromstring.html