将文本文件中的数字数组读入numpy数组-python

Question

假设我有一个看起来像这样的文件

text a

bla bla

1 2 3   
4 5 6

text b

bla

7 8 9
10 11 12

text c

bla bla bla

13 14 15
16 17 18

我试图仅提取数字数组并将其放入numpy数组中：

array([[ 1, 2, 3,
         4, 5, 6,],
       [ 7, 8, 9,
         10, 11, 12],
       [ 13, 14, 15,
         16, 17, 18]])

我尝试使用np.genfromtxt('test.txt',usecols=[0,1,2],invalid_raise=False)

array([[  1.,   2.,   3.],
       [  4.,   5.,   6.],
       [  7.,   8.,   9.],
       [ 10.,  11.,  12.],
       [ nan,  nan,  nan],
       [ 13.,  14.,  15.],
       [ 16.,  17.,  18.]])

但是它不会创建子数组并将文本转换为nans 。 有更好的方法吗？

Answer 1

您可以按照以下方式使用itertools.groupby

>>> import itertools
>>> import numpy as np
>>> 
>>> content = """text a
... 
... bla bla
... 
... 1 2 3   
... 4 5 6
... 
... text b
... 
... bla
... 
... 7 8 9
... 10 11 12
... 
... text c
... 
... bla bla bla
... 
... 13 14 15
... 16 17 18"""
>>> 
>>> import io
>>> filelike = io.StringIO(content)

# you may want to refine this test
>>> allowed_characters = set('0123456789 ')
>>> def isnumeric(line):
...     return set() < set(line.strip()) <= allowed_characters
... 
>>> [np.genfromtxt(gr) for k, gr in itertools.groupby(filelike, isnumeric) if k]
[array([[1., 2., 3.],
       [4., 5., 6.]]), array([[ 7.,  8.,  9.],
       [10., 11., 12.]]), array([[13., 14., 15.],
       [16., 17., 18.]])]

Answer 2

您可能不得不诉诸“手动”解析。 假设这里给出的是一个解决方案（肯定还有其他解决方案）：

import numpy as np

def parser(fname):
    with open(fname) as fh:
        for i, line in enumerate(fh):
            p = i % 7
            if p not in (5, 6):
                continue
            yield line.rstrip()

a = ' '.join(parser(filename))
arr = np.fromstring(a, dtype=int, sep=' ')
arr = arr.reshape((-1, 6))
print(arr)