read text file in block and convert to numpy array

Question

I'm trying to read a text file with tag and turn then into numpy array. my data look likes this

<header1>
0, 4, 7, 9
1, 2, 6, 8
2, 5, 7, 0
<header1>
.
.
.
<header2>
0, 6, 2, 10, 10, 8
1, 22, 56, 18, 7, 9
2, 57, 79, 09, 10, 1
<header2>

I wrote something like this to extract data within headers

def read(infile):
     with open('infile') as fp:
            for line in re.findall('<header1>(.*?)<header1>', fp.read(), re.S):
                print(line)

my question is how do I covert what I have to a numpy array that looks like this:

[[ 0 4 7 9]
 [1 2 6 8 ]
 [2 5 7 0 ]]

Answer 1

Not the best optimized way, but you can try something like this

def read(infile):
     with open(infile) as fp:
            for line in re.findall('<header[0-9]>(.*?)<header[0-9]>', fp.read(), re.S):
                print(np.array([i.strip().split(', ') for i in line.split('\n') if i]) )

[['0' '4' '7' '9']
 ['1' '2' '6' '8']
 ['2' '5' '7' '0']]
[['0' '6' '2' '10' '10' '8']
 ['1' '22' '56' '18' '7' '9']
 ['2' '57' '79' '09' '10' '1']]

Answer 2

If you can collect blocks of lines between 'headers', you can use the standard csv reader to create an array:

In [569]: txt="""0, 4, 7, 9
     ...: 1, 2, 6, 8
     ...: 2, 5, 7, 0""".splitlines()
In [570]: txt
Out[570]: ['0, 4, 7, 9', '1, 2, 6, 8', '2, 5, 7, 0']
In [571]: data = np.genfromtxt(txt, delimiter=',', dtype=int)
In [572]: data
Out[572]: 
array([[0, 4, 7, 9],
       [1, 2, 6, 8],
       [2, 5, 7, 0]])

genfromtxt has more bells-and-whistles than this case needs, but it's easy to use. Here I'm just giving it a list of strings.

Or simply break the lines into strings that can be converted ints:

In [586]: [s.split(',') for s in txt]
Out[586]: [['0', ' 4', ' 7', ' 9'], ['1', ' 2', ' 6', ' 8'], ['2', ' 5', ' 7', ' 0']]
In [587]: np.array([s.split(',') for s in txt],int)
Out[587]: 
array([[0, 4, 7, 9],
       [1, 2, 6, 8],
       [2, 5, 7, 0]])