在python中的特定行之后拆分文本文件

Question

I'm trying to write a code to read Fresco files and plot the results. Fresco produces one big file that looks something like this

theta  sigma
1        0.1
2        0.1
3        0.2
...
END
some text...
theta   sigma
1        0.3
2        0.2
...
END
more data...

I want to produce a new file after every 'END' to analyze the data separately. I tried some of the solutions proposed to other answers like

with open('fort.16', 'r') as infile, open('output_fort.16', 'w') as outfile:
copy= False
for line in infile:
    if line.strip() == '# legend':
        copy = True
        continue
    elif line.strip()=='End':
        copy = False
    elif copy:
        outfile.write(line)

but this is not what i need. Im fairly new to python so any help is much appreciated.

Answer 1

I managed to solve this with a nested generator:

import re

SECTION_START = re.compile(r'^\s*theta\s+sigma\s*$')
SECTION_END = re.compile(r'^\s*END\s*$')

def fresco_iter(stream):
    def inner(stream):
        # Yields each line until an end marker is found (or EOF)
        for line in stream:
            if line and not SECTION_END.match(line):
                yield line
                continue
            break

    # Find a start marker, then break off into a nested iterator
    for line in stream:
        if line:
            if SECTION_START.match(line):
                yield inner(stream)
            continue
        break

The fresco_iter method returns a generator that can be for-looped over. It returns 1 generator per section of theta sigma pairs.

>>> with open('fort.16', 'r') as fh:
...     print(list(fresco_iter(fh)))
[<generator object fresco_iter.<locals>.inner at 0x7fbc6da15678>,
 <generator object fresco_iter.<locals>.inner at 0x7fbc6da15570>]

So to make use of this, you create your own nested loop to process the nested generators.

filename = 'fort.16'

with open(filename, 'r') as fh:
    for nested_iter in fresco_iter(fh):
        print('--- start')
        for line in nested_iter:
            print(line.rstrip())
        print('--- end')

would output...

--- start
1        0.1
2        0.1
3        0.2
--- end
--- start
1        0.3
2        0.2
--- end

This strategy only ever holds 1 line of your input file in memory at a time, so would work for any size file, on even the smallest device... because generators are awesome.

So to take it all the way... separating the output into individual files:

with open(filename, 'r') as fh_in:
    for (i, nested_iter) in enumerate(fresco_iter(fh_in)):
        with open('{}.part-{:04d}'.format(filename, i), 'w') as fh_out:
            for line in nested_iter:
                fh_out.write(line)

Will output just the numbers to separate files named fort.16.part-0000 and fort.16.part-0001 .

I hope this helps, happy coding!

Answer 2

fp = open("random.txt")

data = []
temp = []

for i, line in enumerate(fp):
    if line.strip() == "END":
        new_file = open("file"+str(i)+".txt", "a+")
        for i in temp:
            new_file.write(i+"\n")
        temp = []
        new_file.close()
        continue
    temp.append(line.strip())

fp.close()
print(data)

Here you go this one, creates a new file everytime. The file name is file and the index of where ever the "END" line was found. :)