如何在python中按ID分割文本文件

Question

I have a bunch of text files containing tab separated tables. The second column contains an id number, and each file is already sorted by that id number. I want to separate each file into multiple files by the id number in column 2. Here's what I have.

readpath = 'path-to-read-file'
writepath = 'path-to-write-file'
for filename in os.listdir(readpath):
     with open(readpath+filename, 'r') as fh:
          lines = fh.readlines()
     lastid = 0
     f = open(writepath+'checkme.txt', 'w')
     f.write(filename)
     for line in lines:
          thisid = line.split("\t")[1]
          if int(thisid) <> lastid:
               f.close()
               f = open(writepath+thisid+'-'+filename,'w')
               lastid = int(thisid)
          f.write(line)
     f.close()

What I get is simply a copy of all the read files with the first id number from each file in front of the new filenames. It is as if

thisid = line.split("\t")[1]

is only done once in the loop. Any clue to what is going on?

EDIT

The problem was my files used \\r rather than \\r\\n to terminate lines. Corrected code (simply adding 'rU' when opening the read file and swapping != for <>):

readpath = 'path-to-read-file'
writepath = 'path-to-write-file'
for filename in os.listdir(readpath):
     with open(readpath+filename, 'rU') as fh:
          lines = fh.readlines()
     lastid = 0
     f = open(writepath+'checkme.txt', 'w')
     f.write(filename)
     for line in lines:
          thisid = line.split("\t")[1]
          if int(thisid) != lastid:
               f.close()
               f = open(writepath+thisid+'-'+filename,'w')
               lastid = int(thisid)
          f.write(line)
     f.close()

Answer 1

If you're dealing with tab delimited files, then you can use the csv module, and take advantage of the fact that itertools.groupby will do the previous/current tracking of the id for you. Also utilise os.path.join to make sure your filenames end up joining correctly.

Untested:

import os
import csv
from itertools import groupby

readpath = 'path-to-read-file'
writepath = 'path-to-write-file'

for filename in os.listdir(readpath):
    with open(os.path.join(readpath, filename)) as fin:
        tabin = csv.reader(fin, delimiter='\t')
        for file_id, rows in groupby(tabin, lambda L: L[1]):
            with open(os.path.join(writepath, file_id + '-' + filename), 'w') as fout:
                tabout = csv.writer(fout, delimiter='\t')
                tabout.writerows(rows)