解析.TSV文件并通过重新排列列将数据写入新的.TSV文件

Question

So, I want to read a TSV file (>1M rows) and open another tsv file which will copy the exact data but re-arrange the columns.

For example,

Original tsv file:

A   B . . . . .H
a1  b1.. . . . h1
a2  b2. . . . .h2
a3  b3. . . . .h3
.   .. . . . . . so on. 

(The first line are the headers)

I know how to create, read and write a file but I don't know how to re-arrange the columns.

file_location = 'abc.tsv'
output_filename = 'sample.tsv'


def main():
    file_reader = open(file_location,'r')
    new_file = open(output_filename,'w')

    for rows in file_reader:
        try:
                rows = rows.strip().split('\t')


        except Exception, e:
            print('Error in reading file: %s' % e)
            pass

    file_reader.close()
    new_file.close()


if __name__ == '__main__':
    main()

Expected output :

D   G . . . . B
d1  g1. . . . b1
d2  g2. . . . b2
d3  g3. . . . b3
d4  g4. . . . b4
.   . . . . . .
.  .  . . . . . so on.

Any ideas are appreciated. Thank you.

Answer 1

As I mentioned in a comment, you could use the csv module to do this. It would also be fairly fast (notice there's no explicit loop over the rows or fields of the files, plus the csv module is written in C).

For example:

import csv


file_location = 'abc.tsv'
output_filename = 'sample.tsv'
infields =  'A', 'B', 'C', 'D', 'G', 'H'
outfields = 'D', 'G', 'A', 'H', 'C', 'B'


def main():
    with open(file_location, 'r', newline='') as inp, \
         open(output_filename, 'w', newline='') as outp:

        reader = csv.DictReader(inp, fieldnames=infields, delimiter='\t')
        writer = csv.DictWriter(outp, fieldnames=outfields, delimiter='\t',
                                extrasaction='ignore')

        writer.writerows(reader)


if __name__ == '__main__':
    main()

Answer 2

You can do this with ease using pandas, just convert the file to a pandas dataframe and alter the columns order as you want of the dataframe and then write it back to a TSV file.

For reading the file into a pandas dataframe use:

import pandas as pd    
df = pd.read_csv("abc.tsv", sep='\t', header=0)

You learn about basics of pandas in here

Answer 3

Something like this:

(I did not change the table headers locations)
I have also skipped reading/writing from/to files since I assume this is not a challenge for you.

original_data = [['A','B','C'],['a1','b1','c1'],['a2','b2','c2']]

def switch_columns(column_pairs,entries):
  for pair in column_pairs:
    for idx,entry in enumerate(entries):
      if idx > 0: 
        temp = entry[pair[0]]
        entry[pair[0]] = entry[pair[1]]
        entry[pair[1]] = temp

print('Before:')
print(original_data)
switch_columns([(0,2)],original_data)
print('After:')
print(original_data)

output

Before:
[['A', 'B', 'C'], ['a1', 'b1', 'c1'], ['a2', 'b2', 'c2']]
After:
[['A', 'B', 'C'], ['c1', 'b1', 'a1'], ['c2', 'b2', 'a2']]