解析制表符分隔的文件

Question

I have a tab separated file ( raw.txt ) which has format like :

type    A1    A2    A3    A4    ....
params  int   char  char  char  ...
data    1     abc   cde   fgh   ...
type    B1    B2    B3    B4    ....
feature int   char  char  char  ...
data    2     aaa   bbb   ccc   ...   
type    C1    C2    C3    C4    ....
stats   int   int   char  char  ...
data    2     11    aa    bb    ...
data    3     12    cc    cc    ...
data    4     13    dd    dd    ...
data    5     14    ee    ee    ...
...     ...   ...   ...   ...   ...

I want to parse such file and then I wanted to create a folder named of file name and then inside that folder, I want to create different files based on types . Files will be created with data once type observed in the line for example :

/raw/file1
A1    A2    A3    A4    ....
int   char  char  char  ...
1     abc   cde   fgh   ...

/raw/file2
B1    B2    B3    B4    ....
int   char  char  char  ...
2     aaa   bbb   ccc   ...

/raw/file3
C1    C2    C3    C4    ....
int   int   char  char  ...
2     11    aa    bb    ...
3     12    cc    cc    ...
4     13    dd    dd    ...
5     14    ee    ee    ...
...   ...   ...   ...   ...

and so on... and also i want to create dictionaries like,

dict1 = {A1:['int', [1]], A2:['char', ['abc']], ...}
dict2 = {B1:['int', [2]], B2:['char', ['aaa']], ...}
dict3 = {C1:['int', [2, 3, 4, 5], C2:['int', [11, 12, 13, 14, ...]], ...}

How can I do that because this is a very big file and parsing such huge file hangs the window and also I am not able to figure out how can I get such output from the file.

Answer 1

Here is the code to parse the file and generate the new files. The newfile list temporarily stores the data for each type before writing to file. A line that starts with 'type' will trigger newfile to store results for that type and if there were contents in newfile before it will write them to file. The inc variable is incremented each time you make a call to writefile() and is used as the suffix for the filename. In writefile() I used str.format() to create a dynamic file name and also to write to file left justified with a width of 6 taking arbitrarily many values from linelist .

def writefile(newfilelist, suffix):
  with open('file{}'.format(suffix), 'w') as f:
    for linelist in newfilelist:
      f.write(('{:<6}'*len(linelist)).format(*linelist) + '\n')

import os
with open('raw.txt') as file:
  os.mkdir('raw')
  os.chdir('raw')
  newfile = []
  inc = 0
  for line in file:
    linelist = line.split()
    if linelist[0] == 'type':
      if newfile:
        inc += 1
        writefile(newfile,inc)
        newfile = []
      newfile.append(linelist[1:])
    else:
      newfile.append(linelist[1:])
  if newfile:
    inc += 1
    writefile(newfile,inc)