Python：读取文本文件并将文件拆分为列表变量，每个变量各有4行

Question

I have a text file (fastq file). The file is in the format

1st line - ID
2nd Line - Sequence
3rd Line - something
4th Line - something else.

Then again its repeat of these 4 lines.

Eg:

1  @M9890393393
2 ATCTGTAAAA
3 +
4 FG%@ATAAAA
5  @M9890393394
6 ATGTCTATCC
7 +
8 AA%$$983089

What I am trying to do is , split this file such that I can read this file by lines of 4. Can I make a list , with each variable containing 4 lines each. There in above example, I will have list with 2 variables.

Answer 1

Using a generator, you can define a lazy reader which yields a list of 4 values each time.

You can, as below, either exhaust or lazily iterate the generator.

import csv
from io import StringIO

mystr = StringIO("""1  @M9890393393
2 ATCTGTAAAA
3 +
4 FG%@ATAAAA
5  @M9890393394
6 ATGTCTATCC
7 +
8 AA%$$983089
""")

def gen():
    # replace mystr with open('file.csv', 'r')
    with mystr as fin:
        reader = csv.reader(fin, delimiter=' ',  skipinitialspace=True)
        res = []
        for line in reader:
            res.append(line[1])
            if len(res) == 4:
                yield res
                res = []

Exhausting the generator:

lines = list(gen())

print(lines)

[['@M9890393393', 'ATCTGTAAAA', '+', 'FG%@ATAAAA'],
 ['@M9890393394', 'ATGTCTATCC', '+', 'AA%$$983089']]

Iterating the generator:

for line in gen():
    print(line)

['@M9890393393', 'ATCTGTAAAA', '+', 'FG%@ATAAAA']
['@M9890393394', 'ATGTCTATCC', '+', 'AA%$$983089']

Answer 2

Read all the lines into a list of individual lines then use a list-comp to group the chunks of four lines together:

with open('your_file') as f:
    lines = f.read().strip().split('\n')

four_lines = [lines[i:i+4] for i in range(0,len(lines),4)]

which, with your example, gives four_lines as:

[
  [
    "1  @M9890393393",
    "2 ATCTGTAAAA",
    "3 +",
    "4 FG%@ATAAAA"
  ],
  [
    "5  @M9890393394",
    "6 ATGTCTATCC",
    "7 +",
    "8 AA%22209983089"
  ]
]

Answer 3

If you just want to chunk it up into 4's then you can use:

In []:
with open('your_file') as f:
    result = list(zip(*[map(str.strip, f)]*4))   # Assumes Py3+ use iter(map(...)) in Py2
result

Out[]:
[('@M9890393393', 'ATCTGTAAAA', '+', 'FG%@ATAAAA'),
 ('@M9890393394', 'ATGTCTATCC', '+', 'AA%$$983089')]

The idea of creating variables for each of these generally doesn't make much sense, but a dict maybe useful if the first line contains an ID you want to use:

In []:
with open('your_file') as f:
    result = {head: tail for head, *tail in zip(*[map(str.strip, f)]*4)}
result
Out[]:
{'@M9890393393': ['ATCTGTAAAA', '+', 'FG%@ATAAAA'],
 '@M9890393394': ['ATGTCTATCC', '+', 'AA%$$983089']}

Sorry assumed the line numbers were added for the example rather than part of the data set. You can replace the zip() with below to remove the numbers (borrowed from @jpp`s answer):

from operator import itemgetter

zip(*[map(itemgetter(1), csv.reader(f, delimiter=' ', skipinitialspace=True))]*4)

Answer 4

The fastq format is easy to parse, you can start checking for "@" at the beginning of the line. That is your sequence ID. You can then simply append the next 3 lines and start again. One "rare" problematic case may occur if the quality-score line also starts with "@". But even that case is easy to spot since the quality-score line comes always after a "+" line.

Answer 5

You can use the function below to read in your file.

list = file.readlines()

once you have read in your file you can use a nested loop to complete the task.