如何提高生物信息学脚本的速度？

Question

I'm developing a python script for bioinformatic analysis. First, the script reads the entire file (.fasta – which is basically a very long string) to find all the scaffolds (lines that starts with '>'), then it prints out the number of scaffolds found. I have two similar input files .fasta, one with over 1.5GB that runs in less than a minute, and a second one with 85MB that takes more than 31 HOURS.

import sys

cabecalho = []
sequencia = []
contador = -1
file_open = open('C:\PYTHON\Chr09.fasta', "r")
for line in file_open:
    line = line.rstrip()
    if ">" in line:
        cabecalho.append(line)
        contador += 1
        sequencia.insert(contador, '')
    else:
        sequencia[contador] += line
con = contador + 1
print(con)

What can I do to optimize the running speed of this script? Or how can I check what's wrong with the file (They have the same format, and the same text config.)?

Answer 1

First of all, you don't need to reinvent the weel, Biopython can easly handle the fasta files, for example:

from Bio import SeqIO

myseqs = {}
fasta_sequences = SeqIO.parse(open('C:\PYTHON\Chr09.fasta'), 'fasta')
for fasta in fasta_sequences:
 name, sequence = fasta.description, str(fasta.seq)
 myseqs[name] = sequence

print("total sequences: "+len(myseqs))

In this way you will also have your sequences as a dict to easly access through the fasta header and do whatever you want.

Finally, to install biopython just type

pip install biopython

Other approach without python, if you only want to know the number of scaffolds, you can do it in one line using grep command in unix enviroment

grep -c ">" myfasta.fasta

-c is for count match only

Regards