Python 代码在 CS50x 的 DNA 问题中运行时间过长

Question

I have written a code to solve the DNA problem from week 6 of CS50. However, when I am running it on large.csv databases and sequences, it takes at least one minute to produce an output. On small.csv it produces an output instantly. Because of it, I can't pass check50. I suppose that the problem is on the stage of a function for generating the number of max repeating sequences of STRs, but I don't know how to write it more efficiently. The full description of the problem is here: https://cs50.harvard.edu/x/2021/psets/6/dna/#:~:text=check50%20cs50/problems/2021/x/dna

And here are the source files for databases and sequences: https://cdn.cs50.net/2019/fall/psets/6/dna/

This is my code:

import csv
import sys


def main():
    
    # check a proper input
    if len(sys.argv) != 3:
        sys.exit("Usage: python dna.py data.csv sequence.txt")
    
    # create a list for all data
    data_all = []
    
    # create a list for all STRs
    STR_all = []
    
    # write data to list
    with(open(sys.argv[1])) as data:
        reader = csv.DictReader(data)
        for row in reader:
            row["name"]
            data_all.append(row)
            
    # write header to a list 
    with(open(sys.argv[1])) as data:      
        reader = csv.reader(data)
        headings = next(reader)
        STR_all.append(headings)
    
    # delete "name" from header, it is on the first position    
    STR_all = STR_all[0]
    STR_all.pop(0)
            
    # create a string with DNA sequence
    with(open(sys.argv[2])) as seq:
        line = seq.read()
    
    # create a list with max number of repeating STR from a line(DNA)
    max_seq = []
    
    # enter data with string of STR and it's max repeating time    
    for i in range(len(STR_all)):
        result = f"{compare(STR_all[i], line)}"
        max_seq.append(result)
        
    # create a dictionary with a list of all STRs and according number of repeating sequences
    STR_with_max_seq = dict(zip(STR_all, max_seq))
    
    # compare values from data_all and STR_with_max_seq
    for i in range(len(data_all)):
        # delete name key and store key in variable "name"
        name = data_all[i].pop('name')
        if data_all[i] == STR_with_max_seq:
            print(name)
            sys.exit()
            break
        else:
            continue
        
    # Print if no match found
    print("No match")
        
    # variables that I used to check on different stages of writing a program
            
    # print(data_all)
    # print(line)
    # print(STR_all)
    # print(max_seq)
    # print(STR_with_max_seq)
    
    # print(len(data_all))
    # print(name)

    
def compare(STR, DNA):

    for key in DNA:
        l = len(STR)
        tmp_max = 0
        tmp = 0
        
        # iteration through the whole length of DNA
        for i in range(len(DNA)):
            if tmp > 0:
                tmp = 0
            
            # enters if sequences are equal
            if DNA[i: i + l] == STR:
                tmp += 1
                # increments tmp if its sequence repeats
                while DNA[i - l: i] == DNA[i: i + l]:
                    tmp += 1
                    i += l
                # update the max found number of repeating sequences    
                if tmp > tmp_max:
                    tmp_max = tmp
    
    return tmp_max

    
main()

UPDATE: I have used time.monotonic() to check the total time for the code execution in main(). This is time for small.csv:

• Value of the monotonic clock (in fractional seconds): 661689.405232647
• Time elapsed during the process: 0.02439890895038843

And this is for large.csv:

• Value of the monotonic clock (in fractional seconds): 661943.13288005
• Time elapsed during the process: 108.33000503003132

Answer 1

I know that problem. There are some parts of your code that make it slow.

First, lets try to read each file only once. For instance:

with(open(sys.argv[1])) as data:
    reader = csv.DictReader(data)
    STR_all = reader.fieldnames
    for row in reader:
        row["name"]
        data_all.append(row)

STR_all will be a list, so you can remove the line:

STR_all = STR_all[0]

You can build the dictionary of counters when comparing, so you can avoid iterating twice.

For example, by doing this:

    # enter data with string of STR and it's max repeating time    
    for i in range(len(STR_all)):
        key = STR_all[i]
        STR_with_max_seq[key] = str(compare(key, line))

You can remove this:

    # create a dictionary with a list of all STRs and according number of repeating sequences
    STR_with_max_seq = dict(zip(STR_all, max_seq))

Finally, to improve the function compare, you can avoid the first loop. You want to find the maximum number of successive occurrences of STR inside DNA. So, you have only to iterate through DNA by windows of the length of STR and comparing them. For example:

def compare(STR, DNA):
    l = len(STR)
    tmp_max = 0
    tmp = 0
    i = 0
        
    # iteration through the whole length of DNA
    while i < len(DNA) - l:  # make sure the last str has length = l
        SSTR = DNA[i : i + l]  # Extract a substre of length l

        if SSTR == STR:
            # You can jump l positions here.
            i += l
            tmp += 1
        else:
            i += 1
            if tmp > tmp_max:
                tmp_max = tmp
            tmp = 0                
    
    return tmp_max