Python将输出作为输入之一循环遍历一个进程

Question

I have a python process which takes in two input files and writes out one output file:

def process(fin1,fin2,fout):
    outf = fileout(fout)
    x = readcsv(fin1)
    y = readcsv(fin2)
    hlen = writeheaders(fin1,fin2,outf)
    sub(matchingfields(x,y),x,y,hlen,outf)
    outf.close()

I can run it easily enough like this, (defining the name of output file):

process('csv1a.csv','csv2b.csv','OUTv1.csv')

I want to input more than two files (drag and drop onto a batch file or use the cmd):

filenames = sys.argv[1:]

So the issue is how can I easily (recursively?) loop though my process so that:

run process on filenames[0] with filenames[1]

run process on output of (filenames[0] + filenames[1]) with filenames[2]

run process on output of (filenames[0] + filenames[1] + filenames[2]) with filenames[3]

etc..

I am quite new to programming and cant quite figure out the best way to approach this problem. Thank in advance !!

Answer 1

You already found sys.argv you can use len(sys.argv)-1 to find out how many files were passed to your script. In general len() gives you the length of the array.

You would then create a for loop, which loops over all passed file inputs, starting at the third, except for the last one, as this is the output file. For each input file you use the existing output file and the new file of this iteration and run it with your existing function and write the results to the output file.

In the beginning you should run your existing method with input file 1 and 2 and the output file as output.

Answer 2

For drag-and-drop, create a batch file that contains

@echo off
python myscript.py %*

and save it as myscript.bat . You can now drag-and-drop files on it and it will pass all the filenames as arguments to your script.

Rather than chunking all your data back and forth through files repeatedly, I suggest processing it in memory and then writing the final result to disk.

import sys

def read_csv(fname):
    # read csv file into memory
    # YOUR CODE GOES HERE
    return data

def write_csv(data, fname):
    # write data out to file
    # YOUR CODE GOES HERE

def merge(data1, data2):
    # YOUR CODE GOES HERE
    return merged_data

if __name__ == "__main__":
    if len(sys.argv) < 3:
        print("Usage: python {} file1.csv file2.csv [...]".format(__name__))
    else:
        data = read_csv(sys.argv[1])
        for fname in sys.argv[2:]:
            more_data = read_csv(fname)
            data = merge(data, more_data)
        write_csv(data, "final.csv")