run python script in every file in a directory through command line

Question

I have a python script which takes the filename as a command argument and processes that file. However, i have thousands of files I need to process, and I would like to run the script on every file without having to add the filename as the argument each time.

for example: process.py file1 will do exactly what I want

however, I want to run process.py on a folder containing thousands of files (file1, file2, file3, etc.)

I have found out it that it can be done simply in Bash

for f in *; do python myscript.py $f; done

However, I am on windows and don't want to install something like Cygwin. What would a piece of code for the Windows command line look like that would emulate what the above Bash code accomplishes?

Answer 1

for %%f in (*.py) do (
    start %%f
)

I think that'll work -- I don't have a Windows box handy at the moment to try it

How to loop through files matching wildcard in batch file

That link might help

Answer 2

import os, subprocess
for f in os.listdir('.'):
    if os.path.isfile(f):
        subprocess.call(["python", "myscript.py", f])

this solution will work on every platform, provided the python executable is in the PATH.

Also, if you want to recursively process files in nested subdirectories, you can use os.walk() instead of os.listdir() + os.path.isfile() .

Answer 3

Since you have python, why not use that?

import subprocess
import glob
import sys
import os.path

for fname in glob.iglob(os.path.join('some-directory-name','*')):
    proc = subprocess.Popen([sys.executable, 'myscript.py', fname])
    proc.wait()

What's more, its portable.

Answer 4

For each file in current dir.

for %f in (*) do C:\Python34\python.exe "%f"

Update: Note the quotes on the %f . You need them if your files contain spaces in the name. You can also put any path+executable after the do .

If we imagine your files look like:

./ process.py

./myScripts/ file1.py

./myScripts/ file2.py

./myScripts/ file3.py

...

In your example, would simply be:

for %f in (.\myScripts\*) do process.py "%f"

This would invoke:

process.py ".\myScripts\file1.py"
process.py ".\myScripts\file2.py"
process.py ".\myScripts\file3.py"