需要使用子进程模块在 python 脚本中运行 AWK 命令

Question

c1 = "'BEGIN{FS = OFS = \",\"}{if(toupper($11) ~ \"DVT\"){$(NF+1) = NR==1 ? \"indication\" : \"DVT\"}else if(toupper($11) ~ \"AFIB\"){$(NF+1) = NR==1 ? \"indication\" : \"AFIB\"}else{$(NF+1) = NR==1 ? \"indication\" : \"TESTING\"}} 1'"

print(c1)

p1=subprocess.Popen(["awk",c1,"abc.csv"],stdout=outfile)

p1.communicate()

This command is running fine in shell script. So, the command parameters seem fine. But while running through python i keep getting the error: "invalid char ''' in expression."

Answer 1

The outer quotes are unnecessary. When you have a shell, the single quotes are necessary to protect the Awk script from the shell; but here, you don't have a shell. Then you can and should also get rid of all those backslashes.

You should prefer subprocess.run() over bare Popen if you just need the subprocess to run, and then continue your program when it's done.

c1 = '''BEGIN{FS = OFS = ","}
{if(toupper($11) ~ "DVT"){$(NF+1) = NR==1 ? "indication" : "DVT"}
 else if(toupper($11) ~ "AFIB"){$(NF+1) = NR==1 ? "indication" : "AFIB"}
 else{$(NF+1) = NR==1 ? "indication" : "TESTING"}} 
1'''

print(c1)

result = subprocess.run(["awk", c1, "abc.csv"],
    stdout=outfile)

However, running Awk as a subprocess of Python is almost always unnecessary. You should be able to refactor this into roughly the same amount of Python code.

with open("abc.csv") as infile:
    firstline = True
    for line in infile:
        fields = line.rstrip("\n").split(",")
        if firstline:
            added = "indication"
        else:
            ind = fields[10].upper()
            if "DVT" in ind:
                added = "DVT"
            elif "AFIB" in ind:
                added = "AFIB"
            else:
                added = "TESTING"
        fields.append(added)
        outfile.write(",".join(fields) + "\n")
        firstline = False

This is somewhat more verbose but that's mainly because I used more descriptive variable names. If your input file is really a CSV file, the csv module from the Python standard library could conveniently replace the splitting logic etc, though it then introduces some other idiosyncrasies of its own. If you have to deal with quoted commas etc in CSV, that's where it really adds value.

All things counted, Awk is more terse, but that's because it's more specialized. The main drawback of embedding it in your Python code is that the reader has to understand both languages in order to understand the code (though avoiding an external process is also always nice). I'll venture a guess that you were handed this Awk code without any explanation as to how it works or how to fix it if it breaks...?