iterate over bash command

Question

I need to process my data (about 400+ files) using linux bash command. I'm trying to find a way to iterate the same command to all of my files.

This is my bash command

cat file1.vcf | java -jar ~/snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" > file1_filtered.vcf

I tried to do this but not successful

for f in *.vcf; do echo cat *.vcf | java -jar snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" > filtered.vcf; done

This is the error that I'm getting

Error: Unable to access jarfile snpEff_latest_core/snpEff/SnpSift.jar/
Error: Unable to access jarfile snpEff_latest_core/snpEff/SnpSift.jar/
Error: Unable to access jarfile snpEff_latest_core/snpEff/SnpSift.jar/

Answer 1

Based on your question, I will assume that this command of yours works:

cat file1.vcf | java -jar ~/snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" > file1_filtered.vcf

If that is the case, then we can analyze the differences with the second command:

for f in *.vcf; do echo cat *.vcf | java -jar snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" > filtered.vcf; done

1. The first command provides the contents of the files as input to the java script. The second command provides the string cat followed by the names of the .vcf files as input to the java script.

2. The java scripts are in different locations.

To correct those two issues, try:

for f in *.vcf
do
    java -jar ~/snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" <"$f" > filtered.vcf
done

Where I have also removed a useless use of cat.

Lastly, the above overwrites filtered.vcf each time a file is processed. If you want one file to contain all the results, then try:

for f in *.vcf
do
    java -jar ~/snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" <"$f"
done > filtered.vcf

Answer 2

Your Java program seems to read input line by line. So, if you are not interested in having your output in one file per input file, you can skip the for loop altogether:

cat *.vcf | java -jar ~/snpEff_latest_core/snpEff/SnpSift.jar/ filter " ( QUAL >= 30 )" > files_filtered.vcf

This has the added benefit of the cat actually concatenating files, for once.