如何将 awk 用于压缩文件

Question

How can I change the following command for a compressed file?

awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }' input1.vcf input2.vcf

The command working fine with normal file. I need to change the command for compressed files.

Answer 1

You need to read them compressed files like this:

awk '{ ... }' <(gzip -dc input1.vcf.gz) <(gzip -dc input2.vcf.gz)

Try this:

awk 'FNR==NR { sub(/AA=\.;/,""); array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }' <(gzip -dc input1.vcf.gz) <(gzip -dc input2.vcf.gz) | gzip > output.vcf.gz

Answer 2

zcat FILE | awk '{ ...}'

我无法分辨出所有这些方法中哪一种效果最好，zcat 至少输入速度更快；)

Answer 3

bzip2 -dc input1.vcf.bz2 input2.vcf.bz2 | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

or

gzip -dc input1.vcf.gz input2.vcf.gz | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

EDIT:

To write compressed output just append

| bzip2 >output.vcf.bz2

or

| gzip >output.vcf.gz

This will work with any program that prints results to standard output.

BTW: Editing such large command lines gets tedious very quickly. You should consider writing a small shell script to do the job. This has the additional benefit that you don't have to remember the entire thing and can easily repeat the command or modify it if necessary.

A good starting point for Linux shell programming is the Bash Programming Inroduction by Mike G.