Extracting rows from file based on another file using awk

Question

I have two files.

SNP Allele1 Allele2 Effect  StdErr  PVAL    Direction   HetISq  HetChiSHetDf    HetPVal
rs12266638  t   g   0.4259  0.0838  3.776e-07   +?  0.0 0.000   0   1
rs7995014   t   c   2.2910  0.5012  4.853e-06   +?  0.0 0.000   0   1

Answer 1

You may use this awk<\/code> :

awk 'FNR==NR {a[$3]; next} FNR> 1 && $1 in a' file2 file1

rs12266638  t   g   0.4259  0.0838  3.776e-07   +?  0.0 0.000   0   1

Answer 2

Depending on how big the dataset is, this should be fairly fast, only accessing each file once. Granted, not on a system where I can compare at the moment, so mostly a hunch. A solution like this is probably only suitable if the amount of unique identifiers isn't very large, though.

#!/bin/bash
snp_expression=$(awk 'FNR>1{print $3}' file_2 | sort -u | paste -sd "|")
grep -E "^(${snp})[[:space:]]" file_1 > file_3

Answer 3

A more general solution which works for any position of the SNP field:

# SO71009277.awk
BEGIN {
  fnr = 0
  while ((getline < ARGV[1]) > 0) {
    ++fnr
    if (fnr == 1) {
      for (i=1; i<=NF; i++)
        FIELDBYNAME1[$i] = i # e.g. FIELDBYNAME1["SNP"] = 1
    }
    else {
      SNP_KEY[$FIELDBYNAME1["SNP"]] = $0
    }    
  }
  close(ARGV[1])

  fnr = 0
  while ((getline < ARGV[2]) > 0) {
    ++fnr
    if (fnr == 1) {
      for (i=1; i<=NF; i++)
        FIELDBYNAME2[$i] = i # e.g. FIELDBYNAME2["SNP"] = 3
    }
    else {
      if ($FIELDBYNAME2["SNP"] in SNP_KEY)
        print SNP_KEY[$FIELDBYNAME2["SNP"]]
    }    
  }
  close(ARGV[2])
}

Call:

awk -f SO71009277.awk file1.txt file2.txt
=>
rs12266638  t   g   0.4259  0.0838  3.776e-07   +?  0.0 0.000   0   1