I am using the standard join command to join two sorted files based on column1. The command is simple join file1 file2 > output_file.
But how do I join 3 or more files using the same technique ? join file1 file2 file3 > output_file Above command gave me an empty file. I think sed can help me but I am not too sure how ?
man join
:
NAME
join - join lines of two files on a common field
SYNOPSIS
join [OPTION]... FILE1 FILE2
it only works with two files.
if you need to join three, maybe you can first join the first two, then join the third.
try:
join file1 file2 | join - file3 > output
that should join the three files without creating an intermediate temp file. -
tells the join command to read the first input stream from stdin
One can join multiple files (N>=2) by constructing a pipeline of join
s recursively:
#!/bin/sh
# multijoin - join multiple files
join_rec() {
if [ $# -eq 1 ]; then
join - "$1"
else
f=$1; shift
join - "$f" | join_rec "$@"
fi
}
if [ $# -le 2 ]; then
join "$@"
else
f1=$1; f2=$2; shift 2
join "$f1" "$f2" | join_rec "$@"
fi
I know this is an old question but for future reference. If you know that the files you want to join have a pattern like in the question here eg file1 file2 file3 ... fileN
Then you can simply join them with this command
cat file* > output
Where output will be the series of the joined files which were joined in alphabetical order.
I created a function for this. First argument is the output file, rest arguments are the files to be joined.
function multijoin() {
out=$1
shift 1
cat $1 | awk '{print $1}' > $out
for f in $*; do join $out $f > tmp; mv tmp $out; done
}
Usage:
multijoin output_file file*
While a bit an old question, this is how you can do it with a single awk
:
awk -v j=<field_number> '{key=$j; $j=""} # get key and delete field j
(NR==FNR){order[FNR]=key;} # store the key-order
{entry[key]=entry[key] OFS $0 } # update key-entry
END { for(i=1;i<=FNR;++i) {
key=order[i]; print key entry[key] # print
}
}' file1 ... filen
This script assumes:
<field_number>
<field_number>
is a valid integer. The man
page of join
states that it only works for two files. So you need to create and intermediate file, which you delete afterwards, ie:
> join file1 file2 > temp
> join temp file3 > output
> rm temp
Join joins lines of two files on a common field. If you want to join more - do it in pairs. Join first two files first, then join the result with a third file etc.
Assuming you have four files A.txt, B.txt, C.txt and D.txt as:
~$ cat A.txt
x1 2
x2 3
x4 5
x5 8
~$ cat B.txt
x1 5
x2 7
x3 4
x4 6
~$ cat C.txt
x2 1
x3 1
x4 1
x5 1
~$ cat D.txt
x1 1
Join the files with:
firstOutput='0,1.2'; secondOutput='2.2'; myoutput="$firstOutput,$secondOutput"; outputCount=3; join -a 1 -a 2 -e 0 -o "$myoutput" A.txt B.txt > tmp.tmp; for f in C.txt D.txt; do firstOutput="$firstOutput,1.$outputCount"; myoutput="$firstOutput,$secondOutput"; join -a 1 -a 2 -e 0 -o "$myoutput" tmp.tmp $f > tempf; mv tempf tmp.tmp; outputCount=$(($outputCount+1)); done; mv tmp.tmp files_join.txt
Results:
~$ cat files_join.txt
x1 2 5 0 1
x2 3 7 1 0
x3 0 4 1 0
x4 5 6 1 0
x5 8 0 1 0
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.