猫几千个档案

Question

I have several(60,000) files in a folder that need to be combined into 3 separate files. How would I cat this so that I could have each file containing the contents of ~20,000 of these files?

I know it would be like a loop:

for i in {1..20000}
do
cat file-$i > new_file_part_1
done

Answer 1

Doing:

cat file-$i > new_file_part_1

Will truncate new_file_part_1 every time the loop iterates. You want to append to the file:

cat file-$i >> new_file_part_1

Answer 2

The other answers close and open the file on every iteration. I would prefer

for i in {1..20000}
do
    cat file-$i
done > new_file_part_1

so the output of all cat runs are piped into one file opend once for all.

Answer 3

Assuming it doesn't matter which input file goes to which output file:

for i in {1..60000} 
do 
  cat file$i >> out$(($i % 3))
done

This script uses the modulo operator % to divide the input into 3 bins; it will generate 3 output files:

• out0 contains file3, file6, file9, ...
• out1 contains file1, file4, file7, ...
• out2 contains file2, file5, file8, ...

Answer 4

#!/bin/bash

cat file-{1..20000} > new_file_part_1

This launches cat only once and opens and closes the output file only once. No loop required, since cat can accept all 20000 arguments.

An astute observer noted that on some systems, the 20000 arguments may exceed the system's ARG_MAX limit. In such a case, xargs can be used, with the penalty that cat will be launched more than once (but still significantly fewer than 20000 times).

echo file-{1..20000} | xargs cat > new_file_part_1

This works because, in Bash, echo is a shell built-in and as such is not subject to ARG_MAX .