连接特定数量的文件

Question

I have a bunch of files named uv_set_XXXXXXXX where the 6 Xs stand for the usual format year, month and day. Imagine I have 325 files of this type. I would like to concatenate by groups of 50 files, so in the end I have 7 files (6 files of 50 and 1 of 25).

I have been thinking in using cat but I can't see an option to select a number of files from a list. I could do this with Python, but just wondering if some Unix command line utility does it more directly.

Thanks.

Answer 1

With GNU parallel you can use the following command

parallel -n50 "cat {} > out{#}" ::: uv_set_*

This will merge the first 50 files into out1 , the next 50 files into out2 , and so on.

Answer 2

I would just break down and do this in Awk.

awk 'FNR==1 && (++i%50 == 0) {
    if(NR>1) close p;
    p = "dest_" ++j }
    { print >p }' uv_set_????????

This creates files dest_1 through dest_7 , the first 6 with 50 files in each and the last with the remainder.

Closing the previous file is necessary because the system only allows Awk to have a limited number of open file handles (though the limit is typically higher than 7 so it's probably not important in your example).

---

Thinking out loud dept, just to prevent anyone else from wasting time on repeating this dead end.

You could use xargs -L 50 cat to concatenate 50 files at a time, but there is no simple way to pass in a new redirection for standard output for each invocation. You could try to hack your way around that with something like

# XXX Do not use: incomplete
printf '%s\n' uv_set_???????? |
xargs -L 50 sh -c 'cat "$@" > ... something' _

but I can't come up with elegant way to have a different something each time.