hadoop hdfs java-将文件列表从hdfs复制到hdfs的最佳方法是什么

Question

I have a file that has two columns, the first column is the hdfs path to source files, the second column is the hdfs path to target file:

s1, t1 s2, t2 .., .. sn, tn

what is the fastest way for me to copy the source path to their respective target path. is there such a tool for hadoop?

The list is probably 100-200 lines long each file is a few megabytes.

Answer 1

The list is probably 100-200 lines long each file is a few megabytes.

If this a one-off kind of situation, then this isn't large enough to worry about. A dumb ol' shell loop will do fine:

cat pairs-file | while read pair; do hdfs dfs -cp $pair; done