如何从 HDFS 中的目录结构中删除所有文件？

Question

I am looking to clean up the environment. I want to execute the following command recursively, witout hard coding for each sub folder. As I have hundreds of subfolder in a HDFS directory.

hadoop fs -rm -r /a/b1/c1/d1.txt

hadoop fs -rm -r /a/b2/c2/d2.txt

hadoop fs -rm -r /a/b3/c3/d3.txt

hadoop fs -rm -r /a/bn/cn/dn.txt

I want to delete all the files but not the directory or the sub-directories. I am not sure how to use a recursive command to keep the directory structure in HDFS while delete all the files in it.

Answer 1

hadoop fs -ls -R /directory/structure/ | grep "\.json" | awk '{print $8}' | xargs -I{} hadoop fs -rm "{}"