Hadoop FileSystem-如何删除给定hdfs目录中所有大小为零的文件？

Question

I have a hdfs directory A in the path: /user/A

How do I delete all files within A that are of zero size?

Answer 1

Deleting only files:

hdfs dfs -rm $(hdfs dfs -ls -R /user/A/ | grep -v "^d" | awk '{if ($5 == 0) print $8}')

Test what you get first, since there are lots of HDFS formats, that may have metadata or files with 0 byte sizes (like parquet ..SUCCESS, ..TEMPORARY etc.)

hdfs dfs -ls -R /user/A/ | grep -v "^d" | awk '{if ($5 == 0) print $8}'

Answer 2

这可能会有所帮助

hdfs dfs -ls -R /path/to/directory/ | grep part- | awk '{ if ($5 == 0) print $8 }' | xargs hdfs dfs -rm