how to delete specific block from HDFS

Question

How can I delete a specific block for a file from HDFS?

I have loaded a 1TB file in hdfs and found 1 block is corrupted by using fsck command. I am using this for testing so I haven't created replica of the file in hdfs. Now, straightforward solution for this to delete the complete file and reload the whole file again. But it seems inefficient to load whole file (~1TB data) just because of 128MB corrupted records.

Hence, losing some records is okay for my use case(it is just a poc). Is there any way that I can just remove the corrupted block and use rest of the data?

Any suggestion will be appreciated.

Answer 1

您可以使用命令-hdfs fsck / -delete列出丢失的块的损坏并按照本文进行修复。