How HDFS delete the block?

Question

I am new to Hadoop.

After my deployment, I ran the benchmark, $hadoop jar hadoop-0.20.0-test.jar TestDFSIO -write -nrFiles 1000 -fileSize 10.

After the work done, I looked at the Cluster Summary, Say 1000 files and directories, 1000 blocks = 2000 total.

Then I use the shell cmd to delete the corresponding files. Then the Cluster Summary is: 0 files and directories 1000 blocks = 1000 total.

I think after the deletion there should be no files and no blocks in the Cluster Summary. But the actual situation are: Before I ask this question is 1000 blocks. But now there're 0 blocks.

How can this happen?

Answer 1

Delete is just instructing the NameNode to remove the file from its metadata store. After this has been committed, the NameNode will then reach out to the Datanodes which host the blocks and instruct them to delete the blocks.

The actual file deletion on the data nodes could take a few seconds (depending on the number of datanodes, block size and number of blocks to delete), and then the data nodes will report back to the nameNode, at which point it will update the global number of blocks. As you observed, after a few moments, the total number of blocks will be back to 0

Answer 2

块删除操作异步完成。