Get list of files/directories from hdfs to servlet

Question

I am trying to create a web application in java which can list down all the HDFS files/directories on browser.

Can anyone suggest me how would it be possible? I have tried to get help from google but I was not able to find any direction for it.

Answer 1

You can use HDFS Web UI (in Apache http://namenodehost:50070 ).

However Hadoop HDFS does also provides REST apis provides support to complete FileSystem/FileContext interface for HDFS.

Let's take an example to list files/directories in a specified HDFS location

Request:

curl -i  "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=LISTSTATUS"

Response:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 427

{
  "FileStatuses":
  {
    "FileStatus":
    [
      {
        "accessTime"      : 1320171722771,
        "blockSize"       : 33554432,
        "group"           : "supergroup",
        "length"          : 24930,
        "modificationTime": 1320171722771,
        "owner"           : "webuser",
        "pathSuffix"      : "a.patch",
        "permission"      : "644",
        "replication"     : 1,
        "type"            : "FILE"
      },
      {
        "accessTime"      : 0,
        "blockSize"       : 0,
        "group"           : "supergroup",
        "length"          : 0,
        "modificationTime": 1320895981256,
        "owner"           : "szetszwo",
        "pathSuffix"      : "bar",
        "permission"      : "711",
        "replication"     : 0,
        "type"            : "DIRECTORY"
      },
      ...
    ]
  }
}

link