简体   繁体   中英

Load hdfs partitions files list

I am writing a small program to load hdfs files using java. When i run the code, i get the list of files from the hdfs. But, i want to get the partition files alone. Eg.part-00000 files.

Below is the sample code:

            Configuration conf = new Configuration();
            conf.set("fs.defaultFS", "hdfs://localhost");
            FileSystem hdfs = FileSystem.get(new URI(
                    "hdfs://localhost"), conf);
            RemoteIterator<LocatedFileStatus> fsStatus = hdfs.listFiles(
                    new Path("/hdfs/path"), true);
            while (fsStatus.hasNext()) {
                String path = fsStatus.next().getPath().toString();
                System.out.println(path.matches("part-"));

            }

I assume you want to print that path, not the fact that it matches

if (path.startsWith("part-")) {
    System.out.println(path);
} 

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM