I am writing a small program to load hdfs files using java. When i run the code, i get the list of files from the hdfs. But, i want to get the partition files alone. Eg.part-00000 files.
Below is the sample code:
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://localhost");
FileSystem hdfs = FileSystem.get(new URI(
"hdfs://localhost"), conf);
RemoteIterator<LocatedFileStatus> fsStatus = hdfs.listFiles(
new Path("/hdfs/path"), true);
while (fsStatus.hasNext()) {
String path = fsStatus.next().getPath().toString();
System.out.println(path.matches("part-"));
}
I assume you want to print that path, not the fact that it matches
if (path.startsWith("part-")) {
System.out.println(path);
}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.