I am reading multiple files (1000 files of approx size 5mb) from folder. The code below is working fine to read, load and store the content of file.
public void readAllFiles(String path) {
try (Stream<Path> paths = Files.walk(Paths.get(path)).collect(toList()).parallelStream()) {
paths.forEach(filePath -> {
if (filePath.toFile().exists()) {
String fileName = filePath.getFileName().toString();
try {
List<String> loadedFile = readContent(filePath);
storeFiles(fileName, filePath, loadedFile);
} catch (Exception e) {
LOGGER.info("ERROR WHILE READING THE CONTENT OF FILE");
LOGGER.error(e.getMessage());
}
}
});
} catch (IOException e) {
LOGGER.info("ERROR WHILE READING THE FILES IN PARALLEL");
LOGGER.error(e.getMessage());
}
}
My problem is while reading the files I want to exclude some files, like exclude the file reading if for example the condition satisfies (filename contains "ABC" && flag is true)
Thanks in advance for any suggestions.
Files.walk()
returns Stream<Path>
so you don't need to convert it to list. use the following code to use in parallel and filter it base on conditions.
try (Stream<Path> paths = Files.walk(Paths.get(path)).parallel()
.filter(filePath->filePath.getFileName().toString().contains("ABC"))) {
paths.forEach(filePath -> {
//other staff...
});
} catch (IOException e) {
}
I would rewrite this using the filter
function:
paths.filter(e -> e.toFile().exists()) //Make sure each file exists
.map(path -> path.getFileName().toString()) //Map it to its fileName
.filter(file -> !file.contains("someString")) //Filter
.forEach(fileName -> { //Rest of logic
try {
List<String> loadedFile = readContent(filePath);
storeFiles(fileName, filePath, loadedFile);
} catch (Exception e) {
LOGGER.info("ERROR WHILE READING THE CONTENT OF FILE");
LOGGER.error(e.getMessage());
}
});
Which will map to a String
representation of the before you do the forEach
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.