How to automatically manage a file system directory tree using Java/Spring?

Question

I am trying to create a file system integration for a Spring Boot web application.

I have a currently working solution that stores the file content in file system and file name and location in a database.

Contents on disk:

I am trying to find a way to separate saved files into folders. The root folder should be "file-system" and files should be separated into folders in a way that one folder contains no more than 500 files.

What is the correct way to do this?
Is there a directory tree manager that is built in to Spring or Java that I can use?

My current solution is below.

DBFile :

@Entity
public class DBFile {

    @Id
    private Long id;

    @Column(name = "name")
    private String name;

    @Column(name = "location")
    private String location;
}

FileSystemRepository :

@Repository
public class FileSystemRepository {
    public static final String RESOURCES_ROOT_DIR = "/file-system/";

    public String save(byte[] content, String fileName, String contentType) throws IOException {
        Path path = getPath(fileName, contentType);
        Files.createDirectories(path.getParent());
        Files.write(path, content);
        return path.toAbsolutePath().toString();
    }

    public FileSystemResource findInFileSystem(String location) {
        return new FileSystemResource(Paths.get(location));
    }

    public void deleteInFileSystem(String location) throws IOException {
        Files.delete(Paths.get(location));
    }

    private Path getPath(String fileName, String contentType) {
        Path path = FileSystems.getDefault().getPath("").toAbsolutePath();
        String subDirectory;
        if (contentType.startsWith("image/")) {
            subDirectory = "images/";
        } else {
            subDirectory = "files/";
        }
        return Paths.get(path + RESOURCES_ROOT_DIR + subDirectory + new Date().getTime() + "-" + fileName);
    }
}

FileSystemService :

@Service
public class FileSystemService {

    private final FileSystemRepository fileSystemRepository;
    private final DBFileRepository dbFileRepository;

    public Long save(byte[] bytes, String imageName, String contentType) {
        try {
            String location = fileSystemRepository.save(bytes, imageName, contentType);

            return dbFileRepository.save(new DBFile(imageName, location))
                .getId();
        } catch (IOException e) {
            throw new ResponseStatusException(HttpStatus.BAD_REQUEST);
        }
    }

    public FileSystemResource find(Long id) {
        DBFile image = dbFileRepository.findById(id)
            .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));

        return fileSystemRepository.findInFileSystem(image.getLocation());
    }

    public void delete(Long id) {
        DBFile image = dbFileRepository.findById(id)
            .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));

        try {
            fileSystemRepository.deleteInFileSystem(image.getLocation());
        } catch (IOException e) {
            throw new ResponseStatusException(HttpStatus.NOT_FOUND);
        }

        dbFileRepository.delete(image);
    }
}

Answer 1

I was struggling with this issue for quite a while, but now I found a possible solution that a nice colleague of mine sent to me.

A possible solution is to create a hashed directory structure.

More information and also an example that I used can be found here: https://medium.com/eonian-technologies/file-name-hashing-creating-a-hashed-directory-structure-eabb03aa4091

My final solution to create a path from file name:

  private static Path getPath(String fileName) {
    Path root = FileSystems.getDefault().getPath("").toAbsolutePath();
    int hash = fileName.hashCode();

    int mask = 255;
    int firstDir = hash & mask;
    int secondDir = (hash >> 8) & mask;

    String path = root +
        File.separator +
        RESOURCES_ROOT_DIR +
        File.separator +
        String.format("%03d", firstDir) +
        File.separator +
        String.format("%03d", secondDir) +
        File.separator +
        getDatedFileName(fileName);

    return Paths.get(path);
}

private static String getDatedFileName(String fileName) {
    LocalDateTime today = LocalDateTime.now();
    return String.format("%s-%s-%s_%s_%s_%s-%s",
        today.getYear(),
        today.getMonthValue(),
        today.getDayOfMonth(),
        today.getHour(),
        today.getMinute(),
        today.getSecond(),
        fileName);
}