如何为 gs 注册 URL stream 处理程序：Java Z38008ECDD81C2CEAFF1D71 中的协议

Question

I have code that must be able to handle "normal" urls like "file:///..." and also "gs://..." files.

I have an application.yaml file that has

bannedKeywordsFile: gs://myproject.appspot.com/mybucket/myfile.csv

This fails with a MalformedURLException.

@Value("${bannedKeywordsFile}")
private Resource bannedKeywordsFile;

private URL myUrl() {
    return bannedKeywordsFile.getURL();
}

To be fair, the Google docs explain this is expected unless a protocol handler is registered. Is there a handler that can be registered? Or must one be written from scratch?

Answer 1

gs is a scheme that is used for identifying resources stored in Google Cloud Storage Java URL will throw an exception if the protocol is not valid. Then you got that exception, how you might to use a valid URL you might change

 gs://myproject.appspot.com/mybucket/myfile.csv

to

 https://storage.googleapis.com/my-prject-com/mybucket/myfile.svc

then later validate with Java URL if this is a valid URL.

UPDATED

Based what you want to do, then you need to create a custom validator so that this validator can be added different schemes, you might use UrlValidator of Apache Common library. Then you can add custom schmes by following way:

import org.apache.commons.validator.routines.UrlValidator;


    String[] customSchemes = { "gs", "file" };
    UrlValidator customValidator = new UrlValidator(customSchemes);
    customValidator.isValid("gs://www.somwebsite.org");// true
    customValidator.isValid("file:///home/user/Desktop/fileName.png");  // true

JIC, this dependency should be added, if you are using maven :

    <!-- https://mvnrepository.com/artifact/commons-validator/commons-validator -->
    <dependency>
        <groupId>commons-validator</groupId>
        <artifactId>commons-validator</artifactId>
        <version>1.6</version>
    </dependency>

Or If not, then for gradle

    // https://mvnrepository.com/artifact/commons-validator/commons-validator
    compile group: 'commons-validator', name: 'commons-validator', version: '1.6'

Answer 2

One possibility is to ignore the Google GCS "url" prefix and access the files using HTTP. For example:

gs://bucket/file

can be accessed as:

http://storage.googleapis.com/bucket/file

See:

How do you get or generate a URL to the object in a bucket?

This basically side-steps the original notion of thinking of gs://... as anything special.