How do I include a resource file in Java project to be used with just new File()?

Question

I'm writing a UDF for Pig using Java. It works fine but Pig doesn't give me options to separate environment. What my Pig script is doing is to get Geo location from IP address.

Here's my code on the Geo location part.

private static final String GEO_DB = "GeoLite2-City.mmdb";
private static final String GEO_FILE = "/geo/" + GEO_DB;

 public Map<String, Object> geoData(String ipStr) {
        Map<String, Object> geoMap = new HashMap<String, Object>();

        DatabaseReader reader = new DatabaseReader.Builder(new File(GEO_DB)).build();
            // other stuff
    }

GeoLite2-City.mmdb exists in HDFS that's why I can refer from absolute path using /geo/GeoLite2-City.mmdb .

However, I can't do that from my JUnit test or I have to create /geo/GeoLite2-City.mmdb on my local machine and Jenkins which is not ideal. I'm trying to figure out a way to make my test passed while using new File(GEO_DB) and not getClass().getResourceAsStream('./geo/GeoLite2-City.mmdb') because

getClass().getResourceAsStream('./geo/GeoLite2-City.mmdb')

Doesn't work in Hadoop.

And if I run Junit test it would fail because I don't have /geo/GeoLite2-City.mmdb on my local machine.

Is there anyway I can overcome this? I just want my tests to pass without changing the code to be using getClass().getResourceAsStream and I can't if/else around that because Pig doesn't give me a way to pass in parameter or maybe I'm missing something.

And this is my JUnit test

@Test
@Ignore
public void shouldGetGeoData() throws Exception {
    String ipTest = "128.101.101.101";

    Map<String, Object> geoJson = new LogLine2Json().geoData(ipTest);

    assertThat(geoJson.get("lLa").toString(), is(equalTo("44.9759")));
    assertThat(geoJson.get("lLo").toString(), is(equalTo("-93.2166")));

}

which it works if I read the database file from resource folder. That's why I have @Ignore

Answer 1

Besides, your whole code looks pretty un-testable.

Every time when you directly call new in your production code, you prevent dependency injection; and thereby you make it much harder to test your code.

The point is to not call new File() within your production code. Instead, you could use a factory that gives you a "ready to use" DatabaseReader object. Then you can test your factory to do the right thing; and you can mock that factory when testing this code (to return a mocked database reader).

So, that one file instance is just the top of your "testing problems" here.

Honestly: don't write production code first. Do TDD: write test cases first; and you will quickly learn that such production code that you are presenting here is really hard to test. And when you apply TDD, you start from "test perspective", and you will create production code that is really testable.

Answer 2

You have to make the file location configurable. Eg inject it via constructor. Eg you could create a non-default constructor for testing only.

public class LogLine2Json {
  private static final String DEFAULT_GEO_DB = "GeoLite2-City.mmdb";
  private static final String DEFAULT_GEO_FILE = "/geo/" + GEO_DB;

  private final String geoFile;

  public LogLine2Json() {
    this(DEFAULT_GEO_FILE);
  }

  LogLine2Json(String geoFile) {
    this.geoFile = geoFile;
  }

  public Map<String, Object> geoData(String ipStr) {
    Map<String, Object> geoMap = new HashMap<String, Object>();

    File file = new File(geoFile);
    DatabaseReader reader = new DatabaseReader.Builder(file).build();
    // other stuff
  }
}

Now you can create a file from the resource and use this file in your test.

public class LogLine2JsonTest {
    @Rule
    public final TemporaryFolder folder = new TemporaryFolder();

    @Test
    public void shouldGetGeoData() throws Exception {
      File dbFile = copyResourceToFile("/geo/GeoLite2-City.mmdb");
      String ipTest = "128.101.101.101";

      LogLine2Json logLine2Json = new LogLine2Json(dbFile.getAbsolutePath())
      Map<String, Object> geoJson = logLine2Json.geoData(ipTest);

      assertThat(geoJson.get("lLa").toString(), is(equalTo("44.9759")));
      assertThat(geoJson.get("lLo").toString(), is(equalTo("-93.2166")));
    }

    private File copyResourceToFile(String name) throws IOException {
      InputStream resource = getClass().getResourceAsStream(name);
      File file = folder.newFile();
      Files.copy(resource, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
      return file;
    }
}

TemporaryFolder is a JUnit rule that deletes every file that is created during test afterwards.

You may modify the asserts by using the hasToString matcher. This will give you more detailed information in case of a failing test. (And you have to read/write less code.)

assertThat(geoJson.get("lLa"), hasToString("44.9759"));
assertThat(geoJson.get("lLo"), hasToString("-93.2166"));

Answer 3

You don't. Your question embodies a contradiction in terms. Resources are not files and do not live in the file system. You can either distribute the file separately from the JAR and use it as a File or include it in the JAR and use it as a resource. Not both. You have to make up your mind.