Running JUnit Test on AWS Lambda: java.lang.Exception: No runnable methods

Question

I have written code to download a test case and run it in AWS Lambda on the Java 8 environment.

private static Class<?> loadClass(String className) throws ClassNotFoundException, MalformedURLException {
    // Load compiled class.
    File root = new File("...");
    URLClassLoader classLoader = URLClassLoader.newInstance(new URL[]{root.toURI().toURL()});
    return Class.forName(className, true, classLoader);
}

public static String run(RunRequest req) throws IOException {
    final ByteArrayOutputStream baos = new ByteArrayOutputStream();
    PrintStream ps = new PrintStream(baos, true, "UTF-8");

    try {
        // Download test case
        CloudStorage.downloadChallenge(req.getChallengeId());
        // Get test case class
        String className = Database.getChallengeTestClass(req.getChallengeId());
        // Load test case
        Class<?> test = loadClass(className);

        System.out.println("Test: " + test.getName());

        // Run test case
        JUnitCore junit = new JUnitCore();
        junit.addListener(new TextListener(ps));

        junit.run(test);
    } catch (Exception e) {
        e.printStackTrace(ps);
    } finally {
        return new String(baos.toByteArray(), StandardCharsets.UTF_8);
    }
}

When I test this code locally on my machine, it executes perfectly and I get the following output from JUnit:

Time: 0.002

OK (1 test)

However, when I run the same code on Lambda, it yields the following error:

Time: 0.014
There was 1 failure:
1) initializationError(com.test.util.UnitTest)
java.lang.Exception: No runnable methods
    at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:191)
    at org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:128)
    at org.junit.runners.ParentRunner.validate(ParentRunner.java:416)
    at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
    at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
    ...

FAILURES!!!
Tests run: 1,  Failures: 1

Any ideas why this would happen? I am running junit 4.12 .

Also, the test case being run is as follows:

package com.test.util;

import junit.framework.TestCase;

public class UnitTest {
  public UnitTest() {}

  @org.junit.Test
  public void test() {
    System.out.println("Runnning test...");

    TestCase.assertTrue(Runner.userCode.toString().equals("Hello World!"));
  }
}

The class is being imported correctly, as it prints com.test.util.UnitTest

Answer 1

I figured out the problem.

What's happening is that I am referencing another class Runner with the static variable userCode but I am loading this class with a new ClassLoader. For this ClassLoader, the class Runner does not exist, so the method referencing this static variable isn't being loaded by Java. It had nothing to do with Lambda. I figured this out from this thread which explains some of the behavior of the Java ClassLoader.

The only question I have is why was it working locally on my machine? I can't explain the inconsistency here.

Thanks!