运行 fat jar 与 sbt run

Question

I have a scala program that runs the following piece of code.

1.    public static HttpCredentialProvider loadHttpCredentialProviders(String name) {
2.       ServiceLoader<HttpCredentialProvider> providers = ServiceLoader.load(HttpCredentialProvider.class, BasicAuthCredentialProvider.class.getClassLoader());
3.        Iterator var2 = providers.iterator();
4.
5.        HttpCredentialProvider provider;
6.        do {
7.            if (!var2.hasNext()) {
8.                throw new ConfigException("HttpCredentialProvider not found for " + name);
9.            }
10.
11.           provider = (HttpCredentialProvider)var2.next();
12.       } while(!provider.getScheme().equalsIgnoreCase(name));
13.
14.       return provider;
15.   }

I will create an uber/fat jar using sbt assembly . Then I will use the command java -jar app.jar . The result is the following exception

Caused by: org.apache.kafka.common.config.ConfigException: HttpCredentialProvider not found for BASIC
    at io.confluent.security.auth.client.provider.BuiltInAuthProviders.loadHttpCredentialProviders(BuiltInAuthProviders.java:56)
    at io.confluent.security.auth.client.rest.RestClient.<init>(RestClient.java:117)
    at io.confluent.security.auth.client.rest.RestClient.<init>(RestClient.java:95)
    at io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler.configure(TokenUserLoginCallbackHandler.java:67)
    at io.confluent.kafka.clients.plugins.auth.token.AbstractTokenLoginCallbackHandler.configure(AbstractTokenLoginCallbackHandler.java:86)
    at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:60)
    at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:105)
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:161)

The above error is thrown from line 8 .

I will execute sbt run and I see no error. The application runs perfectly fine.

Things I tried:

1. Since the difference is sbt run and java -jar perhaps the jar does not contain relevant classes (BasicAuthCredentialProvider). I validated that jar -tf infact does have the classes present. If the class was not present I would suspect a ClassNotFound exception.
2. Remote debug both modes of running the application using JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" sbt run and java -jar -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 app.jar . I idea was to find out what could be actually happening. I got some interesting results here. I stepped through the code and saw that this line

ServiceLoader<HttpCredentialProvider> providers = ServiceLoader.load(HttpCredentialProvider.class, BasicAuthCredentialProvider.class.getClassLoader());

calls the ServiceLoader.load which working with some reflection

public static <S> ServiceLoader<S> load(Class<S> service,
                                            ClassLoader loader)
{
    return new ServiceLoader<>(Reflection.getCallerClass(), service, loader);
}

This is where I am exactly blocked. I want to keeo understanding what is causing the issue but I am not sure how to debug through reflection issues (if that what the problem is). When running sbt run the code executes another function that exists in the same class the above code exists. This function is called loadBasicAuthCredentialProvider . However, that function never gets called by java -jar . This tells me that the reflection is playing some role here.

3. I also switched JAVA version from 8 to 11. No Bueno there either.

If anybody has other ideas I could try to debug this issue it would be great!

Thank you!

Answer 1

I figured it out! The fix was in the merge strategy for assembly. It looked like this:

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs@_*) => MergeStrategy.discard
  case _ => MergeStrategy.first
}

We need to make sure META-INF was added for services. So it needs to look like this.

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", "services", xs@_*) =>
    MergeStrategy.first
  case PathList("META-INF", xs@_*) =>
    MergeStrategy.discard
  case x =>
    MergeStrategy.first
}

I found the answer through KAFKA-8340 !

Observe that the services loaded by that invocation do not include the ones described in the META-INF/services/... file contained in the JAR in the plugin's directory