“Invalid JWT: Failed audience check” when using Google Api Services in GraalVM native-image

Question

I have a simple application that sends a message to Google Cloud Logging via the HTTP based google-api-services-logging . I was initially using the gRPC cloud-logging library, but couldn't get it to work with GraalVM at all. But unfortunately, I am also struggling with the HTTP variant. The code is working fine when executed on a traditional Java VM, but fails at runtime when running the native image.

java.io.IOException: Error getting access token for service account: 400 Bad Request
POST https://oauth2.googleapis.com/token
{"error":"invalid_grant","error_description":"Invalid JWT: Failed audience check."}
    at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:444)
    at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:157)
    at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:145)
    at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:603)
    at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:91)
    at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:88)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:422)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
    ...
Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request
POST https://oauth2.googleapis.com/token
{"error":"invalid_grant","error_description":"Invalid JWT: Failed audience check."}
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1113)
    at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:441)
    ... 35 more

native-image.properties

Args = \
  --verbose \
  --no-server \
  --no-fallback \
  --static \
  --install-exit-handlers \
  -H:+ReportExceptionStackTraces \
  -H:+TraceClassInitialization \
  -H:+PrintClassInitialization \
  -H:UseMuslC=/musl/ \
  -H:+RemoveSaturatedTypeFlows \
  --enable-https \
  --enable-http \
  --initialize-at-build-time

reflect-config.json

[
  {
    "name": "com.google.api.client.json.GenericJson",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.services.logging.v2.model.LogEntry",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.googleapis.GoogleUtils",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  }
]

resource-config.json

{
  "resources": [
    { "pattern": "^.*\\.json$" },
    { "pattern": "^.*\\.properties$" },
    { "pattern": "^.*\\.jks$" }
  ]
}

app.scala

val scopes = util.Arrays.asList(LoggingScopes.CLOUD_PLATFORM_READ_ONLY, LoggingScopes.LOGGING_WRITE)
val credentials  = ServiceAccountCredentials.fromStream("service-account.json").createScoped(scopes)
val logging = new Logging.Builder(
  transport,
  JacksonFactory.getDefaultInstance,
  new HttpCredentialsAdapter(credentials)
).setApplicationName("my-project").build()

Dockerfile

FROM        oracle/graalvm-ce:20.1.0-java11 as builder
...
RUN         gu install native-image
...
RUN         sbt assembly
RUN         native-image -jar /root/target/scala-2.13/graal-test-assembly-0.1.0-SNAPSHOT.jar

FROM        scratch

WORKDIR     /app/

COPY        --from=builder /root/graal-test-assembly-0.1.0-SNAPSHOT /app/my-native-image

CMD         ["/app/my-native-image"]

I suspect that this is related to crypto / SSL related features, but I ran out of things to try.

Answer 1

It turned out that the generated JWT token was basically empty because the fields are traversed via reflection when serializing to JSON. Adding the respective rules to reflect-config.json solved that issue and revealed further issues that could be solved by configuration.

[
  {
    "name": "com.google.api.client.json.GenericJson",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.json.webtoken.JsonWebToken",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.json.webtoken.JsonWebToken$Header",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.json.webtoken.JsonWebToken$Payload",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.util.GenericData",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.http.UrlEncodedContent",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.json.webtoken.JsonWebSignature$Header",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.json.webtoken.JsonWebSignature",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.services.logging.v2.model.WriteLogEntriesRequest",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.services.logging.v2.model.WriteLogEntriesResponse",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.services.logging.v2.model.LogEntry",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.client.googleapis.json.GoogleJsonError",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  },
  {
    "name": "com.google.api.services.logging.v2.model.MonitoredResource",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true,
    "allPublicFields": true
  }
]

The abuse of reflection in Google's Java libraries is quite a pain. The least they could do is add the GraalVM native-image configurations to their libraries.

You can find a complete reflect-config.json for google cloud logging http here .