Cloud composer running Docker image inside Kubernetes pod using KubernetesPodOperator

Question

I want to run a docker inside Kube.netes pod via Cloud composer, so to do so I followed all the mentioned steps here and also verified the binding, I have used default service account along with default namespace of Kube.netes pod.

I am trying to access the Form API in cloud composer--> Kube.netesPodOperator --> Docker image(Application accessing the Google Form API) This can pull image and even run it, but, when it tries to access form API, Getting the following error:

[2022-09-23 16:17:49,839] {pod_manager.py:197} INFO - java.io.IOException: Unexpected Error code 403 trying to get security access token from Compute Engine metadata for the default service account: Missing required header "Metadata-Flavor": "Google"
[2022-09-23 16:17:49,840] {pod_manager.py:197} INFO - 
[2022-09-23 16:17:49,840] {pod_manager.py:197} INFO -   at com.google.auth.oauth2.ComputeEngineCredentials.refreshAccessToken(ComputeEngineCredentials.java:75) ~[cli.jar:?]
[2022-09-23 16:17:49,841] {pod_manager.py:197} INFO -   at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:76) ~[cli.jar:?]
[2022-09-23 16:17:49,841] {pod_manager.py:197} INFO -   at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:53) ~[cli.jar:?]
[2022-09-23 16:17:49,841] {pod_manager.py:197} INFO -   at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:40) ~[cli.jar:?]
[2022-09-23 16:17:49,842] {pod_manager.py:197} INFO -   at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:91) ~[cli.jar:?]
[2022-09-23 16:17:49,842] {pod_manager.py:197} INFO -   at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:415) ~[cli.jar:?]
[2022-09-23 16:17:49,843] {pod_manager.py:197} INFO -   at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:525) ~[cli.jar:?]
[2022-09-23 16:17:49,844] {pod_manager.py:197} INFO -   at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:466) ~[cli.jar:?]
[2022-09-23 16:17:49,845] {pod_manager.py:197} INFO -   at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:576) ~[cli.jar:?] 

DAG snippet:

kubernetes_min_pod = KubernetesPodOperator(
        task_id='google-form-importer',
        name='google-form-importer',
        namespace='default',
        service_account_name='default',
        image='pulling to be pulled',
        arguments=["An_argument"],
        get_logs=True,
        image_pull_policy='Always',
        log_events_on_failure=True,
        env_vars= {
'ALL_ENVS':'ItsValue'
        }
) 

Thanks for the help!

Answer 1

Updating to latest API dependencies resolved the error:

    <dependency>
      <groupId>com.google.api-client</groupId>
      <artifactId>google-api-client</artifactId>
      <version>2.0.0</version>
    </dependency>
    <dependency>
      <groupId>com.google.apis</groupId>
      <artifactId>google-api-services-forms</artifactId>
      <version>v1-rev20220329-2.0.0</version>
    </dependency>
    <dependency>
      <groupId>com.google.auth</groupId>
      <artifactId>google-auth-library-oauth2-http</artifactId>
      <version>1.11.0</version>
    </dependency>

Previous versions were 1.35.2, v1-rev20220329-1.32.1, 0.1.0 respectively.