Elastic APM Error | Google Kubernetes Engine

Question

Am trying to run Elastic APM in GKE cluster. I have installed elastic-search , kibana and apm-server . All services are up and running. All these components has been through helm charts. Below are the configuration for each component.

apmConfig:
  apm-server.yml: |
    apm-server:
      host: "0.0.0.0:8200"

    queue: {}

    output.elasticsearch:
      hosts: ["http://elasticsearch-master.monitoring.svc.cluster.local:9200"]
      username: "elastic"
      password: "password"

kibanaConfig: 
  kibana.yml: |
    server.host: 0.0.0.0
    server.port: 5601
    elasticsearch.hosts: "http://elasticsearch-master.monitoring.svc.cluster.local:9200"
    kibana.index: ".kibana"
    server.basePath: "/kibana"
    server.rewriteBasePath: true
    server.publicBaseUrl: "https://mydomain/kibana"
    elasticsearch:
      username: "kibana_system"
      password: "password"

I have tried to add APM integration to one of my service by using the below config:

var apm = require('elastic-apm-node').start({

    // Override the service name from package.json
    // Allowed characters: a-z, A-Z, 0-9, -, _, and space
    serviceName: 'shopping',
    
    // Use if APM Server requires a secret token
    secretToken: '',
    
    // Set the custom APM Server URL (default: http://localhost:8200)
    serverUrl: 'https://mydomain/apm',
    
    // Set the service environment
    environment: 'production'
    })

When I start the service, I get the below error in logs:

{"log.level":"error","@timestamp":"2022-08-18T10:08:31.584Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error (301): Unexpected APM Server response"}

301 - Moved permanently. It would be great, if I could get any help?

Answer 1

The problem have been solved. The reason was because I was using a proxy and hitting port 80, while in the APM server service I was using port as 8200.

After changing the port to 80 and targetPort to 8200 value in apm server service, I was able to correctly instrument Elastic APM with my services.