Unable to access Kibana dashboard service in kubernetes cluser on Ubuntu

Question

I am trying to access the Kibana dashboard while trying to set up fluentd-elasticsearch on premises. This is the link which I followed. I checked the logs of Kibana's pod. It shows the following error:

{"type":"log","@timestamp":"2018-09-19T21:45:42Z","tags":["warning","config","deprecation"],"pid":1,"message":"You should set server.basePath along with server.rewriteBasePath. Starting in 7.0, Kibana will expect that all requests start with server.basePath rather than expecting you to rewrite the requests in your reverse proxy. Set server.rewriteBasePath to false to preserve the current behavior and silence this warning."}
root@mTrainer3:/logging# kubectl logs kibana-logging-66d577d965-mbbg5 -n kube-system
{"type":"log","@timestamp":"2018-09-19T21:45:42Z","tags":["warning","config","deprecation"],"pid":1,"message":"You should set server.basePath along with server.rewriteBasePath. Starting in 7.0, Kibana will expect that all requests start with server.basePath rather than expecting you to rewrite the requests in your reverse proxy. Set server.rewriteBasePath to false to preserve the current behavior and silence this warning."}
{"type":"log","@timestamp":"2018-09-19T21:46:08Z","tags":["status","plugin:kibana@6.4.1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}

Could anybody suggest how I can resolve this issue?

Answer 1

After a discussion it was more clear what seems to be wrong.

You are using a local cluster with no load balancer. You have to set either an ingress or use NodePort as the service type. I am going to describe the solution with NodePort. Two steps to take:

1. Modify the kibana-deployment.yaml and remove the following under env :

- name: SERVER_BASEPATH
  value: /api/v1/namespaces/kube-system/services/kibana-logging/proxy

so that you kibana-deployment.yaml looks like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana-logging
  namespace: kube-system
  labels:
    k8s-app: kibana-logging
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  replicas: 1
  selector:
  matchLabels:
    k8s-app: kibana-logging
  template:
    metadata:
      labels:
        k8s-app: kibana-logging
      annotations:
        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      containers:
      - name: kibana-logging
        image: docker.elastic.co/kibana/kibana-oss:6.3.2
        resources:
          # need more cpu upon initialization, therefore burstable class
          limits:
            cpu: 1000m
          requests:
            cpu: 100m
        env:
          - name: ELASTICSEARCH_URL
            value: http://elasticsearch-logging:9200
        ports:
        - containerPort: 5601
          name: ui
          protocol: TCP

2. Modify kibana-service.yaml to set the service type to NodePort:

apiVersion: v1
kind: Service
metadata:
  name: kibana-logging
  namespace: kube-system
  labels:
    k8s-app: kibana-logging
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "Kibana"
spec:
  type: NodePort
  ports:
  - port: 5601
    protocol: TCP
    targetPort: ui
    nodePort: 30601
  selector:
    k8s-app: kibana-logging

Then execute

kubectl apply -f kibana-deployment.yaml
kubectl apply -f kibana-service.yaml

Kibana should be accessible at http://<clusterip>:30601

Background

You will directly access http://clusterip:30601 without the given base path. So this must be removed, so that kibana is using / as base path. Otherwise it will try to append the base path /api/v1/[...] to your url. You can try it if you want to test it.

This comment from an elastic search guy mentions , that you have to remove the base_path completely if you want to use / .

Modifying the service to NodePort is neccessary as K8s does not publish ports by default. I just answered a similar issue on this question .

Original answer (wrong)

In the repo you were linking I can see that the kibana-deployment.yaml has to environment variables to set:

env:
  - name: ELASTICSEARCH_URL
    value: http://elasticsearch-logging:9200
  - name: SERVER_BASEPATH
    value: /api/v1/namespaces/kube-system/services/kibana-logging/proxy

Did you set them accordingly?

Let's assume you have an ingress, loadbalancer or NodePort directly to the kibana instance so that you want to reach it directly with http://yourserver:9200/ . Then the SERVER_BASEPATH is /

Answer 2

Alternatively, you can also specify Kibana to rewrite server basepath by specifying an environment variable.

Modify kibana-deployment.yaml to add the following.

- name: SERVER_REWRITEBASEPATH
  value: "true"

Now apply: kubectl apply -f kibana-deployment.yaml

Tested this on microk8s and works well with kubectl proxy

http://127.0.0.1:8001/api/v1/namespaces/kube-system/services/kibana-logging/proxy/app/kibana