prometheus is not scraping the metrics from the app deployed on kubernetes

Question

I have added endpoint /metrics in my app and this is the config file i have written for prometheus

kind: ConfigMap
metadata:
  name: prometheus-config
apiVersion: v1
data:
  prometheus.yml: |-
    global:
      scrape_interval: 5s
      evaluation_interval: 5s
    scrape_configs:
    - job_name: 'goserver'
      scheme: http
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      kubernetes_sd_configs:
      - api_server: https://kubernets.default.svc
        role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_app]
        regex: goserver
        action: keep
      - source_labels: [__meta_kubernetes_pod_name]
        target_label: k8s_pod
        action: replace

and below is the deployment file

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: prometheus-deployment
spec:
  replicas: 1
  template:
  metadata:
    labels:
      app: prometheus-server
  spec:
    containers:
    - name: prometheus
      image: prom/prometheus:latest
      args:
        - "-config.file=/etc/prometheus/conf/prometheus.yml"
        # Metrics are stored in an emptyDir volume which
        # exists as long as the Pod is running on that Node.
        # The data in an emptyDir volume is safe across container crashes.
        - "-storage.local.path=/prometheus"
      ports:
        - containerPort: 9090
      volumeMounts:
        - name: prometheus-server-volume
          mountPath: /etc/prometheus/conf
        - name: prometheus-storage-volume
          mountPath: /prometheus
  volumes:
    - name: prometheus-server-volume
      configMap:
        name: prometheus-config
    - name: prometheus-storage-volume
      emptyDir: {} # containers in the Pod can all read and write the same files here.

and this is the deployment file for the app

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: server-deployment
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
         app: goserver
    spec:
      containers:
      - name: goserver
        imagePullPolicy: Always
        image: utkarshmani1997/goserver:v1.4
        ports:
        - containerPort: 8080

I have created the service of prometheus and goserver on port 32514 and 32520 respectively of the type NodePort. This is working fine with the minikube and scrapes the metrics from my app but not working with the kubeadm setup in virtual box (multinode cluster). What changes i have to make in order to get the metrics on kubernetes. I tried to make few changes in rbac-setup.yaml and prometheus-kubernetes.yaml also but it is not working.

I am accessing the browser by ssh -NL 1234:localhost:32514 from outside the cluster,because vagrant box does not have ubuntu desktop ui.

Answer 1

i found the issue,actually the issue was related to rbac. I didn't added the serviceAccountName in deployment file. Now it is working fine with the new change.