使用 postgresql 在 K8s 中使用超集的配置问题

Question

I am trying to install superset using k8s with PostgreSQL as backend database and redis as cache layer.

Superset is up and running but when i try to run SQL queries, I am getting the following error

column dbs.server_cert does not exist at character 814

Below are my config files and helm templates. Any help is appreciated, thank you!

# Source: superset/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: superset
  labels:
    app: superset
    chart: superset-1.1.11
    release: superset
    heritage: Helm
    namespace: test
type: Opaque
data:
  superset_config.py: "attached seperately"
---

# Source: superset/templates/svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: superset
  labels:
    app: superset
    chart: superset-1.1.11
    release: superset
    heritage: Helm
    namespace: test
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 9000
      targetPort: 8088
      protocol: TCP
  selector:
    app: superset
    release: superset
---
# Source: superset/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: superset 
  labels:
    app: superset
    chart: superset-1.1.11
    release: superset
    heritage: Helm
    namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: superset
      release: superset
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      name: superset
      labels:
        app: superset
        chart: superset-1.1.11
        release: superset
        heritage: Helm
      annotations:
        checksum/secrets: 6dd9c87c415d8a5a00981f36d8dca6ad69a2fdeae29d0caa3a02633be4acff07
    spec:
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
      volumes:
        - name: superset-configs
          secret:
            secretName: superset
        - name: storage-volume
          persistentVolumeClaim:
            claimName: superset-storage
      containers:
        - env:
          - name: PYTHONPATH
            value: "${PYTHONPATH}:/home/superset/superset/"
          image: amancevice/superset:0.36.0
          name: superset
          volumeMounts:
          - name: superset-configs
            mountPath: /home/superset/superset
          - name: storage-volume
            mountPath: /var/lib/superset
          ports:
          - name: http
            containerPort: 8088
            protocol: TCP
          livenessProbe:
            failureThreshold: 2
            httpGet:
              path: /health
              port: http
            initialDelaySeconds: 80
            periodSeconds: 10
            timeoutSeconds: 5
          readinessProbe:
            failureThreshold: 2
            httpGet:
              path: /health
              port: http
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
---

 #PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: superset-storage
  namespace: test
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

superset config file

import os

def get_env_variable(var_name, default=None):
    """Get the environment variable or raise exception.

    Args:
        var_name (str): the name of the environment variable to look up
        default (str): the default value if no env is found
    """
    try:
        return os.environ[var_name]
    except KeyError:
        if default is not None:
            return default
        raise RuntimeError(
            'The environment variable {} was missing, abort...'
            .format(var_name)
        )

# Postgres

POSTGRES_USER =  '**'
POSTGRES_PASSWORD = '**'
POSTGRES_HOST = '**'
POSTGRES_PORT = '**'
POSTGRES_DB = '**'

SQLALCHEMY_DATABASE_URI = 'postgresql://{0}:{1}@{2}:{3}/{4}'.format(
    POSTGRES_USER,
    POSTGRES_PASSWORD,
    POSTGRES_HOST,
    POSTGRES_PORT,
    POSTGRES_DB,
)


# Redis

REDIS_HOST = '**' 
REDIS_PORT = '**'


# Celery

class CeleryConfig:
    BROKER_URL = 'redis://{0}:{1}/0'.format(REDIS_HOST, REDIS_PORT)
    CELERY_IMPORTS = ('superset.sql_lab',)
    CELERY_RESULT_BACKEND = 'redis://{0}:{1}/1'.format(REDIS_HOST, REDIS_PORT)
    CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
    CELERY_TASK_PROTOCOL = 1


CELERY_CONFIG = CeleryConfig

FLASK_APP=superset

ERROR

 sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column dbs.server_cert does not exist
LINE 1: ...rm, dbs.impersonate_user AS dbs_impersonate_user, dbs.server...
                                                             ^

[SQL: SELECT dbs.created_on AS dbs_created_on, dbs.changed_on AS dbs_changed_on, dbs.id AS dbs_id, dbs.verbose_name AS dbs_verbose_name, dbs.database_name AS dbs_database_name, dbs.sqlalchemy_uri AS dbs_sqlalchemy_uri, dbs.password AS dbs_password, dbs.cache_timeout AS dbs_cache_timeout, dbs.select_as_create_table_as AS dbs_select_as_create_table_as, dbs.expose_in_sqllab AS dbs_expose_in_sqllab, dbs.allow_run_async AS dbs_allow_run_async, dbs.allow_csv_upload AS dbs_allow_csv_upload, dbs.allow_ctas AS dbs_allow_ctas, dbs.allow_dml AS dbs_allow_dml, dbs.force_ctas_schema AS dbs_force_ctas_schema, dbs.allow_multi_schema_metadata_fetch AS dbs_allow_multi_schema_metadata_fetch, dbs.extra AS dbs_extra, dbs.encrypted_extra AS dbs_encrypted_extra, dbs.perm AS dbs_perm, dbs.impersonate_user AS dbs_impersonate_user, dbs.server_cert AS dbs_server_cert, dbs.created_by_fk AS dbs_created_by_fk, dbs.changed_by_fk AS dbs_changed_by_fk 
FROM dbs ORDER BY dbs.changed_on DESC 
 LIMIT %(param_1)s]

After applying this configuration, I did an exec into the pod and ran the following commands:

1. superset db upgrade
2. superset init

Answer 1

It seems the migrations have not completed successfully. Try executing the following inside the superset container

superset db upgrade

in order to perform the migrations.

Although, you shouldn't have to manually execute commands in the container to get it working. The superset helm chart accepts an initFile as a value to achieve this without having to manually exec into the container.