Livy on K8S, namespace restriction

Question

I have spark (3.0.1), livy (0.8.0) and Jupyterhub (sparkmagic) running on K8S in specific namespace, Kubernetes master is used as a resource manager.

Killing it... 22\/02\/04 12:09:18 WARN InteractiveSession: Error stopping session 2. io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https:\/\/kubernetes.default.svc.cluster.local\/api\/v1\/pods?labelSelector=spark-app-tag%2Cspark-role%3Ddriver%2Cspark-app-selector<\/a> . Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:namespace:livy-acc" cannot list resource "pods" in API group "" at the cluster scope.

Answer 1

During code examination of livy 0.8.0 from this repo: https://github.com/jahstreet/incubator-livy.git --branch merge/first I have discovered some undocumented livy.server.kubernetes.* properties that can be used to configure how livy runs on K8S.

For K8S namespace restriction following properties can be used:

# Comma-separated list of the Kubernetes namespaces to allow for applications creation.
# All namespaces are allowed if empty
livy.server.kubernetes.allowedNamespaces = namespace

# Kubernetes client default namespace
livy.server.kubernetes.defaultNamespace = namespace