istio is failing to install in a Kubernetes cluster built via Kops in AWS

Question

I can't get the demo profile to work with istioctl. It seems like istioctl is having trouble creating IngressGateway and the AddonComponents. I have tried doing the helm installation with similar issues. I did a fresh k8s cluster from kops and the same issue. Any help debugging this issue would be greatly appreciated.

I am following these instructions. https://istio.io/docs/setup/getting-started/#download

I am running

 istioctl manifest apply --set profile=demo --logtostderr

This is the output

2020-04-06T19:59:24.951136Z info    Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.
- Applying manifest for component Base...
✔ Finished applying manifest for component Base.
- Applying manifest for component Pilot...
✔ Finished applying manifest for component Pilot.
- Applying manifest for component IngressGateways...
- Applying manifest for component EgressGateways...
- Applying manifest for component AddonComponents...
✔ Finished applying manifest for component EgressGateways.
2020-04-06T20:00:11.501795Z error   installer   error running kubectl: exit status 1
✘ Finished applying manifest for component AddonComponents.
2020-04-06T20:00:40.418396Z error   installer   error running kubectl: exit status 1
✘ Finished applying manifest for component IngressGateways.
2020-04-06T20:00:40.421746Z info    
Component AddonComponents - manifest apply returned the following errors:
2020-04-06T20:00:40.421823Z info    Error: error running kubectl: exit status 1

2020-04-06T20:00:40.421884Z info    Error detail:


Error from server (Timeout): error when creating "STDIN": Timeout: request did not complete within requested timeout 30s (repeated 1 times)


clusterrole.rbac.authorization.k8s.io/kiali unchanged
clusterrole.rbac.authorization.k8s.io/kiali-viewer unchanged
clusterrole.rbac.authorization.k8s.io/prometheus-istio-system unchanged
clusterrolebinding.rbac.authorization.k8s.io/kiali unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus-istio-system unchanged

serviceaccount/kiali-service-account unchanged
serviceaccount/prometheus unchanged
configmap/istio-grafana unchanged
configmap/istio-grafana-configuration-dashboards-citadel-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-galley-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-istio-mesh-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-istio-performance-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-istio-service-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-istio-workload-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-mixer-dashboard unchanged
configmap/istio-grafana-configuration-dashboards-pilot-dashboard unchanged
configmap/kiali configured
configmap/prometheus unchanged
secret/kiali unchanged
deployment.apps/grafana unchanged
deployment.apps/istio-tracing unchanged
deployment.apps/kiali unchanged
deployment.apps/prometheus unchanged
service/grafana unchanged
service/jaeger-agent unchanged
service/jaeger-collector unchanged
service/jaeger-collector-headless unchanged
service/jaeger-query unchanged
service/kiali unchanged
service/prometheus unchanged
service/tracing unchanged
service/zipkin unchanged


2020-04-06T20:00:40.421999Z info    
Component IngressGateways - manifest apply returned the following errors:
2020-04-06T20:00:40.422056Z info    Error: error running kubectl: exit status 1

2020-04-06T20:00:40.422096Z info    Error detail:

Error from server (Timeout): error when creating "STDIN": Timeout: request did not complete within requested timeout 30s (repeated 2 times)


serviceaccount/istio-ingressgateway-service-account unchanged
deployment.apps/istio-ingressgateway configured
poddisruptionbudget.policy/ingressgateway unchanged
role.rbac.authorization.k8s.io/istio-ingressgateway-sds unchanged
rolebinding.rbac.authorization.k8s.io/istio-ingressgateway-sds unchanged
service/istio-ingressgateway unchanged


2020-04-06T20:00:40.422134Z info    

✘ Errors were logged during apply operation. Please check component installation logs above.

Error: failed to apply manifests: errors were logged during apply operation

I ran the below to verify install before running the above commands.

istioctl verify-install

Checking the cluster to make sure it is ready for Istio installation...

#1. Kubernetes-api
-----------------------
Can initialize the Kubernetes client.
Can query the Kubernetes API Server.

#2. Kubernetes-version
-----------------------
Istio is compatible with Kubernetes: v1.16.7.

#3. Istio-existence
-----------------------
Istio will be installed in the istio-system namespace.

#4. Kubernetes-setup
-----------------------
Can create necessary Kubernetes configurations: Namespace,ClusterRole,ClusterRoleBinding,CustomResourceDefinition,Role,ServiceAccount,Service,Deployments,ConfigMap. 

#5. SideCar-Injector
-----------------------
This Kubernetes cluster supports automatic sidecar injection. To enable automatic sidecar injection see https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/#deploying-an-app

Answer 1

As mentioned in your logs

2020-04-06T19:59:24.951136Z info Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT.

---

As mentioned here

To determine if your cluster supports third party tokens, look for the TokenRequest API:

$ kubectl get --raw /api/v1 | jq '.resources[] | select(.name | index("serviceaccounts/token"))'
{
    "name": "serviceaccounts/token",
    "singularName": "",
    "namespaced": true,
    "group": "authentication.k8s.io",
    "version": "v1",
    "kind": "TokenRequest",
    "verbs": [
        "create"
    ]
}

While most cloud providers support this feature now, many local development tools and custom installations may not. To enable this feature, please refer to the Kubernetes documentation .

---

To authenticate with the Istio control plane, the Istio proxy will use a Service Account token. Kubernetes supports two forms of these tokens:

Third party tokens, which have a scoped audience and expiration. First party tokens, which have no expiration and are mounted into all pods. Because the properties of the first party token are less secure, Istio will default to using third party tokens. However, this feature is not enabled on all Kubernetes platforms.

If you are using istioctl to install, support will be automatically detected. This can be done manually as well, and configured by passing --set values.global.jwtPolicy=third-party-jwt or --set values.global.jwtPolicy=first-party-jwt.

---

If that won't work I would open a new github issue, or add a comment here as issue with installation is similar.