Cannot create deployment of nginx tasks definitions on AWS Fargate using virtual-kubelet

Question

I am unable to to deploy nginx containers using kubectl to AWS Fargate using virtual-kubelet. I am following this guide: https://aws.amazon.com/blogs/opensource/aws-fargate-virtual-kubelet/ . I am having an issue with Step 6: Create Kubernetes objects. I would like to know why the nginx containers are PENDING and why the AWS Fargate task definitions have not been created.

The following is some of my commands I used. I can give more detail upon request.

# ./virtual-kubelet --provider aws --provider-config fargate.toml
...
2019/05/16 06:50:24 Received NodeDaemonEndpoints request.
ERRO[0000] TLS certificates not provided, not setting up pod http server  certPath= keyPath= node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
INFO[0000] Initialized                                   node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
INFO[0000] Created node                                  node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
INFO[0000] Node leases not supported, falling back to only node status updates  node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
INFO[0000] Pod cache in-sync                             node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
2019/05/16 06:50:25 Received GetPods request.
2019/05/16 06:50:25 Responding to GetPods: [].
INFO[0000] starting workers                              node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=
INFO[0000] started workers                               node=virtual-kubelet operatingSystem=Linux provider=aws watchedNamespace=

# kubectl describe node virtual-kubelet
Events:
  Type     Reason               Age                   From              Message
  ----     ------               ----                  ----              -------
  Warning  FailedToCreateRoute  98s (x951 over 160m)  route_controller  (combined from similar events): Could not create route e1e32758-77a6-11e9-a68e-0a95bb07bfa2 100.96.4.0/24 for node virtual-kubelet after 47.871544ms: instance not found

# kubectl get nodes
NAME                                          STATUS   ROLES    AGE   VERSION
ip-172-20-47-10.eu-west-2.compute.internal    Ready    master   30h   v1.14.1
ip-172-20-47-242.eu-west-2.compute.internal   Ready    node     30h   v1.14.1
ip-172-20-59-102.eu-west-2.compute.internal   Ready    node     30h   v1.14.1
virtual-kubelet                               Ready    agent    33m   v1.13.1-vk-v0.9.0-40-g5b3190ac-dev

kubectl create -f nginx-deployment.yaml
# kubectl get deployments -o wide

# kubectl get pods -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
nginx-deployment-c6695csfc-5f7bh    0/1     Pending   0          21m   <none>   <none>   <none>           <none>
nginx-deployment-c6695csfc-bwfb8    0/1     Pending   0          21m   <none>   <none>   <none>           <none>
nginx-deployment-c6695csfc-mcfvw    0/1     Pending   0          21m   <none>   <none>   <none>           <none>

# kubectl describe pod nginx-deployment-c6695csfc-5f7bh

Events:
  Type     Reason            Age                    From               Message
  ----     ------            ----                   ----               -------

Warning  FailedScheduling  2m11s (x191 over 22m)  default-scheduler  0/4 nodes are available: 1 Insufficient cpu, 1 node(s) had taints that the pod didn't tolerate, 3 node(s) didn't match node selector.

Update:

I then ran the command to add the nodeSelector to my nodes using the following command for each node:

kubectl label nodes ip-172-20-47-15.eu-west-2.compute.internal type=virtual-kubelet

type=virtual-kubelet is the nodeSelector specified in the manifest file, nginx-deployment.yaml.

# kubectl get pods -o wide
NAME                               READY   STATUS    RESTARTS   AGE     IP           NODE                                          NOMINATED NODE   READINESS GATES
nginx-deployment-c6695csfc-5f7bh   1/1     Running   0          4m59s   100.96.2.7   ip-172-20-47-242.eu-west-2.compute.internal   <none>           <none>
nginx-deployment-c6695csfc-bwfb8   1/1     Running   0          4m59s   100.96.1.6   ip-172-20-59-102.eu-west-2.compute.internal   <none>           <none>
nginx-deployment-c6695csfc-mcfvw  1/1     Running   0          4m59s   100.96.2.8   ip-172-20-47-242.eu-west-2.compute.internal   <none>

Now when I go to the AWS Fargate Dashboard the associated task definitions are not created as shown in the tutorial.

Answer 1

This issue is resolved. I was able to create the AWS Fargate definitions by adding the ALB Security group to the fargate.toml file and by adding tolerations to the nginx.deployment.yaml file as shown below:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
    tolerations:
    - key: virtual-kubelet.io/provider
      operator: Equal
      value: azure
      effect: NoSchedule