Kubernetes 集群中使用 Python 套接字的服务器-客户端连接

Question

I'm trying to make a server-client communication between pods in a K8s cluster using the python socket library.

When running outside of a cluster, the server-client connection works, however in the k8s the server doesn't even set up:

import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("myApp", 30152))  # it breaks here
server_socket.listen()

And here are my configuration YAMLs:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myApp
  labels:
    app: myApp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myApp
  template:
    metadata:
      labels:
        app: myApp
    spec:
      containers:
        - name: myApp
          image: app_image:version
          ports:
            - containerPort: 30152
          imagePullPolicy: Always

---
apiVersion: v1
kind: Service
metadata:
  name: myApp
  labels:
    app: myApp
spec:
  selector:
    app: myApp
  ports:
    - name: myApp
      protocol: TCP
      port: 30152
  type: ClusterIP
---

The service type is a ClusterIP as the connection would only be between pods in the same cluster. Does anyone know where the problem might come from?

Answer 1

Here, I've build a little bit of an example with a client and a server python apps, talking to each other via k8s service. Almost from scratch (clone all the files from here if you want to follow along)

Server

server.py

import socket
import sys
import os

PORT = int(os.getenv('LISTEN_PORT'))

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('0.0.0.0', PORT)
print('Starting up on {} port {}'.format(*server_address))
sock.bind(server_address)
sock.listen()

while True:
    print('\nWaiting for a connection')
    connection, client_address = sock.accept()
    try:
        print('Connection from', client_address)
        while True:
            data = connection.recv(64)
            print('Received {!r}'.format(data))
            if data:
                print('Sending data back to the client')
                connection.sendall(data)
            else:
                print('No data from', client_address)
                break
    finally:
        connection.close()

Dockerfile

FROM python:3-alpine
WORKDIR /app
COPY server.py .
CMD ["/usr/local/bin/python3", "/app/server.py"]

building an image, tagging, pushing to container repo (GCP):

docker build --no-cache -t q69936079-server .
docker tag q69936079-server gcr.io/<project_id>/q69936079-server
docker push gcr.io/<project_id>/q69936079-server

Client

client.py

import socket
import os
import sys
import time

counter = 0

SRV = os.getenv('SERVER_ADDRESS')
PORT = int(os.getenv('SERVER_PORT'))

while 1:
    if counter != 0:
        time.sleep(5)

    counter += 1
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = (SRV, PORT)
    print("Connection #{}".format(counter))
    print('Connecting to {} port {}'.format(*server_address))
    try:
        sock.connect(server_address)
    except Exception as e:
        print("Cannot connect to the server,", e)
        continue

    try:
        message = b'This is the message. It will be repeated.'
        print('Sending:  {!r}'.format(message))
        sock.sendall(message)

        amount_received = 0
        amount_expected = len(message)

        while amount_received < amount_expected:
            data = sock.recv(64)
            amount_received += len(data)
            print('Received: {!r}'.format(data))
    finally:
        print('Closing socket\n')
        sock.close()

Dockerfile

FROM python:3-alpine
WORKDIR /app
COPY client.py .
CMD ["/usr/local/bin/python3", "/app/client.py"]

building an image, tagging, pushing to container repo (GCP in my case):

docker build --no-cache -t q69936079-client .
docker tag q69936079-client gcr.io/<project_id>/q69936079-client
docker push gcr.io/<project_id>/q69936079-client

K8S

server deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
spec:
  selector:
    matchLabels:
      app: server
  replicas: 1
  template:
    metadata:
      labels:
        app: server
    spec:
      containers:
      - name: server
        image: "gcr.io/<project_id>/q69936079-server:latest"
        env:
        - name: PYTHONUNBUFFERED
          value: "1"
        - name: LISTEN_PORT
          value: "30152"

client deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: client-deployment
spec:
  selector:
    matchLabels:
      app: client
  replicas: 1
  template:
    metadata:
      labels:
        app: client
    spec:
      containers:
      - name: client
        image: "gcr.io/<project_id>/q69936079-client:latest"
        env:
        - name: PYTHONUNBUFFERED
          value: "1"
        - name: SERVER_ADDRESS
          value: my-server-service
        - name: SERVER_PORT
          value: "30152"

server service

apiVersion: v1
kind: Service
metadata:
  name: my-server-service
spec:
  type: ClusterIP
  selector:
    app: server
  ports:
  - protocol: TCP
    port: 30152

Validation

k8s object

k get all

NAME                                     READY   STATUS    RESTARTS   AGE
pod/client-deployment-7dd5d675ff-pvwd4   1/1     Running   0          14m
pod/server-deployment-56bd44cc68-w6jns   1/1     Running   0          13m

NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)     AGE
service/kubernetes          ClusterIP   10.140.0.1      <none>        443/TCP     12h
service/my-server-service   ClusterIP   10.140.13.183   <none>        30152/TCP   38m

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/client-deployment   1/1     1            1           14m
deployment.apps/server-deployment   1/1     1            1           13m

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/client-deployment-7dd5d675ff   1         1         1       14m
replicaset.apps/server-deployment-56bd44cc68   1         1         1       13m

Server logs

k logs -f deployment.apps/server-deployment

Starting up on 0.0.0.0 port 30152

Waiting for a connection
Connection from ('10.136.1.11', 48234)
Received b'This is the message. It will be repeated.'
Sending data back to the client
Received b''
No data from ('10.136.1.11', 48234)

Waiting for a connection
Connection from ('10.136.1.11', 48246)
Received b'This is the message. It will be repeated.'
Sending data back to the client
Received b''
No data from ('10.136.1.11', 48246)

Client logs

k logs -f deployment.apps/client-deployment

Connection #1
Connecting to my-server-service port 30152
Cannot connect to the server, [Errno 111] Connection refused

Connection #2
Connecting to my-server-service port 30152
Cannot connect to the server, [Errno 111] Connection refused

Connection #3
Connecting to my-server-service port 30152
Sending:  b'This is the message. It will be repeated.'
Received: b'This is the message. It will be repeated.'
Closing socket

Connection #4
Connecting to my-server-service port 30152
Sending:  b'This is the message. It will be repeated.'
Received: b'This is the message. It will be repeated.'
Closing socket