无法使用 NodePort 服务连接到 kubernetes 中的 redis pod

Question

我对 kubernetes 相当陌生，我正在尝试使用 MacBook 上的 minikube 编排我的 rails 应用程序。 我的应用程序包括 MySQL、Redis 和 Sidekiq。 我在隔离的 pod 中运行 webapp、sidekiq、redis 和数据库。 Sidekiq pod 未连接到 redis pod。

sidekiq pod 的 kubectl 日志是这样说的：

2020-09-15T14:01:16.978Z 1 TID-gnaz4yzs0 INFO: Booting Sidekiq 4.2.10 with redis options {:url=>"redis://redis:6379/0"}
2020-09-15T14:01:18.475Z 1 TID-gnaz4yzs0 INFO: Running in ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
2020-09-15T14:01:18.475Z 1 TID-gnaz4yzs0 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2020-09-15T14:01:18.475Z 1 TID-gnaz4yzs0 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:459: warning: constant ::Fixnum is deprecated
Error connecting to Redis on redis:6379 (Errno::ECONNREFUSED)
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:345:in `rescue in establish_connection'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:330:in `establish_connection'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:101:in `block in connect'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:293:in `with_reconnect'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:100:in `connect'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:364:in `ensure_connected'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:221:in `block in process'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:306:in `logging'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:220:in `process'
/usr/local/bundle/gems/redis-3.3.1/lib/redis/client.rb:120:in `call'
/usr/local/bundle/gems/redis-3.3.1/lib/redis.rb:251:in `block in info'
/usr/local/bundle/gems/redis-3.3.1/lib/redis.rb:58:in `block in synchronize'
/usr/local/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/usr/local/bundle/gems/redis-3.3.1/lib/redis.rb:58:in `synchronize'
/usr/local/bundle/gems/redis-3.3.1/lib/redis.rb:250:in `info'
/usr/local/bundle/gems/sidekiq-4.2.10/lib/sidekiq.rb:113:in `block in redis_info'
/usr/local/bundle/gems/sidekiq-4.2.10/lib/sidekiq.rb:95:in `block in redis'
/usr/local/bundle/gems/connection_pool-2.2.3/lib/connection_pool.rb:63:in `block (2 levels) in with'
/usr/local/bundle/gems/connection_pool-2.2.3/lib/connection_pool.rb:62:in `handle_interrupt'
/usr/local/bundle/gems/connection_pool-2.2.3/lib/connection_pool.rb:62:in `block in with'
/usr/local/bundle/gems/connection_pool-2.2.3/lib/connection_pool.rb:59:in `handle_interrupt'
/usr/local/bundle/gems/connection_pool-2.2.3/lib/connection_pool.rb:59:in `with'
/usr/local/bundle/gems/sidekiq-4.2.10/lib/sidekiq.rb:92:in `redis'
/usr/local/bundle/gems/sidekiq-4.2.10/lib/sidekiq.rb:106:in `redis_info'
/usr/local/bundle/gems/sidekiq-4.2.10/lib/sidekiq/cli.rb:71:in `run'
/usr/local/bundle/gems/sidekiq-4.2.10/bin/sidekiq:12:in `<top (required)>'
/usr/local/bundle/bin/sidekiq:29:in `load'
/usr/local/bundle/bin/sidekiq:29:in `<main>'

我的 webapp.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checklist-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: railsapp
    spec:
      containers:
        - name: webapp
          image: masettyabhishek/checklist:latest
          command: ["rails", "s", "-p", "3001", "-b", "0.0.0.0", "-e", "PRODUCTION"]
          ports:
          - name: checklist-port
            containerPort: 3001
          env:
            - name: MYSQL_HOST
              value: database-service
            - name: MYSQL_USER
              value: root
            - name: MYSQL_PASSWORD
              value: Mission2019
            - name: MYSQL_DATABASE
              value: checklist
            - name: MYSQL_ROOT_PASSWORD
              value: Mission2019
            - name: REDIS_URL
              value: redis
            - name: REDIS_PORT
              value: "6379"
  selector:
    matchLabels:
      app: railsapp

webapp-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: webapp-service
spec:
  ports:
  - port: 3001
    protocol: TCP
  type: NodePort
  selector:
    app: railsapp

sidekiq.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sidekiq-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        instance: sidekiq
    spec:
      containers:
        - name: sidekiq
          image: masettyabhishek/checklist:latest
          command: ["sidekiq", "-C", "config/sidekiq.yml"]
          env:
            - name: MYSQL_HOST
              value: database-service
            - name: MYSQL_USER
              value: root
            - name: MYSQL_PASSWORD
              value: Mission2019
            - name: MYSQL_DATABASE
              value: checklist
            - name: MYSQL_ROOT_PASSWORD
              value: Mission2019
            - name: REDIS_URL
              value: redis
            - name: REDIS_PORT
              value: "6379"
          ports:
            - name: redis-port
              containerPort: 6379
  selector:
    matchLabels:
      instance: sidekiq

redis.yaml

apiVersion: v1
kind: Pod
metadata:
  name: redis-pod
spec:
  containers:
  - name: redis
    image: redis:alpine
    command: ["redis-server"]
    ports:
      - containerPort: 6379

---
apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  selector:
    name: redis-pod
    instance: sidekiq
    app: railsapp
  type: NodePort
  ports:
  - port: 6379

这是我的 rails 应用程序中的 sidekiq.yml

Sidekiq.configure_server do |config|
    config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end

Sidekiq.configure_client do |config|
    config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end

如果这有助于回答这个问题，这就是 Dockerfile。

FROM ubuntu:16.04

ENV RUBY_MAJOR="2.6" \
    RUBY_VERSION="2.6.3" \
    RUBYGEMS_VERSION="3.0.8" \
    BUNDLER_VERSION="1.17.3" \
    RAILS_VERSION="5.2.1" \
    RAILS_ENV="production" \
    GEM_HOME="/usr/local/bundle"
ENV BUNDLE_PATH="$GEM_HOME" \
    BUNDLE_BIN="$GEM_HOME/bin" \
    BUNDLE_SILENCE_ROOT_WARNING=1 \
    BUNDLE_APP_CONFIG="$GEM_HOME"

ENV PATH="$BUNDLE_BIN:$GEM_HOME/bin:$GEM_HOME/gems/bin:$PATH"

USER root
RUN apt-get update && \
      apt-get -y install sudo
RUN echo "%sudo ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
    addgroup --gid 1024 stars && \
    useradd -G stars,sudo -d /home/user --shell /bin/bash -m user
RUN mkdir -p /usr/local/etc \
    && echo 'install: --no-document' >> /usr/local/etc/gemrc \
    && echo 'update: --no-document' >> /usr/local/etc/gemrc

USER user
RUN sudo apt-get -y install --no-install-recommends vim make gcc zlib1g-dev autoconf build-essential libssl-dev libsqlite3-dev \
    curl htop unzip mc openssh-server openssl bison libgdbm-dev ruby git libmysqlclient-dev tzdata mysql-client
    
RUN sudo rm -rf /var/lib/apt/lists/* \
    && sudo curl -fSL -o ruby.tar.gz "http://cache.ruby-lang.org/pub/ruby/$RUBY_MAJOR/ruby-$RUBY_VERSION.tar.gz" \
    && sudo mkdir -p /usr/src/ruby \
    && sudo tar -xzf ruby.tar.gz -C /usr/src/ruby --strip-components=1 \
    && sudo rm ruby.tar.gz

USER root
RUN cd /usr/src/ruby \
    && { sudo echo '#define ENABLE_PATH_CHECK 0'; echo; cat file.c; } > file.c.new && mv file.c.new file.c \
    && autoconf \
    && ./configure --disable-install-doc

USER user
RUN cd /usr/src/ruby \
    && sudo make -j"$(nproc)" \
    && sudo make install \
    && sudo gem update --system $RUBYGEMS_VERSION \
    && sudo rm -r /usr/src/ruby
RUN sudo gem install bundler --version "$BUNDLER_VERSION"

RUN sudo mkdir -p "$GEM_HOME" "$BUNDLE_BIN" \
    && sudo chmod 777 "$GEM_HOME" "$BUNDLE_BIN" \
    && sudo gem install rails --version "$RAILS_VERSION"
RUN mkdir -p ~/.ssh && \
    chmod 0700 ~/.ssh && \
    ssh-keyscan github.com > ~/.ssh/known_hosts
ARG ssh_pub_key
ARG ssh_prv_key
RUN echo "$ssh_pub_key" > ~/.ssh/id_rsa.pub && \
    echo "$ssh_prv_key" > ~/.ssh/id_rsa && \
    chmod 600 ~/.ssh/id_rsa.pub && \
    chmod 600 ~/.ssh/id_rsa
USER root
RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
RUN apt-get install -y nodejs
USER user
WORKDIR /data
RUN sudo mkdir /data/checklist
WORKDIR /data/checklist
ADD Gemfile Gemfile.lock ./
RUN sudo chown -R user /data/checklist
RUN bundle install
ADD . .
RUN sudo chown -R user /data/checklist
EXPOSE 3001
ENV RAILS_SERVE_STATIC_FILES true
ENV RAILS_LOG_TO_STDOUT true
RUN chmod +x ./config/docker/prepare-db.sh && sh ./config/docker/prepare-db.sh
ENTRYPOINT ["bundle", "exec"]
CMD ["sh", "./config/docker/startup.sh"]

kubectl 描述 svc redis

➜  checklist kubectl describe svc redis
Name:                     redis
Namespace:                default
Labels:                   <none>
Annotations:              <none>
Selector:                 app=railsapp,instance=sidekiq,name=redis-pod
Type:                     NodePort
IP:                       10.103.6.43
Port:                     <unset>  6379/TCP
TargetPort:               6379/TCP
NodePort:                 <unset>  31886/TCP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Answer 1

如您所见， redis服务的Endpoints部分没有 pod IP，这是Connection refused错误的原因。 Pod 需要具有与服务选择器匹配的标签。 使用如下标签更新 redis pod 应该可以解决问题。

apiVersion: v1
kind: Pod
metadata:
  name: redis-pod
  labels:
    instance: sidekiq
    app: ailsapp
    name: redis-pod
spec:
  containers:
  - name: redis
    image: redis:alpine
    command: ["redis-server"]
    ports:
      - containerPort: 6379