在Elastic Beanstalk上與主管一起在后台運行huey任務隊列

Question

我正在嘗試在Flask應用程序所需的彈性beantalk上運行huey任務隊列。 但是沒有內置的方式將huey作為守護進程運行。 huey的作者建議與主管一起運行huey（ 此鏈接 ），並且由於Elastic beantalk已經使用了主管，所以我認為我們可以只添加要由主管管理的程序。 但是我不確定如何以編程方式執行此操作。 當前，我正在配置文件中使用container_commands （ ref link ）鍵運行此文件，但是彈性beantalk在運行於前台的某個時間后會給我超時錯誤。 以下是我正在使用的配置文件。

packages:
  yum:
    gcc: []
    gcc-c++: []
    gcc-gfortran: []
    htop: []
    make: []
    wget: []
    atlas-devel: []
    lapack-devel: []
commands:
  01enable_swap:
    command:
      - sudo dd if=/dev/zero of=/var/swap1 bs=1M count=1024
      - sudo mkswap /var/swap1
      - sudo chmod 644 /var/swap1
      - sudo swapon /var/swap1
    cwd: /home/ec2-user
  02install_redis:
    command:
      - wget "http://download.redis.io/redis-stable.tar.gz"
      - tar -xvzf redis-stable.tar.gz
      - rm redis-stable.tar.gz
      - cd redis-stable
      - sudo make
      - sudo make install
    cwd: /home/ec2-user
container_commands:
  01download_nltk_packages:
    command: "python install_resources.py"
  02run_redis:
    command: "redis-server --host 127.0.0.1 --port 6379 --daemonize yes"
  03run_huey:
    command: "huey_consumer jupiter.huey"

這是我要實現的目標：
1.部署Flask應用程序時，huey應該作為后台進程運行。
2.主管應處理色調過程的自動啟動/停止。

Answer 1

我通過在名為002_supervisor.conf的ebextensions文件中執行以下操作來解決此問題。 這是給django的，但我敢肯定它可以適合燒瓶。

1. 創建主管配置文件
2. 創建主管init.d文件
3. 創建一個huey.conf文件以供主管加載

files:
   /usr/local/etc/supervisord.conf:
    mode: "000755"
    owner: root
    group: root
    content: |
        [unix_http_server]
        file=/tmp/supervisor.sock   ; (the path to the socket file)

        [supervisord]
        logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
        pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
        nodaemon=false               ; (start in foreground if true;default false)

        [rpcinterface:supervisor]
        supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

        [supervisorctl]
        serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

        [include]
        files = /usr/local/etc/*.conf

        [inet_http_server]
        port = 127.0.0.1:9001
   /etc/init.d/supervisord:
    mode: "000755"
    owner: root
    group: root
    content: |
        #!/bin/bash

        # Source function library
        . /etc/rc.d/init.d/functions

        # Source system settings
        if [ -f /etc/sysconfig/supervisord ]; then
            . /etc/sysconfig/supervisord
        fi

        # Path to the supervisorctl script, server binary,
        # and short-form for messages.
        supervisorctl=/usr/local/bin/supervisorctl
        supervisord=${SUPERVISORD-/usr/local/bin/supervisord}
        prog=supervisord
        pidfile=${PIDFILE-/tmp/supervisord.pid}
        lockfile=${LOCKFILE-/var/lock/subsys/supervisord}
        STOP_TIMEOUT=${STOP_TIMEOUT-60}
        OPTIONS="${OPTIONS--c /usr/local/etc/supervisord.conf}"
        RETVAL=0

        start() {
            echo -n $"Starting $prog: "
            daemon --pidfile=${pidfile} $supervisord $OPTIONS
            RETVAL=$?
            echo
            if [ $RETVAL -eq 0 ]; then
                touch ${lockfile}
                $supervisorctl $OPTIONS status
            fi
            return $RETVAL
        }

        stop() {
            echo -n $"Stopping $prog: "
            killproc -p ${pidfile} -d ${STOP_TIMEOUT} $supervisord
            RETVAL=$?
            echo
            [ $RETVAL -eq 0 ] && rm -rf ${lockfile} ${pidfile}
        }

        reload() {
            echo -n $"Reloading $prog: "
            LSB=1 killproc -p $pidfile $supervisord -HUP
            RETVAL=$?
            echo
            if [ $RETVAL -eq 7 ]; then
                failure $"$prog reload"
            else
                $supervisorctl $OPTIONS status
            fi
        }

        restart() {
            stop
            start
        }

        case "$1" in
            start)
                start
                ;;
            stop)
                stop
                ;;
            status)
                status -p ${pidfile} $supervisord
                RETVAL=$?
                [ $RETVAL -eq 0 ] && $supervisorctl $OPTIONS status
                ;;
            restart)
                restart
                ;;
            condrestart|try-restart)
                if status -p ${pidfile} $supervisord >&/dev/null; then
                  stop
                  start
                fi
                ;;
            force-reload|reload)
                reload
                ;;
            *)
                echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload}"
                RETVAL=2
            esac

            exit $RETVAL
   "/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_huey.sh" :
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash

      # Get django environment variables
      env=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g' | sed 's/%/%%/g'`
      env=${env%?}

      # Create huey configuration script
      hueyconf="[program:huey]
      ; Set full path to celery program if using virtualenv
      command=/opt/python/current/app/production.py run_huey
      user=nobody
      numprocs=1
      stdout_logfile=/var/log/huey.log
      stderr_logfile=/var/log/huey.log
      autostart=true
      autorestart=true
      startsecs=10

      ; Need to wait for currently executing tasks to finish at shutdown.
      ; Increase this if you have very long running tasks.
      stopwaitsecs = 60

      ; When resorting to send SIGKILL to the program to terminate it
      ; send SIGKILL to its whole process group instead,
      ; taking care of its children as well.
      killasgroup=true
      environment=$env"

      # Create the celery supervisord conf script
      echo "$hueyconf" | tee /usr/local/etc/huey.conf

      # Update supervisord in cache without restarting all services
      /usr/local/bin/supervisorctl reread
      /usr/local/bin/supervisorctl update

      # Start/Restart huey through supervisord
      /usr/local/bin/supervisorctl -c /usr/local/etc/supervisord.conf restart huey

commands:
  01_start_supervisor:
    command: '/etc/init.d/supervisord restart'
    leader_only: true