How do I get logstash to run as a service on Ubuntu 12.04, where the process is owned by a user other than root?

Question

I'm trying to get logstash to run as a service owned by a user other than root. The init.d script follows:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          tlogserver
# Required-Start:    $local_fs $network $named $time $syslog
# Required-Stop:     $local_fs $network $named $time $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Log Server
# Description:       Talend Logstash Service
### END INIT INFO
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/usr/sbin:/usr/bin:/sbin:/bin

if [ "X${TALEND_LOGSERV}" = "X" ]; then
  TALEND_LOGSERV="/opt/talend/logserv"
fi

if [ "X${TALEND_RUN}" = "X" ]; then
  TALEND_RUN="/opt/talend/run"
fi

DESC="Logstash service for the TAC"
APP_NAME="tlogserver"
DAEMON_START="${TALEND_LOGSERV}/start_logserver.sh"
DAEMON_START_ARGS=""
DAEMON_STOP="${TALEND_LOGSERV}/stop_logserver.sh"
DAEMON_STOP_ARGS=""
PIDFILE="${TALEND_RUN}/${APP_NAME}.pid"
RUN_AS="tomcat7"
RUN_GRP="tomcat7"
SCRIPTNAME="/etc/init.d/${APP_NAME}"

# Exit if the package is not installed
[ -x "${DAEMON_START}" ] || "The daemon is not installed"

# Read configuration variable file if it is present
[ -r /etc/default/${APP_NAME} ] && . /etc/default/${APP_NAME}

# Load the VERBOSE setting and other rcS variables
[ -f /etc/default/rcS ] && . /etc/default/rcS

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start() {
  start-stop-daemon --start --quiet --background --chuid ${RUN_AS}:${RUN_GRP} --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON_START} --test > /dev/null \
    || return 1
  start-stop-daemon --start --quiet --background --chuid ${RUN_AS}:${RUN_GRP} --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON_START} -- \
    ${DAEMON_ARGS} \
    || return 2
}

#
# Function that stops the daemon/service
#
do_stop() {
  start-stop-daemon --start --quiet --background --chuid ${RUN_AS}:${RUN_GRP} --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON_STOP} --test > /dev/null \
    || return 1
  start-stop-daemon --start --quiet --background --chuid ${RUN_AS}:${RUN_GRP} --make-pidfile --pidfile ${PIDFILE} --exec ${DAEMON_STOP} -- \
    ${DAEMON_ARGS} \
    || return 2
}

case "$1" in
  start)
    [ "${VERBOSE}" != no ] && log_daemon_msg "Starting ${DESC}" "${APP_NAME}"
    do_start
    case "$?" in
      0|1) [ "${VERBOSE}" != no ] && log_end_msg 0 ;;
      2) [ "${VERBOSE}" != no ] && log_end_msg 1 ;;
    esac
    ;;
  stop)
    [ "${VERBOSE}" != no ] && log_daemon_msg "Stopping ${DESC}" "${APP_NAME}"
    do_stop
    case "$?" in
      0|1) [ "${VERBOSE}" != no ] && log_end_msg 0 ;;
      2) [ "${VERBOSE}" != no ] && log_end_msg 1 ;;
    esac
    ;;
  restart|force-reload)
    #
    # If the "reload" option is implemented then remove the
    # 'force-reload' alias
    #
    log_daemon_msg "Restarting ${DESC}" "${APP_NAME}"
    do_stop
    case "$?" in
      0|1)
        do_start
        case "$?" in
          0) log_end_msg 0 ;;
          1) log_end_msg 1 ;; # Old process is still running
          *) log_end_msg 1 ;; # Failed to start
        esac
        ;;
      *)
        # Failed to stop
        log_end_msg 1
        ;;
    esac
    ;;
  *)
    echo "Usage: ${SCRIPTNAME} {start|stop|restart}" >&2
    exit 3
    ;;
esac

I get successful return codes when I run this as a service, but the logstash process is not running when I look for it. The start_logserver script follows:

if [ "X${TALEND_LOGSERV}" = "X" ]; then
  TALEND_LOGSERV="/opt/talend/logserv"
fi

cd "$TALEND_LOGSERV"
# ./logstash-1.4.2/bin/logstash agent -f logstash-talend.conf -l /var/log/talend/logserv.log > /dev/null 2>&1 < /dev/null &
echo "inside the start script...." >&2
echo `pwd` >&2
./logstash-1.4.2/bin/logstash agent -f logstash-talend.conf -l /var/log/talend/logserv.log
echo $!

Answer 1

A service script for logstash can be generated for sysv / upstart / systemd with:

LS_HOME=/usr/share/logstash

# use ONE of the following:
${LS_HOME}/bin/system-install ${LS_HOME}/config/startup.options sysv
${LS_HOME}/bin/system-install ${LS_HOME}/config/startup.options upstart
${LS_HOME}/bin/system-install ${LS_HOME}/config/startup.options systemd

These scripts run as logstash:logstash by default.

Answer 2

@Magnus provided a link in a comment that worked beautifully ( elastic/logstash/pkg ). His original comment was:

Any reason you haven't used the init script that ships with Logstash as a starting point? Also, have you tried running these shell scripts with -x to list all commands run? As a final remark, won't the pid captured by start-stop-daemon be that of the shell script rather than the Logstash process?