MongoDB Enterprise failing to start with Wired Tiger as service

Question

I made the installation of WiredTiger, but find that I cannot start the DB using the standard:

sudo service mongod start

It seems when I look into the log file (tail /var/log/mongodb/mongod.log):

2016-08-01T14:48:12.029+0200 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] MongoDB starting : pid=1238 port=27017 dbpath=/var/lib/mongodb 64-bit host=xxx
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] db version v3.2.8
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips  1 Mar 2016
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] modules: enterprise 
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] build environment:
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-08-01T14:48:12.032+0200 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", engine: "wiredTiger", journ
al: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log", quiet: true } }
2016-08-01T14:48:12.052+0200 I STORAGE  [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /var/lib/mongodb/mongod.lock errno:13 Permission denied Is a mongod instance already
 running?, terminating
2016-08-01T14:48:12.052+0200 I CONTROL  [initandlisten] dbexit:  rc: 

100

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  engine: wiredTiger
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1

I have changed the permissions as follows on:

chmod -R 777 /var/log/mongodb
chown -R /var/log/mongodb

But nothing makes a difference. There is no service running. I can run from the command line:

mongod

which gives the following:

2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] MongoDB starting : pid=5776 port=27017 dbpath=/data/db 64-bit host=brett
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] db version v3.2.8
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips  1 Mar 2016
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] modules: enterprise 
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] build environment:
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-08-01T15:35:51.291+0200 I CONTROL  [initandlisten] options: {}
2016-08-01T15:35:51.311+0200 I -        [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-08-01T15:35:51.311+0200 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] 
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] 
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-01T15:35:51.722+0200 I CONTROL  [initandlisten] 
2016-08-01T15:35:51.722+0200 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2016-08-01T15:35:51.722+0200 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-08-01T15:35:51.723+0200 I NETWORK  [initandlisten] waiting for connections on port 27017

But I cannot get mongod as a service! Trying permissions:

sudo chown -R mongodb:mongodb /var/lib/mongodb

or

sudo chown -R user:machine /var/lib/mongodb

or

sudo chown -R root:machine /var/lib/mongodb

If I run the following:

mongod --config /etc/mongod.conf

I can then connect to the DB with mongo

as follows:

MongoDB shell version: 3.2.8
connecting to: test
Server has startup warnings: 
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] 
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] 
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-09T19:46:31.583+0200 I CONTROL  [initandlisten] 

Here is the output from ls -la /var/lib/mongodb

total 216
drwxrwxrwx  4 xxx root   4096 Aug 19 12:37 .
drwxr-xr-x 78 root  root   4096 Aug  9 15:05 ..
-rwxrwxrwx  1 xxx xxx 36864 Aug 10 11:22 collection-0--5694223804331577236.wt
drwxrwxrwx  2 xxx xxx  4096 Aug 10 11:22 diagnostic.data
-rwxrwxrwx  1 xxx xxx 36864 Aug 10 11:22 index-1--5694223804331577236.wt
drwxrwxrwx  2 xxx xxx  4096 Aug  9 19:46 journal
-rwxrwxrwx  1 xxx xxx 16384 Aug 10 11:22 _mdb_catalog.wt
-rwxrwxrwx  1 xxx xxx 36864 Aug 10 11:22 sizeStorer.wt
-rwxrwxrwx  1 xxx xxx    95 Aug  9 19:30 storage.bson
-rwxrwxrwx  1 xxx xxx    46 Aug  9 19:30 WiredTiger
-rwxrwxrwx  1 xxx xxx  4096 Aug 10 11:22 WiredTigerLAS.wt
-rwxrwxrwx  1 xxx xxx    21 Aug  9 19:30 WiredTiger.lock
-rwxrwxrwx  1 xxx xxx   918 Aug 10 11:22 WiredTiger.turtle
-rwxrwxrwx  1 xxx xxx 45056 Aug 10 11:22 WiredTiger.wt

I also tried removing the huge files as was suggested. But I can never find a way to get the service started.

Here is me running mongod:

2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] MongoDB starting : pid=9030 port=27017 dbpath=/data/db 64-bit host=xxx
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] db version v3.2.8
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips  1 Mar 2016
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] modules: enterprise 
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] build environment:
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-08-19T12:53:53.713+0200 I CONTROL  [initandlisten] options: {}
2016-08-19T12:53:53.732+0200 I -        [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-08-19T12:53:53.733+0200 I STORAGE  [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
2016-08-19T12:53:53.733+0200 I CONTROL  [initandlisten] dbexit:  rc: 100

here is:

ls -ld /data/db/
drwxr-xr-x 4 mongodb nogroup 4096 Aug 19 13:10 /data/db/

I can start mongo using

sudo mongod
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] MongoDB starting : pid=12360 port=27017 dbpath=/data/db 64-bit host=brett
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] db version v3.2.8
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] git version: ed70e33130c977bda0024c125b56d159573dbaf0
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g-fips  1 Mar 2016
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] modules: enterprise 
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] build environment:
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten]     distmod: ubuntu1604
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-08-19T13:13:47.117+0200 I CONTROL  [initandlisten] options: {}
2016-08-19T13:13:47.137+0200 I -        [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-08-19T13:13:47.137+0200 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-08-19T13:13:47.548+0200 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-08-19T13:13:47.548+0200 I CONTROL  [initandlisten] 
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] 
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] 
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-19T13:13:47.549+0200 I CONTROL  [initandlisten] 
2016-08-19T13:13:47.549+0200 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2016-08-19T13:13:47.549+0200 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-08-19T13:13:47.550+0200 I NETWORK  [initandlisten] waiting for connections on port 27017

But still no joy getting mongo as a service :(

Here is the /etc/init/mongod.conf

# Ubuntu upstart file at /etc/init/mongod.conf

# Recommended ulimit values for mongod or mongos
# See http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
#
limit fsize unlimited unlimited
limit cpu unlimited unlimited
limit as unlimited unlimited
limit nofile 64000 64000
limit rss unlimited unlimited
limit nproc 64000 64000

kill timeout 300 # wait 300s between SIGTERM and SIGKILL.

pre-start script
  DAEMONUSER=${DAEMONUSER:-mongodb}
  if [ ! -d /var/lib/mongodb ]; then
    mkdir -p /var/lib/mongodb && chown mongodb:mongodb /var/lib/mongodb
  fi
  if [ ! -d /var/log/mongodb ]; then
    mkdir -p /var/log/mongodb && chown mongodb:mongodb /var/log/mongodb
  fi
  touch /var/run/mongodb.pid
  chown $DAEMONUSER /var/run/mongodb.pid
end script

start on runlevel [2345]
stop on runlevel [06]

script
  ENABLE_MONGOD="yes"
  CONF=/etc/mongod.conf 
  DAEMON=/usr/bin/mongod
  DAEMONUSER=${DAEMONUSER:-mongodb}
  DAEMONGROUP=${DAEMONGROUP:-mongodb}

  if [ -f /etc/default/mongod ]; then . /etc/default/mongod; fi

  # Handle NUMA access to CPUs (SERVER-3574)
  # This verifies the existence of numactl as well as testing that the command works
  NUMACTL_ARGS="--interleave=all"
  if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null
  then
    NUMACTL="$(which numactl) -- $NUMACTL_ARGS"
    DAEMON_OPTS=${DAEMON_OPTS:-"--config $CONF"}
  else
    NUMACTL=""
    DAEMON_OPTS="-- "${DAEMON_OPTS:-"--config $CONF"}
  fi

  if [ "x$ENABLE_MONGOD" = "xyes" ]
  then
    exec start-stop-daemon --start \
        --chuid $DAEMONUSER:$DAEMONGROUP \
        --pidfile /var/run/mongodb.pid \
        --make-pidfile \
        --exec $NUMACTL $DAEMON $DAEMON_OPTS
  fi
end script

Answer 1

As others wrote... Error message points that problem is that process cannot create /var/lib/mongodb/mongod.lock

You need to understand that even you say (as root) 'service mongod start' (when you use sudo -command , command is executed as root ), process ( mongod ) is started as user 'mongod' . So, that directory /var/lib/mongodb must be exist and be writable by userid mongod .

mkdir -p /var/lib/mongodb ; chown -R mongod: /var/lib/mongodb 

sudo service mongod restart
(OR 'sudo -u mongod mongod -f /etc/mongod.conf')