ELK stack - logstash doesn't get installed properly

Question

I am trying to install the ELK stack. The Elastic Search and Kibana was successfully installed. It works great. But when installing Logstash, it doesn't get installed properly. When i start the logstash service, it shows that there is no logstash. I tried searching Google but can't find a proper solution. I tried reinstalling the package. Eventhough, it doesn't get solved.

Here's the Error:

paulsteven@smackcoders:~$ sudo apt-get install logstash
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  logstash
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/178 MB of archives.
After this operation, 307 MB of additional disk space will be used.
Selecting previously unselected package logstash.
(Reading database ... 293176 files and directories currently installed.)
Preparing to unpack .../logstash_1%3a6.7.0-1_all.deb ...
Unpacking logstash (1:6.7.0-1) ...
Setting up logstash (1:6.7.0-1) ...
Using provided startup.options file: /etc/logstash/startup.options
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Not enough space' (errno=12)
/usr/share/logstash/bin/system-install: line 88: #: command not found
paulsteven@smackcoders:~$ sudo systemctl logstash status
Unknown operation logstash.

paulsteven@smackcoders:~$ sudo systemctl logstash status
Unknown operation logstash.

paulsteven@smackcoders:~$ sudo apt-get install logstash
Reading package lists... Done
Building dependency tree       
Reading state information... Done
logstash is already the newest version (1:6.7.0-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
paulsteven@smackcoders:~$ 

Answer 1

@Smack Alpha

I am using ELK stack for 1 month and I installed these things in my Ubuntu platform using debian package.

Follow the steps to install logstash and start it as a service.

First go to home directory

cd ~

Download debian package of logstash using wget (Be aware of your logstash version)

sudo wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.4.deb

I am used dpkg for installing logstash

sudo dpkg -i logstash-6.5.4.deb

Configuring Logstash

To change permission of logstash.yml for editing

```sudo chmod 777 /etc/logstash/logstash.yml```

Open logstash.yml in a text editor

sudo gedit /etc/logstash/logstash.yml

change the following lines(REMOVE # TAG)

a) http.host: "0.0.0.0" (YOUR IP ADDRESS)

b) http.port: 9600-9700

Save and exit.

Comment for any installation doubts and errors.

Answer 2

I had the exact same error, as I was trying to install Logstash on my Raspberry Pi. Probably it's not relevant for you anymore after one year, but it might be valuable for others.

It appears as if your system doesn't have enough (free) memory for the JRE to run, as this line suggests:

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Not enough space' (errno=12)

The next line shows that a command in the system-install script failed:

/usr/share/logstash/bin/system-install: line 88: #: command not found

Line 88 looks something like this:

$(ruby_exec "${LOGSTASH_HOME}/lib/systeminstall/pleasewrap.rb" "${allopts[@]}" ${program} ${LS_OPTS})

To get a little more information on why that failed, we could insert a small debug-echo before the command, like that, just to see the outcome:

echo "debug: $(ruby_exec "${LOGSTASH_HOME}/lib/systeminstall/pleasewrap.rb" "${allopts[@]}" ${program} ${LS_OPTS})"

Now, if we run the system-install script again, we get a little more insight when the error occurs:

debug: #
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1006632960 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/share/logstash/lib/systeminstall/hs_err_pid28575.log

Further info is in the mentioned error log:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1006632960 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2757), pid=28588, tid=0xb512c460

That was the error I had, it could be different for you, but you should be able to investigate it like that.

So you might want to check your available memory, if you have enough or if any processes consume too much.

Due to my limited memory, I created a temporary swapfile as a workaround, just to see if it runs at all, and it worked for me. But that's probably not a viable solution.

Answer 3

Is the process registered under /etc/systemd/system/ ? Do you see the logstash service file within /etc/init.d/ ?

On certain systems, you might have to manually run the system-install script found in /usr/share/logstash/bin .