Elasticsearch is running with old java version

Question

I recently upgraded the elasticsearch version in a elasticsearch slave node. Still it is picking the wrong java version even though I set the $JAVA_HOME variable.

[igwuser@hbase3 ~]$ sudo systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2020-05-01 12:06:09 +08; 11min ago
     Docs: http://www.elastic.co
 Main PID: 15809 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─15809 /opt/jdk1.8.0_151/bin/java -Xms15g -Xmx15g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=...
           └─15893 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

May 01 12:06:09 hbase3.api.celcom.com.my systemd[1]: Started Elasticsearch.
May 01 12:06:09 hbase3.api.celcom.com.my elasticsearch[15809]: Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /var/log/elasticsearch/gc.log due to Permission denied
[igwuser@hbase3 ~]$ 
[igwuser@hbase3 ~]$ 
[igwuser@hbase3 ~]$ echo $JAVA_HOME
/usr/java/jdk1.8.0_251-amd64/
[igwuser@hbase3 ~]$ 

Answer 1

As it's mentioned in the logs, your elasticsearch service is using the JDK present at the path /opt/jdk1.8.0_151/bin/java while your JAVA_HOME is pointing to /usr/java/jdk1.8.0_251-amd64/ .

Please make sure, that your elasticsearch service uses the correct path. Please see this official information on how to set it.

Edit As Elasticsearch installation is done using the installer, its owned by elasticsearch user, which uses the bundled JDK mentioned in my previous link, In order to make it work, you need to add your new JDK JAVA_HOME to this user but it's not trivial ,

Hence would suggest installing Elasticsearch from tarball from this official link and when you start elasticsearch now from user igwuser for which JAVA_HOME points to your lastest JDK, your Elasticsearch will also use that and you can see that in startup logs as well or in your elasticsearch process.

For example, in my case, my JAVA_HOME points to

/usr/lib/jvm/java-11-openjdk-amd64

And this is what printed in Elastic startup logs

JVM home [/usr/lib/jvm/java-11-openjdk-amd64]

And same can also be verified in Elastic process by using ps aux | grep elastic ps aux | grep elastic

/usr/lib/jvm/java-11-openjdk-amd64/bin/java -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true