Graylog无法连接到Kubernetes集群中的Elasticsearch

Question

I deployed Graylog on a Kubernetes cluster and everything was working fine, until I decided to add an environment variable and update the graylog deployment.

Now, some things stopped working. I can see that all inputs are running and they are accepting messages:

However, if I try to see the received messages, it returns 500 error with the following message:

The docs say that the Graylog container needs a service called elasticsearch

docker run --link some-mongo:mongo --link some-elasticsearch:elasticsearch -p 9000:9000 -e GRAYLOG_WEB_ENDPOINT_URI="http://127.0.0.1:9000/api" -d graylog2/server

And if I attach to the graylog pod and curl elasticsearch:9200 , I see a successful result:

{
  "name" : "Vixen",
  "cluster_name" : "graylog",
  "cluster_uuid" : "TkZtckzGTnSu3JjERQNf4g",
  "version" : {
    "number" : "2.4.4",
    "build_hash" : "fcbb46dfd45562a9cf00c604b30849a6dec6b017",
    "build_timestamp" : "2017-01-03T11:33:16Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}

But if the graylog logs say that it is trying to connect to the localhost:

Again, everything was working to this day. Why is it trying to connect to the localhost, not the elastic search service?

Answer 1

Looks like it was a version problem. I downgraded the graylog container to the previous stable version: 2.2.3-1 and it started working again.

My guess is that when I updated the images today, it pulled the latest version which corrupted some things

Answer 2

you may want to try add elastichost to graylog.conf

https://github.com/Graylog2/graylog2-server/blob/master/misc/graylog.conf

at line 172

# List of Elasticsearch hosts Graylog should connect to.
# Need to be specified as a comma-separated list of valid URIs for the http ports of your elasticsearch nodes.
# If one or more of your elasticsearch hosts require authentication, include the    credentials in each node URI that
# requires authentication.
#
# Default: http://127.0.0.1:9200
#elasticsearch_hosts = http://node1:9200,http://user:password@node2:19200

you can make your own graylog.conf and add this to your dockerfile then build with it.

Answer 3

Actually, Graylog has shifted to HTTP API in Graylog 2.3. Therefore, the method of connecting to Elasticsearch cluster has changed. You can now just provide the IP addresses of the ES nodes instead of zen_ping_unicast_hosts. This is the commit which changed this setting - https://github.com/Graylog2/graylog2-server/commit/4213a2257429b6a0803ab1b52c39a6a35fbde889 .

This also enables us to connect AWS ES service as well which was not possible earlier. See this thread of discussion to get more insights - https://github.com/Graylog2/graylog2-server/issues/1473