logstash gcp pubsub 输入插件：“NoSuchMethodError: 'void com.google.common.util.concurrent.Futures.addCallback”

Question

I'm trying to run this plugin with the most basic settings:

input {
    google_pubsub {
        # Your GCP project id (name)
        project_id => "cosmic-keep-148903"

        # The topic name below is currently hard-coded in the plugin. You
        # must first create this topic by hand and ensure you are exporting
        # logging to this pubsub topic.
        topic => "slm"

        # The subscription name is customizeable. The plugin will attempt to
        # create the subscription (but use the hard-coded topic name above).
        subscription => "slm-sub"

        # If you are running logstash within GCE, it will use
        # Application Default Credentials and use GCE's metadata
        # service to fetch tokens.  However, if you are running logstash
        # outside of GCE, you will need to specify the service account's
        # JSON key file below.
        json_key_file => "/path/to/pkey.json"

        # Should the plugin attempt to create the subscription on startup?
        # This is not recommended for security reasons but may be useful in
        # some cases.
        #create_subscription => false
    }
}
output { stdout { codec => rubydebug } }

and seeing this error:

RUBY.inputworker(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:410)
RUBY.start_input(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:401)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:318)
java.lang.Thread.run(java/lang/Thread.java:829)
[ERROR] 2022-08-18 02:38:46.111 [Thread-105] googlepubsub - Expected the service InnerService [FAILED] to be RUNNING, but the service has FAILED
Aug 18, 2022 2:38:46 AM com.google.common.util.concurrent.ListenerCallQueue$PerListenerQueue run
SEVERE: Exception while executing callback: com.google.api.core.AbstractApiService$1@f048817 failed({from = STARTING, cause = java.lang.IllegalStateException: Expected the service InnerService [FAILED] to be RUNNING, but the service has FAILED})
java.lang.IllegalStateException: Expected the service InnerService [FAILED] to be RUNNING, but the service has FAILED
    at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:381)
    at com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:305)
    at com.google.api.core.AbstractApiService.awaitRunning(AbstractApiService.java:96)
    at com.google.cloud.pubsub.v1.Subscriber.startConnections(Subscriber.java:390)
    at com.google.cloud.pubsub.v1.Subscriber.startStreamingConnections(Subscriber.java:355)
    at com.google.cloud.pubsub.v1.Subscriber.access$200(Subscriber.java:93)
    at com.google.cloud.pubsub.v1.Subscriber$3.run(Subscriber.java:299)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NoSuchMethodError: 'void com.google.common.util.concurrent.Futures.addCallback(com.google.common.util.concurrent.ListenableFuture, com.google.common.util.concurrent.FutureCallback)'
    at com.google.cloud.pubsub.v1.StreamingSubscriberConnection.initialize(StreamingSubscriberConnection.java:218)
    at com.google.cloud.pubsub.v1.StreamingSubscriberConnection.doStart(StreamingSubscriberConnection.java:110)
    at com.google.api.core.AbstractApiService$InnerService.doStart(AbstractApiService.java:148)
    at com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:251)
    at com.google.api.core.AbstractApiService.startAsync(AbstractApiService.java:120)
    at com.google.cloud.pubsub.v1.Subscriber.startConnections(Subscriber.java:387)
    ... 4 more

Any guidance is appreciated.

Answer 1

You probably encountered a bug in GCP Pub/Sub plugin for Logstash: https://github.com/logstash-plugins/logstash-input-google_pubsub/issues/57

Latest working version of Logstash for this plugin (v1.2.1) is 8.1.3

You can also try Filebeat, it worked for my case with GCP Pub/Sub and it's much easier on resources.