Logstash doesn't replace variables in http output plugin

Question

I am trying to setup a logstash pipeline that outputs logs to differents other logstash instances based on the field [logstash_url] in my log:

http {
    url => "http://%{[logstash_url]}:5044"
    retry_failed => 'false'
    http_method => 'post'
    retry_non_idempotent => 'true'
    format => 'json_batch'
    http_compression => 'true'
}

The problem is that logstash doesn't replace [logstash_url] and I get the following trace:

[ERROR] 2022-11-16 14:20:04.014 [[main]>worker0] http - Could not fetch URL {:url=>"http://%{[logstash_url]}:5044", :method=>:post, :message=>"Malformed escape pair at index 16: http://%{[logstash_url]}:5044", :class=>"Java::JavaNet::URISyntaxException", :will_retry=>false}

The field [logstash_url] does exist and I can even use it in filter or other output types.

The image I am using is docker.elastic.co/logstash/logstash:8.1.0

Answer 1

It turns out that it was the format => 'json_batch' option that was causing the issue. I removed it and the [logstash_url] is now correctly replaced and logs are forwarded to the appropriate logstash instances.