使用单个 Firehose 将数据索引到多个索引

Question

I'm trying to understand if it's possible to index data to elasticsearch using kinesis firehose to multiple indices, by passing it the index name and type (similar to elasticsaerch bulk api)

AWS documentation ( here ) says:

.... Also, the rest.action.multi.allow_explicit_index option for your Elasticsearch cluster must be set to true (default) in order to take bulk requests with an explicit index that is set per record . For more information, see Amazon ES Configure Advanced Options in the Amazon Elasticsearch Service Developer Guide.

but i can't find any documentation of how to do it, and moreover firehose configuration requires index name.

this try didnt work:

 const data = [
    {
        "value": "1",
        "_index": "index1",
        "_type": "type1"
    }];

const params = {
    DeliveryStreamName: 'XXX', /* required */
    Records: [/* required */
        {
            Data: JSON.stringify(data[0]) //new Buffer('...') || 'STRING_VALUE' /* Strings will be Base-64 encoded on your behalf */ /* required */
        }
    ]
};
firehose.putRecordBatch(params, (err, data) => {
    if (err) console.log(err, err.stack); // an error occurred
    else console.log(data);           // successful response
});

assistance will be appreciated

Answer 1

It's quite a while since asked, but still... According to Kinesis FAQs ( https://aws.amazon.com/kinesis/data-firehose/faqs/ ):

A single delivery stream can only deliver data to one Amazon Elasticsearch Service domain and one index currently. If you want to have data delivered to multiple Amazon Elasticsearch domains or indexes, you can create multiple delivery streams.