AWS Firehose 交付到 VPC 中的跨账户 Elasticsearch

Question

I have a Elasticsearch inside the VPC running in account A.

I want to deliver logs from Firehose in Account B to the Elasticsearch in Account A.

Is it possible?

When I try to create delivery stream from AWS CLI I am getting below exception,

$: /usr/local/bin/aws firehose create-delivery-stream --cli-input-json file://input.json --profile devops
An error occurred (InvalidArgumentException) when calling the CreateDeliveryStream operation: Verify that the IAM role has access to the ElasticSearch domain.

The same IAM role, and same input.json works when modified to the Elasticsearch in Account B. I have Transit gateway connectivity enabled between the AWS accounts and I can connect telnet to the Elasticsearch in Account A from EC2 instance in Account B.

Adding my complete terraform code(i got same exception in AWS CLI and also in Terraform): https://gist.github.com/karthikeayan/a67e93b4937a7958716dfecaa6ff7767

Answer 1

It looks like you haven't granted sufficient permissions to the role that is used when creating the stream (from the CLI example provided I'm guessing its a role named 'devops'). At minimum you will need firehose: CreateDeliveryStream .

I suggest adding the below permissions to your role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "firehose:PutRecord",
                "firehose:CreateDeliveryStream",
                "firehose:UpdateDestination"
            ],
            "Resource": "*"
        }
    ]
}

Answer 2

https://forums.aws.amazon.com/message.jspa?messageID=943731

I have been informed from AWS forum that this feature is currently not supported.

Answer 3

You can set up Kinesis Data Firehose and its dependencies, such as Amazon Simple Storage Service (Amazon S3) and Amazon CloudWatch, to stream across different accounts. Streaming data delivery works for publicly accessible OpenSearch Service clusters whether or not fine-grained access control (FGAC) is enabled

https://aws.amazon.com/premiumsupport/knowledge-center/kinesis-firehose-cross-account-streaming/