Why is Apache Camel Main ignoring the destination bucket name in a YAML DSL AWS2-S3 to AWS2-S3 route?

Question

I am using the Apache Camel Main (see https://camel.apache.org/components/next/others/main.html ) component version 3.19 and have the following AWS2-S3 to AWS3-S3 route specified in my route.yaml file:

- route:
    from:
      uri: "aws2-s3:arn:aws:s3:source-bucket"
      parameters:
        region: "eu-central-1"
        accessKey: "xxxxx"
        secretKey: "xxxxx"
        deleteAfterRead: "false"
      steps:
        - to:
            uri: "aws2-s3:arn:aws:s3:destination-bucket"
            parameters:
              region: "eu-central-1"
              accessKey: "xxxxx"
              secretKey: "xxxxx"

My app looks as follows:

public class MyApp {
    public static void main(String[] args) throws Exception {
        Main main = new Main(MyApp .class);
        main.run(args);
    }

}

The purpose of the above route and app is to copy over all files from the source-bucket to the destination-bucket .

When I ran the app, both buckets already existed and while the source-bucket contained a few files, the destination-bucket was empty.

But instead of copying over the files into the destination-bucket it seems that all files have been copied back into the source-bucket while overwriting the existing files. Moreover, after running the app the destination-bucket was still empty.

Is this a bug in Camel Main or is something wrong in my route.yaml ?

Thanks in advance for your help.

Answer 1

This is because the consumer from the original bucket source-bucket is providing an header containing the name of it. This header will override the destination-bucket in the producer to the destination bucket.

Your code should look like:

- route:
    from:
      uri: "aws2-s3:arn:aws:s3:source-bucket"
      parameters:
        region: "eu-central-1"
        accessKey: "xxxxx"
        secretKey: "xxxxx"
        deleteAfterRead: "false"
      steps:
        - remove-header:
          name: "CamelAwsS3BucketName"
        - to:
            uri: "aws2-s3:arn:aws:s3:destination-bucket"
            parameters:
              region: "eu-central-1"
              accessKey: "xxxxx"
              secretKey: "xxxxx"

In this way you won't carry the header to the next endpoint.

This is happening here: https://github.com/apache/camel/blob/main/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/utils/AWS2S3Utils.java#L46

You could also look at the copyObject operation on the producer side: https://github.com/apache/camel/blob/main/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Producer.java#L403

By the way, with removing the header everything should work as expected.