如何在EMR上重命名Hive的输出文件？

Question

The output of Hive on EMR is a file named 000000_0 (perhaps a different number if there is more than 1 reducer).

How do I get this file to be named differently? I see two options:

1) Get Hive to write it differently

2) Rename the file(s) in S3 after it is written. This is could be a problem: from what I've read S3 doesn't really have a "rename". You have to copy it, and delete the original. When dealing with a file that is 1TB in size, for example, this could cause performance problems or increase usage cost?

Answer 1

The AWS Command Line Interface (CLI) has a convenient mv command that you could add to a script:

aws s3 mv s3://my-bucket/000000_0 s3://my-bucket/data1

Or, you could do it programmatically via the Amazon S3 COPY API call .