猪用户定义函数中的aws Amazon S3客户端凭据

Question

I made a user defined function for my pig script to delete a folder. But how do i get the credentials?

REGISTER s3://emr-custom-jars/custom.jar;

if i put register in my pig script it can access files on aws storage. so my credentials must be available for pig. can i access my credentials or AmazoneS3Client without uploading my accessKey and secretKey inside my jar?

    private AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
    private AmazonS3Client s3client = new AmazonS3Client(credentials);  

    @Override
    public void deleteObjectsInFolder(String bucketName, String folderPath) {
        for (S3ObjectSummary file : s3client.listObjects(bucketName, folderPath).getObjectSummaries()){           
            s3client.deleteObject(bucketName, file.getKey());
        }
    }

Answer 1

Yes, there are a couple of ways to provide the access credentials without having to hard-code the values in the jar. AmazonS3Client

 public AmazonS3Client() 

Constructs a new client to invoke service methods on Amazon S3. A credentials provider chain will be used that searches for credentials in this order:

• Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_KEY
• Java System Properties - aws.accessKeyId and aws.secretKey
• Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
• Instance Profile Credentials - delivered through the Amazon EC2 metadata service