Accessing S3 Objects with storage class Glacier

Question

I wrote a piece of (java) software that downloads objects (archives) from an S3 bucket, extracts the data locally and does operations on it. A few days back, I set the lifecycle policy of all the objects in the "folder" within S3 to be moved to glacier automatically 2 days after creation, so that I have the time to DL and extract the data before it's archived. However, when accessing the data programmatically, Amazon Web Services throws an error

Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: The operation is not valid for the object's storage class

I suppose this is due to the fact that the objects' storage classes have been updated to Glacier. So far I have used the following code to access my S3 data:

public static void downloadObjectFromBucket(String bucketName, String pathToObject, String objectName) throws IOException{
    AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider());        
    S3Object object = s3Client.getObject(new GetObjectRequest(bucketName, pathToObject));
    InputStream reader = new BufferedInputStream(object.getObjectContent());
    File file = new File(objectName);      
    OutputStream writer = new BufferedOutputStream(new FileOutputStream(file));
    int read = -1;
        while ( ( read = reader.read() ) != -1 ) {
            writer.write(read);
        }
    writer.flush();
    writer.close();
    reader.close();
}

Do I have to update my code or change some settings in the AWS Console? It is unclear to me, since the objects are still in S3 and accessing every S3 object has been working wonderfully up until a few days ago where I adapted the lifecycle policies....

Answer 1

An Amazon S3 lifecycle policy can be used to archive objects from S3 into Amazon Glacier.

When archived (as indicated by a Storage Class of Glacier ), the object still "appears" to be in S3 (it appears in listings, you can see its size and metadata) but the contents of the object is kept in Glacier . Therefore, the contents cannot be accessed.

To retrieve the contents of an object in S3 with a Storage Class of Glacier , you will need to RestoreObject to retrieve the contents into S3. This takes 3-5 hours. You also nominate a duration for how long the contents should remain in S3 (where is will be stored with a Storage Class of Reduced Redundancy ). Once the object is restored, you can retrieve the contents of the object.