incorrect JSON response from S3 Multiple Objects Delete

Question

I am trying to delete a list of S3 items but noticed that the response from the API call does not seems correct:

according to the API the response should contain the Deleted list only in this case:

Deleted

Container element for a successful delete. It identifies the object that was successfully deleted.

Children: Key, VersionId

Type: Container

Ancestor: DeleteResult

delete_json is:

{
   'Objects':[
      {
         'Key':'test1'
      },
      {
         'Key':'test2'
      },
      {
         'Key':'test3'
      },
      {
         'Key':'test4'
      }
   ],
   'Quiet':False
}

Python code to delete:

response = self.client.delete_objects(Bucket=s3_bucket, Delete=delete_json)

and the response is:

{
   'Deleted':[
      {
         'DeleteMarkerVersionId':'null',
         'Key':'test1',
         'DeleteMarker':True
      },
      {
         'DeleteMarkerVersionId':'null',
         'Key':'test2',
         'DeleteMarker':True
      },
      {
         'DeleteMarkerVersionId':'null',
         'Key':'test3',
         'DeleteMarker':True
      },
      {
         'DeleteMarkerVersionId':'null',
         'Key':'test4',
         'DeleteMarker':True
      }
   ],
   'ResponseMetadata':{
      'HostId':'JDl4XNrK02lfBLXCDDIJjuPXG6tOovOp0dtCv7eMlHc0hGC2L7eSV/wb/XRBFyYQ9mHHzb0O/Yg=',
      'HTTPStatusCode':200,
      'RequestId':'2F2A9FAE7F906B8'
   }
}

is it a bug?

Answer 1

As you have confirmed in comments, this bucket had versioning enabled, and subsequently suspended.

This is not the same thing as a bucket that has never had versioning enabled. Once a bucket has had versioning enabled, it's always going to be a versioned bucket, but the versioning behavior for new objects, overwrites, and deletes, is different.

From the API documentation, the page you cited:

Because versioning is enabled on the bucket, Amazon S3 does not delete the object. Instead, it adds a delete marker for this object. The response indicates that a delete marker was added (the DeleteMarker element in the response as a value of true) and the version number of the delete marker it added.

Take that information, then add to it the behavior that necessarily must follow when versioning is suspended. The version number of the delete marker is null, because that's how delete markers work in versioning suspended buckets. The delete marker replaces the null version of the object, if present, and assumes the id of null itself.

http://docs.aws.amazon.com/AmazonS3/latest/dev/DeletingObjectsfromVersioningSuspendedBuckets.html

It also follows that your delete operation may not actually have removed the desired object, if it wasn't the null version of the object. The object will only appear to have been deleted.

The API response seems correct.