[英]How to get the oldest added object from Amazon S3 Bucket?
我在具有某些对象的亚马逊存储桶中有一个文件夹。
如何获取最早添加的对象?
public FileMetaData Poll()
{
var config = new AmazonS3Config();
config.ServiceURL = "s3.amazonaws.com";
config.CommunicationProtocol = Protocol.HTTP;
string bucketName = "bucketname1";
string accessKey = "accesskey1";
string secretKey = "secretkey1";
Amazon.S3.AmazonS3 client = AWSClientFactory.CreateAmazonS3Client(accessKey,
secretKey,
config);
var request = new GetObjectRequest();
request.WithBucketName(bucketName);
// how to get the oldest object?
GetObjectResponse response = client.GetObject(request);
// todo
return null;
}
我尝试了下面的代码,效果很好,但是问题是它加载了所有对象,然后找到了我认为不可行的最旧的代码:
var request = new ListObjectsRequest()
.WithBucketName(bucketName)
.WithPrefix(this._folderPath);
ListObjectsResponse response = client.ListObjects(request);
S3Object s3Object = response.S3Objects
.Where(p => !p.Key.EndsWith("_$folder$"))
.OrderBy(k => k.LastModified).FirstOrDefault();
var getObjectRequest = new GetObjectRequest()
.WithBucketName(bucketName)
.WithKey(s3Object.Key);
GetObjectResponse getObjectResponse = client.GetObject(getObjectRequest);
// provider
string provider = getObjectResponse.Metadata.Get("x-amz-meta-provider");
string site = getObjectResponse.Metadata.Get("x-amz-meta-sitename");
string identifier = s3Object.Key.Remove(0, this._folderPath.Length);
string xmlData = new StreamReader(getObjectResponse.ResponseStream, true).ReadToEnd();
return new FileMetaData()
{
Identifier = identifier,
Provider = provider,
SiteName = site,
XmlData = xmlData
};
您的代码似乎很好。 对于“列出对象请求”,您只会损失几秒钟,但是据我所知这是强制性的。
我在您的代码中确实看到的一个问题是,您无法处理每个请求最多返回的键数量为1000的事实。如果您可能拥有更多的键,则必须检查列表是否被截断,请更改请求标记下一个,并发出更多请求。
var request = new ListObjectsRequest()
.WithBucketName(bucketName)
.WithPrefix(this._folderPath);
ListObjectsResponse response;
S3Object s3Object = null;
do
{
response = client.ListObjects(request);
S3Object tempS3Object = response.S3Objects
.Where(p => !p.Key.EndsWith("_$folder$"))
.OrderBy(k => k.LastModified).FirstOrDefault();
if (s3Object != null)
{
if (s3Object.LastModified < tempS3Object.LastModified)
s3Object = tempS3Object;
}
else s3Object = tempS3Object;
request.Marker = response.NextMarker;
} while (response.IsTruncated);
var getObjectRequest = new GetObjectRequest()
.WithBucketName(bucketName)
.WithKey(s3Object.Key);
GetObjectResponse getObjectResponse = client.GetObject(getObjectRequest);
// provider
string provider = getObjectResponse.Metadata.Get("x-amz-meta-provider");
string site = getObjectResponse.Metadata.Get("x-amz-meta-sitename");
string identifier = s3Object.Key.Remove(0, this._folderPath.Length);
string xmlData = new StreamReader(getObjectResponse.ResponseStream, true).ReadToEnd();
return new FileMetaData()
{
Identifier = identifier,
Provider = provider,
SiteName = site,
XmlData = xmlData
};
它加载所有对象,然后找到最旧的
实际上,您并没有加载所有对象,而是要列出它们(有很大的不同。)S3不是数据库(也不是文件系统),因此如果要轻松访问,则必须构建自己的本地索引。 (或使用DynamoDB,SimpleDB,RDS等)
更改你的钥匙,包括日期/时间不会帮助你多少 。 您可以使用前缀和定界符来帮助缩小搜索范围。 (即,如果每个文件都以“ YYYY-MM-DD-HHMM”开头,则可以将定界符设置为“-”以查找最早的年份,然后将前缀=“ YYYY-”和定界符“-”以查找最早的月份,等等) 。)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.