从Amazon S3 Bucket iOS下载多个项目

Question

I've recently implemented the new AWS 2.0 iOS SDK in my application (yay, cocoapods!), and using the sample code from Amazon managed to properly configure access and downloads. I can successfully download a single item without issue, but I need to be able to download multiple files dynamically generated based on the current tableview. There doesn't appear to be a way to set up a batch download, so I'm simply trying to loop through an array of objects and trigger a download with each one. It works, but if the list includes more than a few items, it starts randomly misfiring. For example, if my dynamically created list has 14 items in it, 12 will be downloaded, and the other 2 aren't even attempted. The request just vanishes. In my testing, I added a sleep(1) timer, and then all 14 are triggered and downloaded, so I'm guessing that I'm overwhelming the download requests and they are getting dropped unless I slow it down. Slowing it down is not ideal... perhaps there is another way? Here is the code:

 - (IBAction)downloadAllPics:(UIBarButtonItem *)sender {
if (debug==1) {
    NSLog(@"Running %@ '%@'", self.class, NSStringFromSelector(_cmd));
}

CoreDataHelper *cdh =
[(AppDelegate *)[[UIApplication sharedApplication] delegate] cdh];

// for loop iterates through all of the items in the tableview
for (Item *item in self.frc.fetchedObjects) {

    NSString *docDir = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
    NSString *downloadingFilePath1 = [NSString stringWithFormat:@"%@/%@@2x.jpg",docDir, item.imageName];
    NSURL *downloadingFileURL1 = [NSURL fileURLWithPath:downloadingFilePath1];

    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSError *error = nil;
    if ([fileManager fileExistsAtPath:downloadingFilePath1]) {
        fileAlreadyExists = TRUE;
        if (![fileManager removeItemAtPath:downloadingFilePath1
                                     error:&error]) {
            NSLog(@"Error: %@", error);
        }
    }
    __weak typeof(self) weakSelf = self;

    self.downloadRequest1 = [AWSS3TransferManagerDownloadRequest new];
    self.downloadRequest1.bucket = S3BucketName;
    //  self.downloadRequest1.key = S3KeyDownloadName1;
    self.downloadRequest1.key = [NSString stringWithFormat:@"images/%@@2x.jpg", item.imageName];
    self.downloadRequest1.downloadingFileURL = downloadingFileURL1;
    self.downloadRequest1.downloadProgress = ^(int64_t bytesWritten, int64_t totalBytesWritten, int64_t totalBytesExpectedToWrite){
        // update progress
        dispatch_sync(dispatch_get_main_queue(), ^{
            weakSelf.file1AlreadyDownloaded = totalBytesWritten;
            weakSelf.file1Size = totalBytesExpectedToWrite;
        });

    };

     // this launches the actual S3 transfer manager - it is successfully launched with each pass of loop
    [self downloadFiles];
}

[cdh backgroundSaveContext];

 }

That launches the downloadFiles method:

 - (void) downloadFiles {
//if I add this sleep, all 14 download. If I don't usually 11-13 download.
sleep(1);
AWSS3TransferManager *transferManager = [AWSS3TransferManager defaultS3TransferManager];

__block int downloadCount = 0;

[[transferManager download:self.downloadRequest1] continueWithExecutor:[BFExecutor mainThreadExecutor] withBlock:^id(BFTask *task) {
     if (task.error != nil){
        if(task.error.code != AWSS3TransferManagerErrorCancelled && task.error.code != AWSS3TransferManagerErrorPaused){
            NSLog(@"%s Errorx: [%@]",__PRETTY_FUNCTION__, task.error);
        }
    } else {
        self.downloadRequest1 = nil;

    }
    return nil;
}];        
 }

There has got to be a way to download a dynamic list of files from an Amazon S3 bucket, right? Maybe there is a transfer manager that allows an array of files instead of doing them individually?

Any and all help is appreciated. Zack

Answer 1

Sounds like request timeout interval setting issue.

First, when you configure AWSServiceConfiguration *configuration = ... try to configure the timeoutIntervalForRequest property. Also, maxRetryCount as well. maxRetryCount will attempt to download if failure for downloading each operation.

AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:DefaultServiceRegionType
                                                                     credentialsProvider:credentialsProvider];
[configuration setMaxRetryCount:2]; // 10 is the max
[configuration setTimeoutIntervalForRequest:120]; // 120 seconds

Second, for the multiple items downloading try to collect each AWSTask into one array and get the result at the end of group operation. ex)

// task collector
NSMutableSet *uniqueTasks = [NSMutableSet new];

// Loop
for (0 -> numOfDownloads) {
     AWSS3TransferManagerDownloadRequest *downloadRequest = [AWSS3TransferManagerDownloadRequest new];
     [downloadRequest setBucket:S3BucketNameForProductImage];
     [downloadRequest setKey:filename];
     [downloadRequest setDownloadingFileURL:sourceURL];
     [showroomGroupDownloadRequests addObject:downloadRequest];

    AWSTask *task = [[AWSS3TransferManager defaultS3TransferManager] download:downloadRequest];
    [task continueWithBlock:^id(AWSTask *task) {
    // handle each individual operation
    if (task.error == nil) {

    }
    else if (task.error) {

    }
    // add to the tasks
    [uniqueTasks addObject:task];

    return nil;
}

[[AWSTask taskForCompletionOfAllTasks:tasks] continueWithBlock:^id(AWSTask *task) {
    if (task.error == nil) {
        // all downloads succeess
    }
    else if (task.error != nil) {
        // failure happen one of download
    }

   return nil;
}];

Answer 2

The reason some requests seem to vanish is that you define AWSS3TransferManagerDownloadRequest as a property. self.downloadRequest1 = nil; is executed on the background thread, and it is possible that when [transferManager download:self.downloadRequest1] is executed, self.downloadRequest1 is nil .

You should remove the property and simply pass an instance of AWSS3TransferManagerDownloadRequest as an argument for - downloadFiles: .