AWS sdk Amazon S3 Stream Wrapper in php

Question

I am new in the amazon s3.In my cakephp3.0 application I need to copy file from one bucket to another. For that I installed AWS SDK using composer and in my controller constructor function I have initilized s3 client with credentials

public function initialize() {
    parent::initialize();
    $this->s3Client = new S3Client([
        'version' => 'latest',
        'region' => 'mumbai',
        'credentials' => [
            'key' => 'key',
            'secret' => 'secrect',
        ],
    ]);
}

And in my function to copy file

public function amazonCopy() {      
    $this->s3Client->registerStreamWrapper(); 
        $sourceBucket ="collect";
        $sourceKeyname = "testFolder";

       $s3Client->copyObject(array(
           'Bucket'     => "test1",
           'Key'        => testFolder,
           'CopySource' => "{$sourceBucket}/{$sourceKeyname}",
       )); 
    }
}

Now I am getting an error

Error executing "ListBuckets" on " https://s3.mumbai.amazonaws.com/ "; AWS HTTP error: Error creating resource: [message] fopen(): php_network_getaddresses: getaddrinfo failed: Name or service not known [file] /var/www/mm/Src/php/operations/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php [line] 312 [message] fopen( https://s3.mumbai.amazonaws.com/ ): failed to open stream: php_network_getaddresses: getaddrinfo failed: Name or service not known [file] /var/www/mm/Src/php/operations/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php [line] 312 Aws\\S3\\Exception\\S3Exception

I want to Create a zip file using of data in one bucket and copy it to other bucket without writing the file to disk using stream. What will be the cause of this error? Any help will be appreciated.

Now the zipping as stream is working properly when I used the zipstream library for that I used the code

$this->s3Client->registerStreamWrapper();
$zip = new ZipStream\ZipStream('example.zip');

$zip->addFile('hello.txt', 'This is the contents of hello.txt');     

$zip->addFileFromPath('happy_children-wide.jpg', 's3://mmvideo-test/Music/happy_children-wide.jpg');
$zip->addFileFromPath('txte', 's3://test/Music/test1.txt');
$zip->finish();

But again the problem is that I want to write the zip output on the fly to the bucket2. Because my files will contain videos of large size so it is not possible to store it in disk as temporary storage. it should write to the bucket as zip on the fly.

Answer 1

Tried but could not get it to work with streams only. Using streams only to download and zip into a tmp file.

$client = new S3Client(array(
    'region'      => 'ap-northeast-1',
    'version'     => 'latest',
    'credentials' => $provider
));

$client->registerStreamWrapper();

$objects = $client->listObjectsV2([
    'Bucket' => $fromBucket,
    'Prefix' => $fromPrefix
]);
$keys = array();
foreach ($objects['Contents'] as $o) {
    $keys[] = $o['Key'];
}

$tmp = tempnam(sys_get_temp_dir(), 'zipstream');
echo "creating temp zip: $tmp\n";
$ostream = fopen($tmp, 'w');
$zip = new ZipStream(null, array(ZipStream::OPTION_OUTPUT_STREAM => $ostream));

foreach($keys as $k){
    echo "$k\n";
    $istream = fopen("s3://$fromBucket/$k", 'r');
    $zip->addFileFromStream($k, $istream);
    fflush($ostream);
    fclose($istream);
}

$zip->finish();

$client->putObject(array(
    'Bucket'       => $toBucket,
    'Key'          => $toZip,
    'SourceFile'   => $tmp
));