Writing Zip Files to GAE Blobstore

Question

I'm using the Java API for reading and writing to the Google App Engine Blobstore.

I need to zip files directly into the Blobstore, meaning I have String objects which I want to be stored in the Blobstore when zipped.

My problem is that standard zipping methods are using OutputStream to write, while it seems that GAE doesn't provide one for writing to the Blobstore.

Is there a way to combine those APIs, or are there different APIs I can use (I haven't found such)?

Answer 1

If I am not wrong, you can try to use the Blobstore low level API . It offers a Java Channel ( FileWriteChannel ), so you could probably convert it to an OutputStream :

Channels.newOutputStream(channel)

And use that output stream with the java.util.zip.* classes you are currently using ( here you have a related example that uses Java NIO to zip something to a Channel / OutputStream )

I have not tried it.

Answer 2

Here is one example to write content file and zip it and store it into blobstore:

AppEngineFile file = fileService.createNewBlobFile("application/zip","fileName.zip");

try {

     FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock);

     //convert as outputstream
    OutputStream blobOutputStream = Channels.newOutputStream(writeChannel);

    ZipOutputStream zip = new ZipOutputStream(blobOutputStream);                    

     zip.putNextEntry(new ZipEntry("fileNameTozip.txt"));

     //read the content from your file or any context you want to get
     final byte data[] = IOUtils.toByteArray(file1InputStream);                    

     //write byte data[] to zip
      zip.write(bytes);

     zip.closeEntry();                    
     zip.close();

     // Now finalize
     writeChannel.closeFinally();
 } catch (IOException e) {
     throw new RuntimeException(" Writing file into blobStore", e);
 }

Answer 3

The other answer is using BlobStore api, but currently the recommended way is to use App Engine GCS client.

Here is what I use to zip multiple files in GCS :

public static void zipFiles(final GcsFilename targetZipFile,
                            Collection<GcsFilename> filesToZip) throws IOException {

    final GcsFileOptions options = new GcsFileOptions.Builder()
            .mimeType(MediaType.ZIP.toString()).build();
    try (GcsOutputChannel outputChannel = gcsService.createOrReplace(targetZipFile, options);
         OutputStream out = Channels.newOutputStream(outputChannel);
         ZipOutputStream zip = new ZipOutputStream(out)) {

        for (GcsFilename file : filesToZip) {
            try (GcsInputChannel readChannel = gcsService.openPrefetchingReadChannel(file, 0, MB);
                 InputStream is = Channels.newInputStream(readChannel)) {
                final GcsFileMetadata meta = gcsService.getMetadata(file);
                if (meta == null) {
                    log.warn("{} NOT FOUND. Skipping.", file.toString());
                    continue;
                }
                final ZipEntry entry = new ZipEntry(file.getObjectName());
                zip.putNextEntry(entry);

                ByteStreams.copy(is, zip);
                zip.closeEntry();
            }
            zip.flush();
        }

    }