Why Google Cloud Stores files less in size than the uploaded file size using Google app engine Java

Question

When uploading the file to Google Cloud Storage the size of the stored file differs from the original uploaded file. The Stored files are very less in size than the original size. My upload servlet is as shown below,

public class UploadServlet extends HttpServlet{
    private static final long serialVersionUID = 1L;
    private StorageService storage = new StorageService();
    private static int BUFFER_SIZE =1024 * 1024* 10;
    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    @Override
    public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        ServletFileUpload upload = new ServletFileUpload();
        boolean isMultipart = ServletFileUpload.isMultipartContent(req);
        if (!isMultipart) {
            resp.getWriter().println("File cannot be uploaded !");
            return;
        }
        FileItemIterator iter;
        try {
            iter = upload.getItemIterator(req);
            while (iter.hasNext()) {
                FileItemStream item = iter.next();
                String fileName = item.getName();
                String fieldName = item.getFieldName();
                String mime = item.getContentType();
                if (fieldName.equals("file")) {  // the name of input field in html
                    InputStream is1 = item.openStream();
                    try {
                        FileService fileService = FileServiceFactory.getFileService();
                        AppEngineFile file = fileService.createNewBlobFile(mime, fileName);
                        boolean lock = true;
                        FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock);
                        byte[] b1 = new byte[BUFFER_SIZE];
                        int readBytes1;
                        while ((readBytes1 = is1.read(b1)) != -1) {
                            writeChannel.write(ByteBuffer.wrap(b1, 0, readBytes1));
                            storage.init(fileName, mime);
                            storage.storeFile(b1, readBytes1);
                        }
                        writeChannel.closeFinally();
                        String blobKey = fileService.getBlobKey(file).getKeyString();
                        Entity input = new Entity("Input");
                        input.setProperty("Input File", blobKey);
                        datastore.put(input);
                        is1.close();
                        storage.destroy();
                    } catch (Exception e) {
                        e.printStackTrace(resp.getWriter());
                    }
                }
            }
        } catch (FileUploadException e) {
            // log error here
        }
    }
}

My Storage Code is shown below,

public class StorageService {
    public static final String BUCKET_NAME = "MyBucket";
    private FileWriteChannel writeChannel = null;
    FileService fileService = FileServiceFactory.getFileService();
    private BufferedOutputStream bos = null;
    private static final Logger log = Logger.getLogger(StorageService.class.getName());
    private BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
    public void init(String fileName, String mime) throws Exception {
        System.out.println("Storage service:init() method:  file name:"+fileName+" and mime:"+mime);
        log.info("Storage service:init() method:  file name:"+fileName+" and mime:"+mime);
        GSFileOptionsBuilder builder = new GSFileOptionsBuilder()
            .setAcl("public_read")
            .setBucket(BUCKET_NAME)
            .setKey(fileName)
            .setMimeType(mime);
        AppEngineFile writableFile = fileService.createNewGSFile(builder.build());
        boolean lock = true;
        writeChannel = fileService.openWriteChannel(writableFile, lock);
        bos = new BufferedOutputStream(Channels.newOutputStream(writeChannel));
    }
    public void storeFile(byte[] b, int readSize) throws Exception {
        bos.write(b,0,readSize);
        System.out.println(readSize);
        bos.flush();
    }
    public void destroy() throws Exception {
        log.info("Storage service: destroy() method");
        bos.close();
        writeChannel.closeFinally();
    }
    public BlobKey getBlobkey (String filename) {
        BlobKey bk = blobstoreService.createGsBlobKey("/gs/MyBucket/"+filename);
        return bk;
    }
}

When i upload a file of 2 MB file it stores only 155 bytes. I cant understand where am going wrong.

Kindly suggest me an idea,

Your help is appreciated.

Answer 1

In your UploadServlet class, you invoke storage.init(...) multiple times per file, always creating the file anew. This should not happend:

while ((readBytes1 = is1.read(b1)) != -1) {
    writeChannel.write(ByteBuffer.wrap(b1, 0, readBytes1));
    storage.init(fileName, mime);                          // <- HERE
    storage.storeFile(b1, readBytes1);
}

You need to move the init statement out of the loop, in front of it.