stackoom
  简体   繁体   中英

Upload File to Google Cloud Storage via appengine

 原文  2016-07-26 13:17:36  8   5    java/ google-app-engine/ google-cloud-storage/ google-cloud-endpoints

Question

I am trying to upload a file to Google Cloud Storage. My Servlet code is 

public class UploadFile extends HttpServlet {

    private final String BUCKET = "XXXXXXXXX";
       private boolean isMultipart;
       private String filePath;
       private int maxFileSize = 5 * 1024 * 1024;
       private int maxMemSize = 5 * 1024 * 1024;
       private File file ;

       public void init( ){
          // Get the file location where it would be stored.
          filePath = 
                 getServletContext().getInitParameter("file-upload"); 
       }
       public void doPost(HttpServletRequest request, 
                   HttpServletResponse response)
                  throws ServletException, java.io.IOException {
          // Check that we have a file upload request
          isMultipart = ServletFileUpload.isMultipartContent(request);
          response.setContentType("text/html");
          java.io.PrintWriter out = response.getWriter( );
          if( !isMultipart ){
             out.println("<html>");
             out.println("<head>");
             out.println("<title>Servlet upload</title>");  
             out.println("</head>");
             out.println("<body>");
             out.println("<p>No file uploaded</p>"); 
             out.println("</body>");
             out.println("</html>");
             return;
          }
          DiskFileItemFactory factory = new DiskFileItemFactory();
          // maximum size that will be stored in memory
          factory.setSizeThreshold(maxMemSize);
          // Location to save data that is larger than maxMemSize.
          factory.setRepository(new File("/temp/image/"));

          // Create a new file upload handler
          ServletFileUpload upload = new ServletFileUpload(factory);
          // maximum file size to be uploaded.
          upload.setSizeMax( maxFileSize );

          try{ 
          // Parse the request to get file items.
          List fileItems = upload.parseRequest(request);

          // Process the uploaded file items
          Iterator i = fileItems.iterator();

          out.println("<html>");
          out.println("<head>");
          out.println("<title>Servlet upload</title>");  
          out.println("</head>");
          out.println("<body>");
          while ( i.hasNext () ) 
          {
             FileItem fi = (FileItem)i.next();
             if ( !fi.isFormField () )  
             {
                // Get the uploaded file parameters
                String fieldName = fi.getFieldName();
                String fileName = fi.getName();
                String contentType = fi.getContentType();
                boolean isInMemory = fi.isInMemory();
                long sizeInBytes = fi.getSize();
                // Write the file
                if( fileName.lastIndexOf("\\") >= 0 ){
                   file = new File( filePath + 
                   fileName.substring( fileName.lastIndexOf("\\"))) ;
                }else{
                   file = new File( filePath + 
                   fileName.substring(fileName.lastIndexOf("\\")+1)) ;
                }

               String path = Events.uploadFile ( fileName, "image/*", file, BUCKET );

                // fi.write( file ) ;
                out.println("Uploaded Filename: " + fileName + "<br>"+ " File Path:"+ path);
             }
          }
          out.println("</body>");
          out.println("</html>");
       }catch(Exception ex) {
           System.out.println(ex);
       }
       }
       public void doGet(HttpServletRequest request, 
                           HttpServletResponse response)
            throws ServletException, java.io.IOException {

            throw new ServletException("GET method used with " +
                    getClass( ).getName( )+": POST method required.");
       } 
    }

web.xml 

<servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>XXXXXXXXXX.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

fileUpload function to save file to GCS 

public static String uploadFile ( String name, String contentType, File file, String bucketName )
            throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, new FileInputStream ( file ) );

        // Setting the length improves upload performance
        contentStream.setLength ( file.length () );

        StorageObject objectMetadata = new StorageObject ()
                // Set the destination object name
                .setName ( name )
                // Set the access control list to publicly read-only
                .setAcl ( Arrays.asList ( new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

        // Do the insert
        Storage client = StorageFactory.getService ();
        Storage.Objects.Insert insertRequest = client.objects ().insert ( bucketName, objectMetadata, contentStream );

        insertRequest.execute ();
        return "https://storage.cloud.google.com/" + BUCKET + "/" + file.getName ();
    }

But when i try to test this with some API testing client it gives error 

org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found

Further after integrating it with UI which is in Angular and testing it locally, i am facing this issue 

Cross-Origin Request Blocked Reason: CORS header 'Access-Control-Allow-Origin' missing

I tried to resolve this issue but found no solution corresponding to google appengine.

Initially i am trying to upload a image through this code but in near future same code is to be used to upload .pdf and .html files to GCS.

For reference: I am using Google Endpoints for my other data communication needs with client end. Client End is a webapp build in Angular but it will be extended to android and ios.

Any help will be appreciated.

Thank You

UPDATE 1/8/2016

Now i am getting file on server but i don't know where i have to temporarily save the file before sending it to Google Cloud Storage. On storing file in 

war\WEB-INI\

and exception i am facing is 

 java.security.AccessControlException: access denied ("java.io.FilePermission" "\war\WEB-INI\profile.png" "read")

5 answers

solution1
 4 ACCPTED  2016-08-12 13:15:02

Finally I am able to uploading a file from client end to Google Cloud Storage via appengine.

I assume that before executing these steps you have following things ready

  • JSON file from your service account.
  • Created a default bucket.

Step 1: Make a Servlet like this 

package XXXXXXXXX;

import java.io.IOException;
import java.io.InputStream;
import java.security.GeneralSecurityException;
import java.util.Arrays;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.api.client.http.InputStreamContent;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.model.ObjectAccessControl;
import com.google.api.services.storage.model.StorageObject;

import XXXXXXXXXXXXX.StorageFactory;

//@author Umesh Chauhan

/**
 * Save File to GCS
 *
 * @param fileName        File Name with format
 * @header Content-Type   "*/*"
 * @return file path
 * @throws Exception      Any Error during upload
 */
public class UploadFile extends HttpServlet
{

    private static final long serialVersionUID = 1L;
    private final String BUCKET = "YOUR BUCKET NAME";
    private int maxFileSize = 6 * 1024 * 1024;

    @Override
    protected void doOptions ( HttpServletRequest req, HttpServletResponse resp ) throws ServletException, IOException
    {
        // pre-flight request processing
        resp.setHeader ( "Access-Control-Allow-Origin", "*" );
        resp.setHeader ( "Access-Control-Allow-Methods", "*" );
        resp.setHeader ( "Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept" );
    }

    @Override
    public void doPost ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {

        try
        {
            String path = uploadFile ( request.getParameter ( "fileName" ), request.getContentType (),
                    request.getInputStream (), BUCKET, request.getInputStream ().available () );
            // Sending Response
            response.setStatus ( HttpServletResponse.SC_OK );
            response.getWriter ().write ( path );
            response.getWriter ().flush ();
            response.getWriter ().close ();

        }
        catch ( GeneralSecurityException e )
        {
            e.printStackTrace ();
        }
    }

    public String uploadFile ( String name, String contentType, InputStream input, String bucketName,
            int contentLength ) throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, input );

        if ( contentLength < maxFileSize )
        {

            // It is done Automatically.
            /*
             * // Setting the length improves upload performance
             * contentStream.setLength ( contentLength );
             */

            StorageObject objectMetadata = new StorageObject ()
                    // Set the destination object name
                    .setName ( name )
                    // Set the access control list to publicly read-only
                    .setAcl ( Arrays.asList (
                            new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

            // Do the insert
            Storage client = StorageFactory.getService ();

            Storage.Objects.Insert insertRequest = client.objects ()
                    .insert ( bucketName, objectMetadata, contentStream );

            insertRequest.execute ();

            return "https://storage.cloud.google.com/" + BUCKET + "/" + name;
        }
        else
        {
            throw new GeneralSecurityException ( "File size canot be more then 6 MB !" );
        }
    }

    public void doGet ( HttpServletRequest request, HttpServletResponse response )
            throws ServletException, java.io.IOException
    {
        throw new ServletException ( "GET method used with " + getClass ().getName () + ": POST method required." );
    }

}

Step 2: Your Storage Factory 

package XXXXXXXXXXXX;

import java.io.IOException;
import java.net.URL;
import java.security.GeneralSecurityException;
import java.util.Collection;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.StorageScopes;

//@author Umesh Chauhan 

public class StorageFactory
{

    private static Storage instance = null;

    public static synchronized Storage getService () throws IOException, GeneralSecurityException
    {
        if ( instance == null )
        {
            instance = buildService ();
        }
        return instance;
    }

    private static Storage buildService () throws IOException, GeneralSecurityException
    {

        HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport ();
        JsonFactory jsonFactory = new JacksonFactory ();

        GoogleCredential credential = GoogleCredential.fromStream (
                new URL ( "HERE GOES THE URL FOR YOUR SERVICE ACCOUNT JSON - I USED GOOGLE DRIVE DIRECT DOWNLOAD LINK TO MY JSON FILE" )
                        .openStream () );

        // Depending on the environment that provides the default credentials
        // (for
        // example: Compute Engine, App Engine), the credentials may require us
        // to
        // specify the scopes we need explicitly. Check for this case, and
        // inject
        // the Cloud Storage scope if required.
        if ( credential.createScopedRequired () )
        {
            Collection<String> scopes = StorageScopes.all ();
            credential = credential.createScoped ( scopes );
        }

        return new Storage.Builder ( transport, jsonFactory, credential ).setApplicationName ( "YOUR PROJECT NAME" ).build ();
    }
}

Step 3: Update you web.xml 

    <servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>PACKAGE.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

solution2
 2  2016-11-14 07:37:46

Here is the right way for uploading to storage according to the docs 

import com.google.appengine.tools.cloudstorage.*;

public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
  String buffer = "the data to be saved in GCS bla bla";
  GcsFileOptions instance = GcsFileOptions.getDefaultInstance();
  GcsFilename fileName = new GcsFilename("bucketName", "fileName");
  GcsOutputChannel outputChannel;
  gcsService.createOrReplace(fileName, instance, ByteBuffer.wrap(buffer.getBytes()));
}

you'll find the full code here

solution3
 1  2016-07-26 17:43:40

First you need to solve CORS problem in pre flight request processing, you need to do it on back end: on Google App Engine it done by adding doOptions method like: 

@Override
protected void doOptions(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException 
{ 
    resp.setHeader("Access-Control-Allow-Origin", "*");
    resp.setHeader("Access-Control-Allow-Methods", "*");
    resp.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
}

then you need to make sure that you send you request with header like Content-Type: multipart/form-data , otherwise your file will be encoded incorrectly. In angular2 request headers are set as third(optional) parameter in your post request like: 

let headers = new Headers();
headers.append('content-type', 'multipart/form-data');
http.post(url, body, {
                  headers:headers
                })

solution4
 1  2016-07-27 02:54:13

I'm sorry this does not directly address your question, but I want to point it out anyway. I recommend using one of your normal endpoints to generate a temporary upload URL that your Angular client can use to send the file directly to Cloud Storage, without going through your app. One way to do that is via the blobstore API, as described here . You can upload to Cloud storage via that API as described here (further down the same page).

This reduces the amount of upload code you need on your server, is not subject to the 32MB limit for GAE requests, and is in line with Google's recommendations (as seen in the documentation links above).

solution5
 1  2017-11-29 06:16:36

use com.google.cloud.storage

  • when running on appengine - no need for authentication

  • when running locally run gcloud auth application-default login command see authentication 

     import com.google.cloud.storage.*; public static void upload(String bucketName, String fileId, String content) throws IOException { Storage storage = StorageOptions.newBuilder().build().getService(); BlobInfo fileInfo = BlobInfo.newBuilder(bucketName, fileId) .build(); InputStream fileIS = IOUtils.toInputStream(content, "UTF-8"); storage.create(fileInfo, fileIS); }

暂无
暂无

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

Related Question Google Appengine Cloud Storage how to add metadata to file when uploading to google-cloud-storage via appengine Upload a file via google cloud Endpoints to google cloud Storage via Android Client File missing in Google Cloud Storage on AppEngine after following instructions Upload File (image or video) from JSP to google cloud storage via Google app engine(JAVA) Upload a file to Google Cloud Storage (Java) Google Cloud Storage upload a file using java Upload file to google cloud storage (JAVA) Google appengine cloud storage free quota Upload file to Google cloud storage from Google App Engine (Java)
Related Tags
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM