Copy between two Google Cloud Storage buckets using Java

Question

To Copy between two Google Cloud Storage buckets, need equivalent of the below command from Java SDK

gsutil cp gs://bucket-name/filename gs://bucket-name

I have 2 options to use one of the below. Experts can you suggest If there is any better approach?

1. Java.lang.Runtime.exec()

2. Java.lang.ProcessBuilder

Answer 1

As is mentioned in the documentation you can use the Java Client library to copy objects between buckets as direct alternative of the command

gsutil cp gs://SOURCE_BUCKET_NAME/SOURCE_OBJECT_NAME gs://DESTINATION_BUCKET_NAME/NAME_OF_COPY

This is the example code, and you can check and download the full example project from github , please check the readme file to know which dependecies are necesaries

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class CopyObject {
  public static void copyObject(
      String projectId, String sourceBucketName, String objectName, String targetBucketName) {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of the bucket the original object is in
    // String sourceBucketName = "your-source-bucket";

    // The ID of the GCS object to copy
    // String objectName = "your-object-name";

    // The ID of the bucket to copy the object to
    // String targetBucketName = "target-object-bucket"

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
    Blob blob = storage.get(sourceBucketName, objectName);

    // This keeps the original name, you could also do
    // copyTo(targetBucketName, "target-object-name") to change the name
    blob.copyTo(targetBucketName);

    System.out.println(
        "Copied object "
            + objectName
            + " from bucket "
            + sourceBucketName
            + " to "
            + targetBucketName);
  }
}

Answer 2

If 1) you want the blob (file) names inside the buckets to remain the same and 2) You have several (>100) files to copy the best way to do this is to use a transfer job.

This job is made by google with their own equipment, and it's much faster that copying files one by one from one bucket to another because they can talk with the underlying storage.

Take a lookt at the Google-provided java sample code for transfer job and API Documentation .