Cannot access azure blobs through rest api

Question

I was able to create a Container in Storage Account and upload a blob to it through the Client Side Code.

I was able to make the blob available for Public access as well , such that when I hit the following query from my browser, I am able to see the image which I uploaded.

https://MYACCOUNT.blob.core.windows.net/MYCONTAINER/MYBLOB

I now have a requirement to use the rest service to retrieve the contents of the blob. I wrote down the following java code.

package main;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.TimeZone;

public class GetBlob {


public static void main(String[] args) {





    String url="https://MYACCOUNT.blob.core.windows.net/MYCONTAINER/MYBLOB";


    try {
        System.out.println("RUNNIGN");
    HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();


 connection.setRequestProperty("Authorization", createQuery());
 connection.setRequestProperty("x-ms-version", "2009-09-19");

        InputStream response = connection.getInputStream();
        System.out.println("SUCCESSS");
        String line;
        BufferedReader reader = new BufferedReader(new InputStreamReader(response));
        while ((line = reader.readLine()) != null) {
            System.out.println(line);
        }   



    } catch (IOException e) {
        e.printStackTrace();
    }

}
public static String createQuery()
{
    String dateFormat="EEE, dd MMM yyyy hh:mm:ss zzz";
    SimpleDateFormat dateFormatGmt = new SimpleDateFormat(dateFormat);
    dateFormatGmt.setTimeZone(TimeZone.getTimeZone("UTC"));
    String date=dateFormatGmt.format(new Date());


    String Signature="GET\n\n\n\n\n\n\n\n\n\n\n\n" +
            "x-ms-date:" +date+
            "\nx-ms-version:2009-09-19" ;

            // I do not know CANOCALIZED RESOURCE 
            //WHAT ARE THEY??
//          +"\n/myaccount/myaccount/mycontainer\ncomp:metadata\nrestype:container\ntimeout:20";

    String SharedKey="SharedKey";
    String AccountName="MYACCOUNT";

    String encryptedSignature=(encrypt(Signature));

    String auth=""+SharedKey+" "+AccountName+":"+encryptedSignature;

        return auth;

}





public static String encrypt(String clearTextPassword)   {  
    try {
      MessageDigest md = MessageDigest.getInstance("SHA-256");
      md.update(clearTextPassword.getBytes());
      return new sun.misc.BASE64Encoder().encode(md.digest());
    } catch (NoSuchAlgorithmException e) {
    }
    return "";
  }



}

However , I get the following error when I run this main class...

RUNNIGN
java.io.IOException: Server returned HTTP response code: 403 for URL:     https://klabs.blob.core.windows.net/delete/Blob_1
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at main.MainClass.main(MainClass.java:61)

Question1: Why this error, did I miss any header/parameter?

Question2: Do I need to add headers in the first place, because I am able to hit the request from the browser without any issues.

Question3: Can it be an SSL issue? What is the concept of certificates, and how and where to add them? Do I really need them? Will I need them later, when I do bigger operations on my blob storage(I want to manage a thousand blobs)?

Will be thankful for any reference as well, within Azure and otherwise that could help me understand better. :D

---

---

AFTER A FEW DAYS

---

---

Below is my new code for PutBlob I azure. I believe I have fully resolved all header and parameter issues and my request is perfect. However I am still getting the same 403. I do not know what the issue is. Azure is proving to be pretty difficult.

A thing to note is that the containers name is delete, and I want to create a blob inside it, say newBlob. I tried to initialize the urlPath in the code below with both "delete" and "delete/newBlob". Does not work..

package main;

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.HttpURLConnection;
import java.net.URISyntaxException;
import java.net.URL;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.TimeZone;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

import com.sun.org.apache.xml.internal.security.exceptions.Base64DecodingException;
import com.sun.org.apache.xml.internal.security.utils.Base64;

public class Internet {

    static String key="password";
    static String account="klabs";
private static Base64 base64 ; 
private static String createAuthorizationHeader(String canonicalizedString) throws InvalidKeyException, Base64DecodingException, NoSuchAlgorithmException, IllegalStateException, UnsupportedEncodingException     {  
          Mac mac = Mac.getInstance("HmacSHA256");  
          mac.init(new SecretKeySpec(base64.decode(key), "HmacSHA256"));  
          String authKey = new String(base64.encode(mac.doFinal(canonicalizedString.getBytes("UTF-8"))));  
          String authStr = "SharedKey " + account + ":" + authKey;  
          return authStr;  
     } 

public static void main(String[] args) {

System.out.println("INTERNET");
String key="password";
String account="klabs";
long blobLength="Dipanshu Verma wrote this".getBytes().length;
File f = new File("C:\\Users\\Dipanshu\\Desktop\\abc.txt");
String requestMethod = "PUT";  
String urlPath = "delete";
String storageServiceVersion = "2009-09-19";


SimpleDateFormat fmt = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:sss");  
fmt.setTimeZone(TimeZone.getTimeZone("UTC"));  
String date = fmt.format(Calendar.getInstance().getTime()) + " UTC";
String blobType = "BlockBlob"; 
String canonicalizedHeaders = "x-ms-blob-type:"+blobType+"\nx-ms-date:"+date+"\nx-ms-version:"+storageServiceVersion;  
String canonicalizedResource = "/"+account+"/"+urlPath;  

String stringToSign = requestMethod+"\n\n\n"+blobLength+"\n\n\n\n\n\n\n\n\n"+canonicalizedHeaders+"\n"+canonicalizedResource;

try {
    String authorizationHeader = createAuthorizationHeader(stringToSign);
    URL myUrl = new URL("https://klabs.blob.core.windows.net/" + urlPath);

    HttpURLConnection connection=(HttpURLConnection)myUrl.openConnection();
    connection.setRequestProperty("x-ms-blob-type", blobType);
    connection.setRequestProperty("Content-Length", String.valueOf(blobLength));
    connection.setRequestProperty("x-ms-date", date);
    connection.setRequestProperty("x-ms-version", storageServiceVersion);
    connection.setRequestProperty("Authorization", authorizationHeader);
    connection.setDoOutput(true);
    connection.setRequestMethod("POST");
    System.out.println(String.valueOf(blobLength));
    System.out.println(date);
    System.out.println(storageServiceVersion);
    System.out.println(stringToSign);
    System.out.println(authorizationHeader);
    System.out.println(connection.getDoOutput());


    DataOutputStream  outStream = new DataOutputStream(connection.getOutputStream());


       // Send request
       outStream.writeBytes("Dipanshu Verma wrote this");
       outStream.flush();
       outStream.close();
       DataInputStream inStream = new DataInputStream(connection.getInputStream());
       System.out.println("BULLA");

       String buffer;
       while((buffer = inStream.readLine()) != null) {
           System.out.println(buffer);
       }

       // Close I/O streams
       inStream.close();
       outStream.close();





} catch (InvalidKeyException | Base64DecodingException | NoSuchAlgorithmException | IllegalStateException | UnsupportedEncodingException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}
}

}

I know only a proper code reviewer might be able to help me, please do it if you can. Thanks

Answer 1

Question1: Why this error, did I miss any header/parameter?

Most likely you're getting this error is because of incorrect signature. Please refer to MSDN documentation for creating correct signature: http://msdn.microsoft.com/en-us/library/azure/dd179428.aspx . Unless your signature is correct you'll not be able to perform operations using REST API.

Question2: Do I need to add headers in the first place, because I am able to hit the request from the browser without any issues.

In your current scenario, because you can access the blob directly (which in turn means the container in which the blob exist has Public or Blob ACL) you don't really need to use REST API. You can simply make a HTTP request using Java and read the response stream which will have blob contents. You would need to go down this route if the container ACL is Private because in this case your requests need to be authenticated and the code above creates an authenticated request.

Question3: Can it be an SSL issue? What is the concept of certificates, and how and where to add them? Do I really need them? Will I need them later, when I do bigger operations on my blob storage(I want to manage a thousand blobs)?

No, it is not an SSL issue. Its an issue with incorrect signature.

Answer 2

Finally found the mistake!!

In the code above , I was using a String "password" as key for my SHA2

base64.decode(key)

It should have been the key associated with my account with AZURE. Silly One!! Took me 2 weeks to find.