How to generate RSA SHA signature using OpenSSL in C

Question

I need some help using OpenSSL to generate a signature of a block of data using C (Windows and Linux). The application has to do with Google authentication. The instructions from Google's documentation are:

"Sign the UTF-8 representation of the input using SHA256withRSA (also known as RSASSA-PKCS1-V1_5-SIGN with the SHA-256 hash function) with the private key obtained from the Google Developers Console. The output will be a byte array."

The private key is a .p12 file containing 1660 bytes of binary data.

Do I use the RSA_sign() API for this? If so am I supposed to hash the data first? Using what key value? I assume the RSA * parameter is supposed to hold the private key? How does it get loaded? I have used the HMAC function to generate a SHA hash, but I'm a bit lost here - any help or sample code would be appreciated. (And yes, I know Google has libraries to do this, but none for C, which I need to use).

Answer 1

You can use RSA_sign to sign the data with SHA256 hash. You can call this

RSA_sign(NID_sha256, digest, digest_len, &sign_buffer, sign_len, rsa_key);

You have calculate SHA256 hash of the data into digest buffer. rsa_key should be initialized.

Since, you have .p12 file, you need to do the following.

Create PKCS12 structure from .p12 file.

PKCS12 * p12 = d2i_PKCS12_fp(fp_to_p12_file, NULL);

Parse the p12 structure to certificate and key.

PKCS12_parse(p12, passphrase, &evp_pkey, &x509_cert, NULL);

It will give EVP_PKEY structure from which you can get RSA structure using EVP_PKEY_get1_RSA or access evp_pkey->pkey.rsa member of the structure.

This should help you to start.