decryption of MS office

Question

I'm working on decryption of encrypted MS Excel(RC4 encryption with SHA1),password is already known.In vs2010,I've could decrypt it correctly,however,my program hasto work under both Win and linux.And I have no idea to get the encryption key under linux right now,which is something like below under Win:

int getEncrypKey(HCRYPTKEY *hKey, int blocknum)
{
    //------------------------H0 = H(salt, password)-----
    BYTE *pbSaltandPwdHash = NULL;
    DWORD dwSaltandPwdLen = 0;

    pbSaltandPwdHash = SHA1_2(psalt, 16, ppwd, strlen(pwd)/2, &dwSaltandPwdLen);
    printf("SHA1 of SaltandPwd:\n");
    for(DWORD i = 0 ; i < dwSaltandPwdLen ; i++) {
    printf("%2.2x ",pbSaltandPwdHash[i]);
    }
    printf("\n");
    //------------------------H0 = H(salt, password)-----

    //------------------------Hfinal = H(H0, block)-----
    HCRYPTHASH hHash1 = 0;

    CryptCreateHash( hCryptProv, CALG_SHA1, 0, 0, &hHash1) ;
    CryptHashData( hHash1, pbSaltandPwdHash, dwSaltandPwdLen, 0) ;
    CryptHashData( hHash1, (unsigned char*)&blocknum, sizeof(blocknum), 0) ;
    //------------------------Hfinal = H(H0, block)-----

    CryptDeriveKey(hCryptProv, CALG_RC4, hHash1, 0x00280000, hKey);

    if(hHash1 != 0) CryptDestroyHash(hHash1);
    if(pbSaltandPwdHash != NULL) free(pbSaltandPwdHash);

    return 0;
} 

I knew how to get H0 under linux,but I dont know how to get the hHash1 and hKey .

Answer 1

This post sounds like it does the same thing: Implement Windows CryptoAPI CryptDeriveKey Using OpenSSL APIs

A more general way of generating hashes in openssl is below:

Before you do anything:

#include <ssl/evp.h>

int main(int argc, char argv[]) // or in an "initialise" type function
{
     OpenSSL_add_all_digests()
     ...
}

Then to generate the hash ( error checking omitted ):

const EVP_MD *digest;
EVP_MD_CTX context;
unsigned char hash[EVP_MAX_MD_SIZE];
unsigned int hash_len;

digest = EVP_get_digestbyname("sha1"); /* choose the hash type here */

EVP_MD_CTX_init(&context);
EVP_DigestInit_ex(&contxt, digest, NULL);
EVP_DigestUpdate(&context, pbSaltandPwdHash, dwSaltandPwdLen);
EVP_DigestUpdate(&context, &blocknum, sizeof(blocknum));
EVP_DigestFinal_ex(&context, hash, &hash_len);
EVP_MD_CTX_cleanup(&context);

/* Now use hash and hash_len as required */