Translate Windows RC4 CryptDeriveKey to PHP for openssl

Question

This is the second component of the legacy system translation we've been trying to do. We have managed to match exactly the initial binary password/key that Windows ::CryptHashData generates.

That password/key is passed to ::CryptDeriveKey where it performs a number of steps to create the final key to be used by ::CryptEncrypt. My research has led me to the CryptDeriveKey documentation where it clearly describes the steps required to derive the key for ::CryptEncrypt but so far I haven't been able to get it to decrypt the file on the PHP side. https://docs.microsoft.com/en-us/windows/desktop/api/wincrypt/nf-wincrypt-cryptderivekey

Based on the ::CryptDeriveKey documentation there may be some additional undocumented steps for our specific legacy key size that may not be well understood. The current Windows ::CryptDeriveKey is set for ZERO SALT by default which is apparently different from NO_SALT somehow. See salt value functionality here: https://docs.microsoft.com/en-us/windows/desktop/SecCrypto/salt-value-functionality

The parameters on the CryptAPI for our legacy system are as follows:

Provider type: PROV_RSA_FULL

Provider name: MS_DEF_PROV

Algo ID CALG_RC4

Description RC4 stream encryption algorithm

Key length: 40 bits.

Salt length: 88 bits. ZERO_SALT

Special Note: A 40-bit symmetric key with zero-value salt, however, is not equivalent to a 40-bit symmetric key without salt. For interoperability, keys must be created without salt. This problem results from a default condition that occurs only with keys of exactly 40 bits.

I'm not looking to export the key, but reproduce the process that creates the final encryption key that is passed to ::CryptEncrypt for the RC4 encryption algorithm and have it work with openssl_decrypt.

Here is the current windows code that's working fine for encrypt.

try {
    BOOL bSuccess;
    bSuccess = ::CryptAcquireContextA(&hCryptProv, 
                                      CE_CRYPTCONTEXT, 
                                      MS_DEF_PROV_A, 
                                      PROV_RSA_FULL, 
                                      CRYPT_MACHINE_KEYSET);

    ::CryptCreateHash(hCryptProv, 
                      CALG_MD5, 
                      0, 
                      0, 
                      &hSaveHash);

    ::CryptHashData(hSaveHash, 
                    baKeyRandom, 
                    (DWORD)sizeof(baKeyRandom), 
                    0);

    ::CryptHashData(hSaveHash, 
                    (LPBYTE)T2CW(pszSecret), 
                    (DWORD)_tcslen(pszSecret) * sizeof(WCHAR), 
                     0);

    ::CryptDeriveKey(hCryptProv, 
                     CALG_RC4, 
                     hSaveHash, 
                     0, 
                     &hCryptKey);

    // Now Encrypt the value
    BYTE * pData = NULL;
    DWORD dwSize = (DWORD)_tcslen(pszToEncrypt) * sizeof(WCHAR); 
    // will be a wide str
    DWORD dwReqdSize = dwSize;

    ::CryptEncrypt(hCryptKey, 
                   NULL, 
                   TRUE, 
                   0, 
                   (LPBYTE)NULL, 
                   &dwReqdSize, 0);

    dwReqdSize = max(dwReqdSize, dwSize);

    pData = new BYTE[dwReqdSize];

    memcpy(pData, T2CW(pszToEncrypt), dwSize);

    if (!::CryptEncrypt(hCryptKey, 
                        NULL, 
                        TRUE, 
                        0, 
                        pData, 
                        &dwSize, 
                        dwReqdSize)) {

            printf("%l\n", hCryptKey);
            printf("error during CryptEncrypt\n");
            }

    if (*pbstrEncrypted)
    ::SysFreeString(*pbstrEncrypted);
    *pbstrEncrypted = ::SysAllocStringByteLen((LPCSTR)pData, dwSize);
    delete[] pData;
    hr = S_OK;
}

Here is the PHP code that tries to replicate the ::CryptDeriveKey function as described in the documentation.

Let n be the required derived key length, in bytes. The derived key is the first n bytes of the hash value after the hash computation has been completed by CryptDeriveKey. If the hash is not a member of the SHA-2 family and the required key is for either 3DES or AES, the key is derived as follows:

1. Form a 64-byte buffer by repeating the constant 0x36 64 times. Let k be the length of the hash value that is represented by the input parameter hBaseData. Set the first k bytes of the buffer to the result of an XOR operation of the first k bytes of the buffer with the hash value that is represented by the input parameter hBaseData.

2. Form a 64-byte buffer by repeating the constant 0x5C 64 times. Set the first k bytes of the buffer to the result of an XORoperation of the first k bytes of the buffer with the hash value that is represented by the input parameter hBaseData.

3. Hash the result of step 1 by using the same hash algorithm as that used to compute the hash value that is represented by the hBaseData parameter.

4. Hash the result of step 2 by using the same hash algorithm as that used to compute the hash value that is represented by the hBaseData parameter.

5. Concatenate the result of step 3 with the result of step 4.

6. Use the first n bytes of the result of step 5 as the derived key.

PHP Version of ::CryptDeriveKey.

function cryptoDeriveKey($key){

    //Put the hash key into an array
    $hashKey1 = str_split($key,2);
    $count = count($hashKey1);
    $hashKeyInt = array();

    for ($i=0; $i<$count; $i++){
        $hashKeyInt[$i] = hexdec($hashKey1[$i]);
    }
    $hashKey = $hashKeyInt;

    //Let n be the required derived key length, in bytes.  CALG_RC4 = 40 bits key or 88 salt bytes
    $n = 40/8;

    //Let k be the length of the hash value that is represented by the input parameter hBaseData
    $k = 16;

    //Step 1 Form a 64-byte buffer by repeating the constant 0x36 64 times   
    $arraya = array_fill(0, 64, 0x36);

    //Set the first k bytes of the buffer to the result of an XOR operation of the first k bytes of the buffer with the hash value 
    for ($i=0; $i<$k; $i++){
        $arraya[$i] = $arraya[$i] ^ $hashKey[$i];
    }

    //Hash the result of step 1 by using the same hash algorithm as hBaseData
    $arrayPacka = pack('c*', ...$arraya);
    $hashArraya = md5($arrayPacka);

    //Put the hash string back into the array
    $hashKeyArraya = str_split($hashArraya,2);
    $count = count($hashKeyArraya);
    $hashKeyInta = array();
    for ($i=0; $i<$count; $i++){
        $hashKeyInta[$i] = hexdec($hashKeyArraya[$i]);
    }

    //Step 2 Form a 64-byte buffer by repeating the constant 0x5C 64 times. 
    $arrayb = array_fill(0, 64, 0x5C);

    //Set the first k bytes of the buffer to the result of an XOR operation of the first k bytes of the buffer with the hash value
    for ($i=0; $i<$k; $i++){
        $arrayb[$i] =  $arrayb[$i] ^ $hashKey[$i];
    }

    //Hash the result of step 2 by using the same hash algorithm as hBaseData    
    $arrayPackb = pack('c*', ...$arrayb);
    $hashArrayb = md5($arrayPackb);

    //Put the hash string back into the array
    $hashKeyArrayb = str_split($hashArrayb,2);
    $count = count($hashKeyArrayb);
    $hashKeyIntb = array();
    for ($i=0; $i<$count; $i++){
        $hashKeyIntb[$i] = hexdec($hashKeyArrayb[$i]);
    }

    //Concatenate the result of step 3 with the result of step 4.
    $combined = array_merge($hashKeyInta, $hashKeyIntb);

    //Use the first n bytes of the result of step 5 as the derived key.
    $finalKey = array();
    for ($i=0; $i <$n; $i++){
        $finalKey[$i] =  $combined[$i];
    }
    $key = $finalKey;

    return $key;
}

PHP Decrypt Function

function decryptRC4($encrypted, $key){
    $opts = OPENSSL_RAW_DATA | OPENSSL_ZERO_PADDING;
    $cypher = ‘rc4-40’;
    $decrypted = openssl_decrypt($encrypted, $cypher, $key, $opts);
    return $decrypted; 
}

So here are the big questions:

Has anyone been able to successfully replicate ::CryptDeriveKey with RC4 on another system?

Does anyone know what is missing from the PHP script we created that prevents it from creating the same key and decrypt the Windows CryptoAPI encrypted file with openssl_decrypt?

Where and how do we create the 88 bit zero-salt that is required for the 40bit key?

What are the correct openssl_decrypt parameters that would accept this key and decrypt what was generated by ::CryptDeriveKey?

Yes, we know this isn't secure and its not being used for passwords or PII. We would like to move away from this old and insecure method, but we need take this interim step of translating the original encryption to PHP first for interoperability with the existing deployed systems. Any help or guidance would be appreciated.

Answer 1

Just in case anyone else wanders down this path here are the answers to all the questions above.

You can replicate ::CryptDeriveKey on PHP using openssl but there are some prerequisites that have to be met on the windows side first.

CryptDeriveKey MUST be set to CRYPT_NO_SALT as follows:

::CrypeDeriveKey(hCryptProv, CALG_RC4, hSaveHash, CRYPT_NO_SALT, &hCryptKey)

This will allow you to create a key from your hash and generate a matching key in PHP that will work on openssl. If you don't set any salt parameters you will get a key that is created with an unknown proprietary salt algorithm that cant be matched on another system.

The reason that you have to set CRYPT_NO_SALT is because both the CryptAPI and openssl have proprietary salt algorithms and there is no way to get them to match. So you should do your salting separately. There are more details about this salt value functionality here: https://docs.microsoft.com/en-us/windows/desktop/SecCrypto/salt-value-functionality

Here is what the PHP script needs to look like to create an equivalent passkey for for openssl to use.

<?php
$random = pack('c*', 87,194,...........);
$origSecret = 'ASCII STRING OF CHARACTERS AS PASSWORD'; 

//Need conversion to match format of Windows CString or wchar_t*
//Windows will probably be UTF-16LE and LAMP will be UTF-8
$secret = iconv('UTF-8','UTF-16LE', $origSecret);

//Create hash key from Random and Secret
//This is basically a hash and salt process.
$hash = hash_init("md5");
hash_update($hash, $random);
hash_update($hash, $secret);
$key = hash_final($hash);

$key = cryptoDeriveKey($key);
//Convert the key hex array to a hex string for openssl_decrypt
$count = count($key);
$maxchars = 2;
for ($i=0; $i<$count; $i++){
    $key .= str_pad(dechex($key[$i]), $maxchars, "0", STR_PAD_LEFT);
}

IMPORTANT: OpenSSL expects the key to be the raw hex values that are derived from the hash, unfortunately openssl_decrypt() wants the same value as a string or password. Therefor you have to do a hex to string conversion at this point. There is a great write up here on why you have to do this. http://php.net/manual/en/function.openssl-encrypt.php

$opts = OPENSSL_RAW_DATA | OPENSSL_ZERO_PADDING;
//Convert key hex string to a string for openssl_decrypt
//Leave it as it is for openssl command line.
$key = hexToStr($key);
$cipher = 'rc4-40';
$encrypted = “the data you want to encrypt or decrypt”;
$decrypted = openssl_decrypt($encrypted, $cipher, $key, $opts);  

echo $decrypted;  //This is the final information you’re looking for


function cryptoDeriveKey($key){
//convert the key into hex byte array as int
    $hashKey1 = str_split($key,2);
    $count = count($hashKey1);
    $hashKeyInt = array();
    for ($i=0; $i<$count; $i++){
        $hashKeyInt[$i] = hexdec($hashKey1[$i]);
    }
    $hashKey = $hashKeyInt;
    //Let n be the required derived key length, in bytes.  CALG_RC4 = 40 bits key with 88 salt bits
    $n = 40/8;
    //Chop the key down to the first 40 bits or 5 bytes.
    $finalKey = array();
    for ($i=0; $i <$n; $i++){
        $finalKey[$i] =  $hashKey[$i];
    }
    return $finalKey;
}


function hexToStr($hex){
    $string='';
    for ($i=0; $i < strlen($hex)-1; $i+=2){
        $string .= chr(hexdec($hex[$i].$hex[$i+1]));
    }
return $string;
}
?>

If you're having trouble getting the correct values after using the code above you can try exporting your key value from CryptoAPI and testing it with openssl command line.

First you have to set CryptDeriveKey to allow the key to be exported with CRYPT_EXPORTABLE and CRYPT_NO_SALT

::CrypeDeriveKey(hCryptProv, CALG_RC4, hSaveHash, CRYPT_EXPORTABLE | CRYPT_NO_SALT, &hCryptKey)

If you want to know how to display a PLAINTEXTKEYBLOB from the exported key follow this link. https://docs.microsoft.com/en-us/windows/desktop/seccrypto/example-c-program--importing-a-plaintext-key

Here is an example exported key blob 0x08 0x02 0x00 0x00 0x01 0x68 0x00 0x00 0x05 0x00 0x00 0x00 0xAA 0xBB 0xCC 0xDD 0xEE

0x08 0x02 0x00 0x00 0x01 0x68 0x00 0x00 //BLOB header matches almost exactly 0x05 0x00 0x00 0x00 //Key length in bytes is correct 5 bytes 0xAA 0xBB 0xCC 0xDD 0xEE //First 5 bytes of our created hash key!!

Use your exported key value from the BLOB as the Hex Key Value in the openssl enc command below.

openssl enc -d -rc4-40 -in testFile-NO_SALT-enc.txt -out testFile-NO_SALT-dec.txt -K "Hex Key Value" -nosalt -nopad

This will decrypt the file that was encrypted on the Windows machine using CryptEncrypt.

As you can see, when you set the CryptDeriveKey to CRYPT_NO_SALT all you need for the openssl password or key is the first “keylength” bits of your CryptHashData password. Simple enough to say but a real pain to get to. Good luck and hope this helps someone else with legacy Windows translation issues.