Decrypt OpenSSL encrypted data using java

Question

I am trying to understand OpenSSL encryption. The requirement for me is to decrypt the data encrypted using the below code. It is creating an anonymous ID for receiver. I need to get received ID back.

Based on the documentation I understood encryption is happening using EVP_EncryptFinal . But what is PKCS5_PBKDF2_HMAC_SHA1 doing ?

The code is using KEY and IV.

int create_anon_id(uint32_t recvr_id, uint32_t smartcard_id, const char *hw_id, unsigned char *anon_id)
{
    EVP_CIPHER_CTX ctx;
    unsigned char ibuf[sizeof(recvr_id) + sizeof(smartcard_id)] = {};
    unsigned char obuf[sizeof(ibuf) * 2] = {};
    int olen = sizeof(obuf);

    /* Convert to big endian. */
    recvr_id = bswap32(recvr_id);
    smartcard_id = bswap32(smartcard_id);
    hw_id = bswap32(hw_id);

    /* Fill input buffer with recvr_id | smartcard_id */
    memcpy(ibuf, &recvr_id, sizeof(recvr_id));
    memcpy(ibuf + sizeof(recvr_id), &smartcard_id, sizeof(smartcard_id));

    if (EVP_EncryptInit(&ctx, EVP_des_ede3_cbc(), KEY, IV) == 1)
    {
        if (EVP_EncryptUpdate(&ctx, obuf, &olen, ibuf, sizeof(ibuf)) == 1)
        {
            olen = sizeof(obuf) - olen;
            if (EVP_EncryptFinal(&ctx, obuf + sizeof(obuf) - olen, &olen) == 1)
            {
                return PKCS5_PBKDF2_HMAC_SHA1((const char *)obuf, olen, (unsigned char *)hw_id, HW_ID_SIZE, ROUNDS, ANON_ID_BIN_SIZE, anon_id);
            }
        }
    }
}

Answer 1

That code is quite poor, although by bad luck some of the errors are masked by using an input which is exactly one triple-DES data block (64 bits or 8 bytes). If coded correctly the encryption would be done by both EVP_EncryptUpdate and EVP_EncryptFinal ; as written the only encryption actually used, of the unpadded block, is done by Update .

But the value returned in anon_id is NOT (JUST) ENCRYPTED . PKCS5_PBKDF2_HMAC[_SHA1] is an implementation of the PBKDF2 algorithm and as used here it computes essentially a slow hash of

• recvr_id concatenated with smartcard_id (update: apparently not swapped) then triple-DES encrypted with KEY and IV whatever those are in this code, as the 'passphrase' with

• hw_id (update: apparently not swapped, and thus 'accidentally' type-valid) as the salt,

• and PRF = HMAC-SHA1 for ROUNDS (update: 10001) iterations

• (update) expanded slightly to ANON_ID_BIN_SIZE (22) bytes. (This requires an extra 'pass' of PBKDF2 which actually hurts security; it slows the defender without slowing the attacker.)

PBKDF2 with a decent PRF (HMAC-SHA1 is) is a one-way function and cannot be reversed , as encryption can if you know or guess the key. It is also designed to be costly to bruteforce if the salt and iteration count are chosen well. 32-bits of salt is definitely on the low side, and (update) ROUNDS 10001 is somewhat on the low side but not crazy.

As a better-case example (for you as attacker) if the 32-bit recvr_id and smartcard_id each contain only 16-bits of entropy, and so does hw_id , and ROUNDS was 1000 which was the recommended value when rfc2898 was written at the turn of the century, you'd have to try up to 256 trillion possibilities and a typical desktop computer can do maybe 30 thousand per second in CPU, and with some luck and skill maybe a thousand times that in GPU, so that will take about 3 months.

On the other hand, if the 32-bit inputs are fully entropic and ROUNDS was 1000000, it is impossible with just a computer. If you have a few million dollars to spend on FPGAs (or even better ASICs), and the output of a power plant (or maybe your own hydroelectric dam) to run them, it will still take many millions of years.

Update: with ROUNDS 10001 you are most likely in between these situations, but just where depends strongly on the entropy of the inputs.

And in general this may not produce the desired result anyway. If the output size is less than or even roughly equal to the entropy of the inputs, there will be collisions -- multiple input values (combinations) that produce the desired output, possibly a great many (millions, billions, etc) of such inputs. If you need the original input, rather than merely some input that produces (forges) a known output, you'll need some other method or criteria to select among the possible inputs that bruteforcing finds. Update: for output 22 bytes with at most 96 bits entropy, this risk is negligible .