OpenSSL command line decryption hiccups

Question

I hope everyone is doing well. I'm hoping someone could give me a pointer in the right direction here.

Around 2013 some time I encrypted a text file on a Mac OS machine using OpenSSL. Not really sure which version of OpenSSL or MacOS X it was.

I encrypted it using;

openssl des3 -in unencrypted.txt -out encrypted

The output of that seemed like some sort of binary format, so I used xxd to convert it to hex, which then output a text file which I saved somewhere.

xxd encrypted >> encryptedhex.txt

In order to reverse the process I used

xxd -r encryptedhex.txt encrypted.bin
openssl des3 -d -in encrypted.bin -out decrypted.txt

I remember testing this a few times and it seemed to work fine. Now fast forward to 2017.

I'm on a Debian 9 machine, and I'm trying to decrypt this file.

Whenever I supply the password that I think is correct, I get no error, but the output file seems to be garbage. I don't know if it's possible to get no error with incorrect passwords, but whenever I give a password that I know is wrong it spits out

bad decrypt 140492140782848:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:../crypto/evp/evp_enc.c:535:

I realise now there are better ways to have done this, but I'm stuck with what I have and it doesn't seem to want to work.

Are the newer versions of openssl backwards compatible? Is it possible that the encrypted file got corrupted? (If I look at the raw binary encrypted file I starts with 'salted__', so it looks like I remember it looking back then..)

Any suggestions that you guys can think of? Thank you!

Answer 1

Try adding -md md5 to your decryption command line. The default hash used for generating the key from a password changed from md5 to sha256 in OpenSSL 1.1.0:

https://www.openssl.org/docs/faq.html#USER3