Decrypt RSA from Windows Crypto API with OpenSSL/PHP Crypto Libs

Question

I've developed a C application with a component that Encrypts a small plaintext with an RSA 2048 key. The encryption is done using the Windows Enhanced crypto api. It seems to work perfectly and outputs a 128 byte Ciphertext to file.

I have no idea what sort of cryptographic form the Ciphertext is in (Whether it's padded, encoding etc) (I'm just importing the public key and encryptiong with it (No extra settings))

Anyway I need to be able to decrypt the Ciphertext back to plaintext using the private key with a php script.

I've tried using openssl over command line during testing but I got a data greater than mod len error.

Are there any transformations I need to do with the Ciphertext before trying to decrypt it or any settings I need to specify with the openssl_private_decrypt. (I currently have the private key in pem format but I can convert it to many more formats)

Also any suggestions for other php libraries that might be more effective for this than openssl would be appreciated.

Thanks.

Answer 1

If you want to decrypt the data using OpenSSL, reverse the data byte by byte. This is because OpenSSL considers them as big endian.

This question may help you. I faced similar problem for calculating hash. When I reversed the byte order, it works perfectly in OpenSSL. So, this might help you.