Decrypting Crypto++ RSA cipher text in C# causes exception

Question

I've written 3 functions in C++ using Crypto++ to generate key pairs, encrypt and decrypt a string. Crypto++ side :

//Decode public key
RSA::PublicKey pbKeyDecoded;
StringSource ss2(publicKey, true, new Base64Decoder);
pbKeyDecoded.BERDecode(ss2);

Integer m = Integer((const byte*)plaintext.data(), plaintext.size());
Integer crypted = pbKeyDecoded.ApplyFunction(m);
...

What I do is, generate the key, DER Encode it, and then encode it to Base64. After than, I'm ciphering a plaintext via the public key and save both the private key and the cipher as base64 encoded strings in two separate files.

Now to C#. I'm reading the base64 string, decoding them and load them via AsnParser, which seem to load just fine. Then I call Decrypt . C# side :

AsnKeyParser keyParser = new AsnKeyParser("rsa-public.der");
RSAParameters publicKey = keyParser.ParseRSAPublicKey();

CspParameters csp = new CspParameters;
csp.KeyContainerName = "RSA Test (OK to Delete)";    
csp.ProviderType = PROV_RSA_FULL;    // 1
csp.KeyNumber = AT_KEYEXCHANGE;      // 1

RSACryptoServiceProvider rsp = new RSACryptoServiceProvider(csp);
rsp.PersistKeyInCsp = false;
rsp.ImportParameters(privateKey);

//Causes exception here..
var data = rsp.Decrypt(cipherArr, true);
...

But I'm getting exception error when I try to decrypt it with fOAEP = true : CryptographicException: Error occurred while decoding OAEP padding. If I pass fOAEP = false then I get CryptographicException: The parameter is incorrect.

Why am I getting an exception in C# when attempting to decrypt the Crypto++ cipher text?

Answer 1

... I'm getting exception error when I try to decrypt it: CryptographicException: Error occurred while decoding OAEP padding. That's if I pass true for the fOAEP bool, if I pass false to it I get CryptographicException: The parameter is incorrect.

You are having the same problem as Encrypt and Decrypt a message using raw RSA algorithim in Crypto++? and How to sync Crypto++ RSA with C# RSA crypto service provider? It must be our month for the "Raw RSA" schemes...

On the Crypto++ side of the equation, you are performing raw RSA . You are simply applying the forward function, which is exponentiation, and you are not formatting the message:

//Decode public key
RSA::PublicKey pbKeyDecoded;
StringSource ss2(publicKey, true, new Base64Decoder);
pbKeyDecoded.BERDecode(ss2);

Integer m = Integer((const byte*)plaintext.data(), plaintext.size());
Integer crypted = pbKeyDecoded.ApplyFunction(m);
...

On the C# side of things, you are performing RSA decryption using PKCS #1 with either PKCS #1.5 padding or OAEP padding:

RSACryptoServiceProvider rsp = new RSACryptoServiceProvider(csp);
rsp.PersistKeyInCsp = false;
rsp.ImportParameters(privateKey);

//Causes exception here..
var data = rsp.Decrypt(cipherArr, true);

Its not clear to me if the C# version of your code can perform OAEP padding because its it requires a certain version of the CLR. You may only have PKCS padding available.

---

I believe you have two choices. First, you can use a standard RSA encryption method in Crypto++. The Crypto++ wiki lists them at RSA Cryptography and RSA Encryption Schemes :

typedef RSAES<PKCS1v15>::Decryptor RSAES_PKCS1v15_Decryptor;
typedef RSAES<PKCS1v15>::Encryptor RSAES_PKCS1v15_Encryptor;

typedef RSAES<OAEP<SHA> >::Decryptor RSAES_OAEP_SHA_Decryptor;
typedef RSAES<OAEP<SHA> >::Encryptor RSAES_OAEP_SHA_Encryptor;

Second, you need to perform Raw RSA in C#. To perform Raw RSA in C#, you will need to get a BigInteger class and apply the inverse function manually.

I would encourage you to use RSA Encryption with OAEP padding. If OAEP is not available, then the second choice would be PKCS padding. Finally, if all you have is Raw RSA, then I would look for another encryption system because Raw RSA is so insecure.