Crypto++ unexplained crashes

Question

I need to use cryptography in my project (visual c++ 2008 sp1, exe with several dlls, use some third party libs). I chose Crypto++ library. I use it as static library. First, I tested all functions I need in test console application, and everything works fine.

Then I started to integrate this function to the project.

And it crashes on many of Crypto++ function calls, including very simple like this one:

CryptoPP::FileSource file("publicKeySign.der", true);

During the call of this line of code, application crashes with:

Unhandled exception at 0x00c56619 in Starter.exe: 0xC0000005: Access violation reading location 0x00006f70.

Here is my call stack:

Starter.exe!CryptoPP::member_ptr<CryptoPP::AlgorithmParametersBase>::reset(CryptoPP::AlgorithmParametersBase * p=0x00000000)  Line 50 + 0x9 bytes   C++
Starter.exe!CryptoPP::AlgorithmParameters::operator()<char const *>(const char * name=0x00d8ec04, const char * const & value=0x00d88a44, bool throwIfNotUsed=true)  Line 356    C++
Starter.exe!CryptoPP::MakeParameters<char const *>(const char * name=0x00d8ec04, const char * const & value=0x00d88a44, bool throwIfNotUsed=true)  Line 388 + 0x2d bytes    C++
Starter.exe!CryptoPP::FileSource::FileSource(const char * filename=0x00d88a44, bool pumpAll=true, CryptoPP::BufferedTransformation * attachment=0x00000000, bool binary=true)  Line 65 + 0x6d bytes C++
Starter.exe!PDD::PDDApp2::Open()  Line 237  C++
Starter.exe!WinMain(HINSTANCE__ * __formal=0x00c50000, HINSTANCE__ * __formal=0x00c50000, HINSTANCE__ * __formal=0x00c50000, HINSTANCE__ * __formal=0x00c50000)  Line 387 + 0xb bytes   C++
Starter.exe!__tmainCRTStartup()  Line 578 + 0x1d bytes  C
kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes    
ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes   
ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes    

Both my project and Crypto++ use Multi-threaded DLL runtime libraries.

I tried to change Crypto++ project options for them to be the same as my project options, but it doesn't help (after some changes application crashes with another call stack).

Any ideas will be appreciated!

Answer 1

When I digged into cryptopp I found that it does some hidden validations. One of them was in fipstest.cpp. But try/catch inside of the library hides actual exception message.

When I got correct message I found that parameter passed was simply too short for specific algorithm (RSA). Those 64 bits were just taken from example on the web. When I changed the key to 1024 problem gone.

Cheers ;)

AutoSeededRandomPool prng;
RSA::PrivateKey privKey;
//privKey.GenerateRandomWithKeySize(prng, 64);  <- throws exception
privKey.GenerateRandomWithKeySize(prng, 1024);