Verify RSA signature in c++ using openssl

Question

I have a public key and a signature of some message, in the form of a byte array. I'm trying to verify the signature using the public key. That's what I tried:

int test2()
{
    const unsigned char *MESSAGE = (unsigned char *) "My message";
    const int MESSAGE_LEN = 10;
    unsigned char pubKey[] = {0x2D, 0x7E, 0xDC, 0xFE, 0x5B, 0x17, 0xE4, 0x30, 0x64, 0x09, 0x75,
                              0xCA, 0xBB, 0x6C, 0x8D, 0xAE, 0x93, 0xB3, 0x0F, 0xA0, 0x0F, 0x59,
                              0xE2, 0xA5, 0x2F, 0x66, 0x45, 0x3D, 0x89, 0x6D, 0x66, 0x60, 0x60,
                              0x5B, 0x83, 0xC2, 0xA7, 0xC8, 0x01, 0x52, 0x2D, 0xA9, 0x62, 0xC2,
                              0x1C, 0xED, 0x85, 0x98, 0xD1, 0xC2, 0x84, 0x01, 0xF4, 0x45, 0xFA,
                              0x2C, 0x78, 0x05, 0x99, 0x1F, 0xCF, 0x96, 0xDA, 0xC5, 0xC0, 0x46,
                              0x26, 0xF8, 0x0D, 0x31, 0x01, 0x42, 0x77, 0x46, 0x9C, 0x5B, 0xC5,
                              0x9D, 0x0D, 0x68, 0xBA, 0x1F, 0x11, 0x89, 0xCC, 0xC3, 0x75, 0xC6,
                              0x17, 0x4B, 0xFD, 0x1E, 0x1A, 0x19, 0x32, 0x23, 0x2F, 0x9D, 0xAB,
                              0x51, 0xFB, 0x5A, 0xD3, 0xC7, 0xCE, 0xB0, 0xDE, 0x07, 0x01, 0xB1,
                              0xB7, 0x90, 0x28, 0x36, 0x9F, 0x15, 0xEB, 0x07, 0x6E, 0x6F, 0x5A,
                              0x1B, 0x9E, 0x22, 0xC4, 0x02, 0x8F, 0x90, 0x47, 0xEE, 0x78, 0xF4,
                              0xE8, 0x1D, 0x63, 0xD7, 0x56, 0x2C, 0x13, 0x71, 0xCD, 0xCD, 0x84,
                              0x50, 0x1D, 0x1C, 0x36, 0x56, 0xD5, 0xDE, 0x20, 0x3A, 0xB5, 0xE8,
                              0x1B, 0x9A, 0x89, 0xDE, 0x51, 0xAB, 0x1A, 0x0C, 0x51, 0x08, 0xD2,
                              0x22, 0xB0, 0x74, 0x6D, 0xA3, 0x84, 0x11, 0xF5, 0x91, 0x27, 0xAB,
                              0x05, 0xB4, 0x01, 0x14, 0xBF, 0xB4, 0x03, 0xD2, 0x38, 0x89, 0xDA,
                              0xA6, 0x16, 0xBF, 0xB4, 0x0B, 0xD2, 0x7A, 0xFF, 0xB0, 0x0F, 0x42,
                              0x8A, 0x0A, 0x32, 0xD4, 0x28, 0x77, 0x54, 0xA0, 0xA0, 0xFB, 0x96,
                              0xB1, 0x69, 0x08, 0x43, 0xB0, 0x25, 0x47, 0x01, 0x58, 0xA0, 0xBE,
                              0xB3, 0xDF, 0xC4, 0x51, 0xE2, 0x62, 0x2D, 0x95, 0xB3, 0x8D, 0x80,
                              0x5C, 0x4E, 0x24, 0xC3, 0x2F, 0x9F, 0x6D, 0x7A, 0x02, 0xB7, 0xD9,
                              0x7F, 0xEE, 0xF1, 0x1C, 0xAF, 0x51, 0xBE, 0x45, 0x5E, 0x85, 0x2E,
                              0x05, 0x74, 0xB9};
    unsigned char exp[] = {0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00};
    unsigned char signature[] = {0x1A, 0x84, 0x32, 0x9F, 0x73, 0x3F, 0x46, 0xDF, 0x63, 0x41, 0xB4,
                                 0xDC, 0xC5, 0x7B, 0x5B, 0x9E, 0xA8, 0x98, 0xFD, 0xE5, 0x65, 0x96,
                                 0xD5, 0xF4, 0xEC, 0xFB, 0x19, 0x76, 0x0F, 0x7E, 0x23, 0xAA, 0xB8,
                                 0x54, 0x87, 0x08, 0x94, 0xAB, 0x8E, 0x00, 0x76, 0x99, 0xB9, 0x17,
                                 0xFE, 0x35, 0x39, 0xD4, 0x10, 0x95, 0x82, 0xB4, 0x7E, 0x7C, 0xC3,
                                 0x17, 0xCD, 0x51, 0x89, 0x2D, 0xCC, 0x71, 0xEC, 0xA2, 0x0E, 0x33,
                                 0xFA, 0xB9, 0x77, 0x3D, 0xD5, 0xD5, 0xE7, 0x6C, 0x6C, 0x64, 0xDE,
                                 0xEA, 0x10, 0x6A, 0xC9, 0x5E, 0x82, 0x4D, 0x00, 0xF6, 0x55, 0x45,
                                 0x37, 0xEC, 0xF1, 0x61, 0xD4, 0x68, 0x65, 0x42, 0xAE, 0xF3, 0xC6,
                                 0x51, 0xBC, 0xF3, 0x27, 0x1F, 0x02, 0x45, 0x05, 0x20, 0xD5, 0x0B,
                                 0x2B, 0x82, 0x39, 0x49, 0x29, 0x1E, 0x7E, 0xC0, 0xB5, 0xE3, 0xB2,
                                 0x46, 0xCF, 0x20, 0xAE, 0x9C, 0xE9, 0x51, 0x29, 0xAA, 0x4F, 0xB0,
                                 0xAD, 0xBF, 0x8C, 0x45, 0x70, 0xB1, 0x66, 0xA5, 0xAE, 0x56, 0x9E,
                                 0xA6, 0x9E, 0x57, 0x21, 0x92, 0xEE, 0xFB, 0x59, 0xA3, 0x6D, 0x61,
                                 0x3A, 0xC8, 0xCA, 0xFD, 0x53, 0xA8, 0xFB, 0x54, 0x12, 0x5F, 0x28,
                                 0x3E, 0x6D, 0x01, 0x3F, 0x19, 0xD9, 0x2B, 0x1E, 0xC2, 0x30, 0xF2,
                                 0x0E, 0xA5, 0x87, 0xEC, 0x12, 0x5A, 0x7E, 0x96, 0xDF, 0x75, 0xA9,
                                 0x92, 0xDF, 0x65, 0x96, 0x45, 0xFA, 0xDF, 0xDF, 0x8C, 0x53, 0xF4,
                                 0x08, 0xAB, 0x5C, 0x01, 0xA0, 0xB1, 0x4C, 0x10, 0xEB, 0xA9, 0x4C,
                                 0x91, 0x52, 0x3A, 0x4B, 0x84, 0x8A, 0x55, 0x5E, 0x85, 0x1A, 0x8E,
                                 0x70, 0x77, 0x3B, 0x25, 0x2D, 0x63, 0x4E, 0xE4, 0x19, 0xCF, 0x76,
                                 0x07, 0xDE, 0xC4, 0x7B, 0x77, 0x41, 0x3F, 0x46, 0x5E, 0x4F, 0xC9,
                                 0xEE, 0x57, 0xB6, 0x65, 0xFD, 0xFF, 0xD4, 0xC7, 0x0A, 0x10, 0x26,
                                 0xA7, 0x3F, 0xF0};

    RSA *rsa = RSA_new();

    rsa->n = BN_bin2bn(pubKey, 256, NULL);
    rsa->e = BN_bin2bn(exp, 8, NULL);

    int errCode = RSA_verify(RSA_PKCS1_PADDING, MESSAGE, MESSAGE_LEN, signature, 256, rsa);

    if (errCode == 1)
    {
        std::cout << "Message signature was verified successfully" << std::endl;
    }
    else
    {
        std::cout << "Message signature is incorrect" << std::endl;
    }

    return 0;
}

That is the right signature for the message, but I keep getting a wrong signature result. What am I doing wrong?

Answer 1

Thanks to jww 's comment about the exponent being 72058693549555712, I realized I provided the numbers in little-endian form, where the BN_bin2bn function expect a buffer in big-endian form. I managed to find here a function called BN_bin2lbn that should be perfect for that case, as it expects a buffer in little-endian form. Unfortunately this function doesn't seem to exist in the bn.h file on my computer (which is where BN_bin2bn is declared).

Eventually I managed to overcome this by turning my numbers into big-endian form, using:

std::reverse(pubKey, pubKey + 256);
std::reverse(exp, exp + 8);

Now everything works fine.