Avoid newline when using Crypto++ Base64 encoder

Question

Im trying to generate the SHA512 of a base64 encoded string with Crypto++.

My sample input is: test123

Base64: dGVzdDEyMw==

SHA512 of B64 (expected):

f78fa0aa79abd53b8181c5d21bdeb882bf45cd462a6e6e1b5043417de1800626
ed2a51b1a56626e9b9558da66a2f609d31db76bd88e80afbb7b03cda518b207d

SHA512 of B64 (not expected): 9f012fff26c89f2650f7446a37e80ba6466d69ffc77bb9ffc8c09ab779b24a23bb6a2f3c28512668ebca8628303ab5a31067d930cd1af60c745a2c34e5b4b1d2

SHA512 calculation:

byte *digest = new byte[CryptoPP::SHA512::DIGESTSIZE];
std::string encoded;
std::string test("test123");

CryptoPP::StringSource ss((byte*)test.data(), test.size(), true, new CryptoPP::Base64Encoder(new CryptoPP::StringSink(encoded))); // StringSource

// Calculate hash
CryptoPP::SHA512().CalculateDigest(digest, (byte*)encoded.data(), encoded.size());

If i leave out base64 and calculate the SHA512 directly, i get the correct hash. Therefore the calculation can't be completely wrong.

But why doesn't it work with base64?

Answer 1

SHA512 of B64 (correct):

 f78fa0aa79abd53b8181c5d21bdeb882bf45cd462a6e6e1b5043417de1800626 ed2a51b1a56626e9b9558da66a2f609d31db76bd88e80afbb7b03cda518b207d 

It sounds like whatever is calculating this hash is producing unexpected results. Its probably due to a newline, missing padding, etc.

I can reproduce it with the following. It appears to be a newline issue.

$ echo 'dGVzdDEyMw' | sha512sum
9c00af94b3dc300fab0fd1fdad7e9eeb20bb0bdff6e6c75d9072e241976b0cc8
56f2a1d355c35f29c3d354895565f971721f58cbb20f0608f57a882b0afb412c

$ echo -n 'dGVzdDEyMw' | sha512sum
c20144e3136f57d5aae2374aa48759911364bb44167fe642cc8b4da140396584
04ce9e2f3dfc9bd69d69cfbb449384e6ea5377c39a07fdb2c2920d78a2a56a80

$ echo  'dGVzdDEyMw==' | sha512sum
9f012fff26c89f2650f7446a37e80ba6466d69ffc77bb9ffc8c09ab779b24a23
bb6a2f3c28512668ebca8628303ab5a31067d930cd1af60c745a2c34e5b4b1d2

$ echo -n 'dGVzdDEyMw==' | sha512sum
f78fa0aa79abd53b8181c5d21bdeb882bf45cd462a6e6e1b5043417de1800626
ed2a51b1a56626e9b9558da66a2f609d31db76bd88e80afbb7b03cda518b207d

Here is the constructor for Base64Encoder . You can find the docs at either the manual or the wiki .

Base64Encoder(BufferedTransformation *attachment = NULL,
              bool insertLineBreaks = true,
              int maxLineLength = 72)

You should use insertLineBreaks = false . Maybe something like:

StringSource ss((byte*)test.data(), test.size(), true,
    new Base64Encoder(new StringSink(encoded), false /* Newline */));

Since you are using a Pipeline , you can do it in one shot with the following. I unrolled all the new 's to help with visualization as data flows from the source to the sink.

SHA512 hash;
StringSource ss(test /* std::string */, true,
    new Base64Encoder(
        new HashFilter(hash,
            new StringSink(encoded)
        ),
    false /* Newline */)
);