Swift HMAC doesn't match NodeJS HMAC, but only sometimes!

Question

I have discovered a HUGE issue in my code, and I have literally no idea what is causing this.

SO, when I send requests to my server I hash a string thats in the request. This is sometimes user input.

My app is multi language so I have to support all "ä" chars etc.

So with the normal english letters/chars numbers etc, this hashing method works like a dream. BUT when the string being hashed and compared contains a "ä" or a "ö" (Not specifically those, it literally might be that any char not in the Base64 set will cause this) the hash doesn't match!

This is an absolute and complete disaster, and I have not noticed it this far. I have tried basically everything I know to try to fix this, and googling, and I am out of luck so far.

I generate the hash in Swift inputting the string and secretToken into this function and saving the output as a HTTP header:

func hmac(string: String, key: String) -> String {

    var digest = [UInt8](repeating: 0, count: Int(CC_SHA256_DIGEST_LENGTH))

    CCHmac(CCHmacAlgorithm(kCCHmacAlgSHA256), key, key.count, string, string.count, &digest)

    let data = Data(digest)

    return data.map { String(format: "%02hhx", $0) }.joined()

}

How I compare the hash in NodeJS:

if (hashInTheRequest === crypto.createHmac('sha256', secretToken).update(stringToHash).digest('hex')) {
    //Good to go
}

Thanks in advance!

Answer 1

This could be due to a composition issue. You mentioned non-latin characters, but didn't specify any concrete examples, where you had problems.

What is composition?

Unicode aims to be able to represent any character used by humanity. However, many characters are similar, such as u , ü , û and ū . The original idea was to just assign a code point to every possible combination. As one might imagine, this is not the most effective way to store things. Instead, the "base" character is used, and then a combining character is added to it.

Let's look at an example: ü

ü can be represented as U+00FC , also known as LATIN SMALL LETTER U WITH DIAERESIS .

ü can also be represented as U+0075 ( u ), followed by U+0308 ( ◌̈ ), also known as LATIN SMALL LETTER U , followed by COMBINING DIARESIS .

Why is this problematic?

Because hash functions don't know what a string is. All they care about is bytes. As such, a string has to be decoded to a string of bytes. As was shown above, there are multiple different ways to decode a string, which means that two different systems can decode the same logical string to different bytes, thus resulting in different hashes.

How can I fix this?

You have to explicitly define how the string will be decoded on both platforms, to ensure that both decode the strings in the exact same manner.