服务器和客户端上具有相同值的不同SHA1散列

Question

On the client I'm using Rusha, which I've put into a wrapper:

function cSHA1(m){
  return (new Rusha).digest(m);
}

On the server I'm using Node's native crypto module,

function sSHA1(m){
  var h = crypto.createHash('sha1');
  h.update(m);
  return h.digest('hex');
}

Let's try it:

cSHA1('foo')
"0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"

sSHA1('foo')
'0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'

cSHA1('bar')
"62cdb7020ff920e5aa642c3d4066950dd1f01f4d"

sSHA1('bar')
'62cdb7020ff920e5aa642c3d4066950dd1f01f4d'

So far, so good.

Now let's throw them a curveball...

cSHA1(String.fromCharCode(10047))
"5bab61eb53176449e25c2c82f172b82cb13ffb9d"

sSHA1(String.fromCharCode(10047))
'5bab61eb53176449e25c2c82f172b82cb13ffb9d'

Ok, fine.

I have a string, and it shouldn't be important how I got it, and it's a long story, anyway, but:

s.split('').map(function(c){
    return c.charCodeAt();
})

yields the exact same result in both places:

[58, 34, 10047, 32, 79]

Now, let's hash it:

s
":"✿ O"

cSHA1(s)
"a199372c8471f35d14955d6abfae4ab12cacf4fb"

s
':"? O'
sSHA1(s)
'fc67b1e4ceb3e57e5d9f601ef4ef10c347eb62e6'

This has caused me a fair bit of grief; what the hell?

Answer 1

I have run into the same problem with the German Umlaut character when comparing SHA1 hashes of PHPs sha1 and Rusha.

The reason is simple: some stoned fool decided Javascript strings are UTF16 - and PHP doesn't give a sh*t about encoding, it just takes what is there. So, if you supply PHP a json_decode("\ä"), it will turn this into a 2-byte string 0xc3 0xa4 (UTF8).

JS instead will make a single UTF16 byte out of this (0xE4) - and Rusha's manual explicitly says all codepoints must be below 256 .

To help yourself, use the UTF18-to-8 library at http://www.onicos.com/staff/iz/amuse/javascript/expert/utf.txt like sha.digest(utf16to8("\ä")) . This will feed rusha correct codepoints.