Why does php md5() always different from python's hash.md5() if using chinese character?

Question

here is my php code:

$str = '你好';
$input_encoding = mb_detect_encoding($str, array('ASCII','GB2312','GBK','UTF-8'), true);
echo sprintf('input encoding:%s', $input_encoding);

$str_gb = iconv($input_encoding, 'GBK', true);
echo sprintf("utf8 encoding:%s\n", $str);
echo sprintf("gb encoding md5:%s\n", md5($str_gb));
echo sprintf("utf8 encoding md5:%s\n", md5($str));

here is my python code：

#!/usr/bin/env python
#coding:utf-8


import urllib
import hashlib

str_u = u'你好'
str_gb = str_u.encode('gbk')
str_u8 = str_u.encode('utf-8')

m = hashlib.md5()
m.update(str_gb)
str_gb_md5 = m.hexdigest()
m.update(str_u8)
str_u8_md5 = m.hexdigest()

print 'gb md5:%s' % str_gb_md5
print 'utf-8 md5:%s' % str_u8_md5

PHP code result is:

input encoding:CP936
utf8 encoding:你好
gb encoding md5:c4ca4238a0b923820dcc509a6f75849b
utf8 encoding md5:7eca689f0d3389d9dea66ae112e5cfd7

python code result is:

gb md5:b94ae3c6d892b29cf48d9bea819b27b9
utf-8 md5:a8a343223373c7d78c3fb8bad2d786c3

And here is my programming environment: PHP 5.5.4 (cli)&Python 2.6.8

THX!

Answer 1

There are mistakes in both your php and python code. The gbk md5 in you php code and utf8 md5 in you python code are wrong.

Python part:

You misunderstand the usage of Python hashlib's hash.update function.

hash.update(arg)

Update the hash object with the string arg. Repeated calls are equivalent to a single call with the concatenation of all the arguments: m.update(a); m.update(b) is equivalent to m.update(a+b) .

Fix:

print  hashlib.md5(str_u8).hexdigest()

7eca689f0d3389d9dea66ae112e5cfd7

PHP part:

You forget to pass $str to the iconv function, instead you passed a true value( which is coverts to 1).

Fix:

$str = '你好';
$str_gb = iconv('UTF-8', 'GBK', $str);
echo sprintf("gb encoding md5:%s\n", md5($str_gb));

output:

gb encoding md5:b94ae3c6d892b29cf48d9bea819b27b9

iconv definition :

string iconv ( string $in_charset , string $out_charset , string $str )