将字符串转换为缓冲区并返回 javascript 时出现无关字符

Question

In node v16.14.2,

> String.fromCharCode.apply(null, Buffer.from('°'))
'°'

This works fine though:

> Buffer.from('°').toString('utf-8')
'°'

I want to know why the first scenario adds an extra character in the output.

Browser example:

 console.log(String.fromCharCode.apply(null, ethereumjs.Buffer.Buffer.from('°'))) /// '°' console.log(ethereumjs.Buffer.Buffer.from('°').toString('utf-8')) // '°' console.log(new Uint8Array(ethereumjs.Buffer.Buffer.from('°'))) // [ 194, 176 ] console.log(new Uint8Array(ethereumjs.Buffer.Buffer.from(String.fromCharCode(176)))) // [ 194, 176 ]

 <script src="https://cdn.jsdelivr.net/gh/ethereumjs/browser-builds/dist/ethereumjs-tx/ethereumjs-tx-1.3.3.min.js"></script>

EDIT : Another funny thing:

> new Uint8Array(Buffer.from('°'))
Uint8Array(2) [ 194, 176 ]
> new Uint8Array(Buffer.from(String.fromCharCode(176)))
Uint8Array(2) [ 194, 176 ]

Answer 1

The default string encoding of Buffer.from is UTF-8 which encodes characters in the Extended ASCII range using two bytes. (It can't just use one byte because the highest order bit of that byte is used to signal multibyte encoding.)

UTF-8 two-byte encoding has the form:

110xxxxx 10xxxxxx

which gives 11 bits to represent a character code.

The ASCII code of '°' is 176

> '°'.charCodeAt(0)
176

which in binary is:

> (176).toString(2)
'10110000'

So replacing the x with this binary value we have:

110xxxxx 10xxxxxx
110xxx10 10110000
      ^^   ^^^^^^

which means the UTF-8 two-byte encoding of '°' is

11000010 10110000

or

194 176

as expected:

> [...Buffer.from('°')]
[ 194, 176 ]

> String.fromCharCode(194)
'Â'

> String.fromCharCode(176)
'°'