用Erlang re解析\\“ – \\”

Question

I've parsed an HTML page with mochiweb_html and want to parse the following text fragment

0 – 1

Basically I want to split the string on the spaces and dash character and extract the numbers in the first characters.

Now the string above is represented as the following Erlang list

[48,32,226,128,147,32,49]

I'm trying to split it using the following regex:

{ok, P}=re:compile("\\xD2\\x80\\x93"), %% characters 226, 128, 147

re:split([48,32,226,128,147,32,49], P, [{return, list}])

But this doesn't work; it seems the \\xD2 character is the problem [if I remove it from the regex, the split occurs]

Could someone possibly explain

• what I'm doing wrong here ?
• why the '–' character seemingly requires three integers for representation [226, 128, 147]

Thanks.

Answer 1

226,128,147 is E2,80,93 in hex.

> {ok, P} = re:compile("\xE2\x80\x93").
...
> re:split([48,32,226,128,147,32,49], P, [{return, list}]).  
["0 "," 1"]

Answer 2

As to your second question, about why a dash takes 3 bytes to encode, it's because the dash in your input isn't an ASCII hyphen (hex 2D), but is a Unicode en-dash (hex 2013). Your code is recieving this in UTF-8 encoding , rather than the more obvious UCS-2 encoding . Hex 2013 comes out to hex E28093 in UTF-8 encoding.

If your next question is "why UTF-8", it's because it's far easier to retrofit an old system using 8-bit characters and null-terminated C style strings to use Unicode via UTF-8 than to widen everything to UCS-2 or UCS-4. UTF-8 remains compatible with ASCII and C strings, so the conversion can be done piecemeal over the course of years, or decades if need be. Wide characters require a "Big Bang" one-time conversion effort, where everything has to move to the new system at once. UTF-8 is therefore far more popular on systems with legacies dating back to before the early 90s, when Unicode was created.