[英]Nodejs error encoding when get external site's content
I used get
method of request
module to get content of external site. 我使用
request
模块的get
方法来获取外部站点的内容。 If encoding of external site is utf-8, it is ok, but it has display error with other encodings such as shift-jis 如果外部站点的编码为utf-8,则可以,但是与其他编码(如shift-jis)一起显示错误
function getExternalUrl(request, response, url){
mod_request.get(url, function (err, res, body) {
//mod_request.get({uri: url, encoding: 'binary'}, function (err, res, body) {
if (err){
console.log("\terr=" + err);
}else{
var result = res.body;
// Process res.body
response.write(result);
}
response.end();
});
}
How can I get content of external site with correct encoding? 如何获得具有正确编码的外部站点的内容?
I found the way to do: 我找到了方法:
Get with binary
encoding 使用
binary
编码获取
var mod_request = require('request'); var mod_request = require('request');
mod_request.get({ uri: url, encoding: 'binary', headers: headers }, function(err, res, body) {}); mod_request.get({uri:url,编码:'binary',headers:headers},function(err,res,body){});
Create a Buffer
with binary
format 创建
binary
格式的Buffer
var contentBuffer = new Buffer(res.body, 'binary'); var contentBuffer = new Buffer(res.body,'binary');
Get real encoding of page by detect-character-encoding
npm 通过
detect-character-encoding
npm获取页面的真实编码
var mod_detect_character_encoding = require('detect-character-encoding'); var mod_detect_character_encoding = require('detect-character-encoding');
var charsetMatch = mod_detect_character_encoding(contentBuffer); var charsetMatch = mod_detect_character_encoding(contentBuffer);
Convert page to utf-8
by iconv
npm 通过
iconv
npm将页面转换为utf-8
var mod_iconv = require('iconv').Iconv; var mod_iconv = require('iconv')。Iconv;
var iconv = new mod_iconv(charsetMatch.encoding, 'utf-8'); var iconv = new mod_iconv(charsetMatch.encoding,'utf-8');
var result = iconv.convert(contentBuffer).toString(); var result = iconv.convert(contentBuffer).toString();
P/S: This way is only applied for text file (html, css, js). Please do not apply for image file or others which is not text
P / S:
This way is only applied for text file (html, css, js). Please do not apply for image file or others which is not text
This way is only applied for text file (html, css, js). Please do not apply for image file or others which is not text
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.