[英]knox S3 upload corrupted or truncated file
This is a brain teaser question where I actually know the answer. 这是我真正知道答案的脑筋急转弯问题。 I'm throwing a bounty on it because it represents a valuable Node programming safety tip (and that's the first hint).
我对此悬赏,因为它代表了宝贵的Node编程安全性提示(这是第一个提示)。
I'm using 我正在使用
var knox = require('knox');
var s3 = knox.createClient({
key: ...,
secret: ...,
bucket: ...
});
// The bug is below:
var stringVal = JSON.stringify(<2d javascript array from a large spreadsheet>)
var req = s3.put(path + filename, {
'Content-Length': stringVal.length,
'Content-Type': 'application/json'
});
req.end(stringVal);
The resulting upload is either truncated or otherwise corrupted. 结果上传被截断或损坏。 We have
stringVal.length === 322889
, and the resulting S3 item size matches that. 我们有
stringVal.length === 322889
,并且生成的S3项目大小与此匹配。 But downloading and reloading the file results in a string which has length 322140
. 但是下载并重新加载文件会导致字符串长度为
322140
。 No errors show up along the way until trying to JSON.parse the string which (predictably) results in a syntax error. 在尝试进行JSON解析之前,不会出现任何错误。解析该字符串(可预测)会导致语法错误。
What's up? 这是怎么回事?
From the source of the knox
-module ( https://github.com/LearnBoost/knox/blob/master/lib/client.js ) you can learn that it uses standard http
-requests. 从
knox
-module( https://github.com/LearnBoost/knox/blob/master/lib/client.js )的源代码中,您可以了解到它使用了标准的http
-requests。
req.write
and req.end
converts strings from 'utf8' by default ( http://nodejs.org/api/http.html#http_request_end_data_encoding ). req.write
和req.end
默认情况下会从'utf8'转换字符串( http://nodejs.org/api/http.html#http_request_end_data_encoding )。
So what really happens is that you by accident cut off the end of the string by setting the string-length instead of the number of bytes in the 'Content-Length'-field. 因此,实际上发生的是,您无意中通过设置字符串长度而不是“ Content-Length”字段中的字节数来切断字符串的末尾。 The server throws away everything longer than that;
服务器将所有东西扔掉的时间更长; so when you parse the string you get an error.
因此,当您解析字符串时,会出现错误。
Quickest fix would be: 最快的解决办法是:
'Content-Length': new Buffer(stringVal).length,
Or even faster: just remove the 'Content-Length'-line. 甚至更快:只需删除“ Content-Length”行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.