[英]Piping stream from busboy to request post
I have multipart/form-data
that I am posting to an express endpoint /data/upload
, form markup below: 我有多multipart/form-data
,我发布到快速端点/data/upload
,下面的表单标记:
form(enctype="multipart/form-data", action="/data/upload", method="post")
input(type="file", name="data")
I'm using busboy
to read the file stream, which is working fine. 我正在使用busboy
来读取文件流,这工作正常。 From there, I want to send the stream again as multipart/form-data
to a second Java backend, using the request
npm module. 从那里,我想使用request
npm模块将流再次作为multipart/form-data
到第二个Java后端。 JS client/Java server code below: JS客户端/ Java服务器代码如下:
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
var reqBody = {
url: server.baseURL + 'api/data',
headers: {
'Connection': 'keep-alive',
'Content-Type': 'multipart/form-data'
},
formData: {
file: fileStream
}
};
request.post(reqBody, function (err, r, body) {
// Do rendering stuff, handle callback
});
});
Java endpoint (api/data) Java端点(api / data)
@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
public void addData(FormDataMultiPart formDataMultiPart) {
// Handle multipart data here
}
I don't think I'm sending the file correctly as multipart/form-data
here... but I'm having a hard time figuring out how to essentially pipe the stream from busboy
directly to request
without reading/writing from a temp file on the client-side. 我不认为我正在将文件正确发送为multipart/form-data
...但是我很难弄清楚如何从busboy
直接将流传输到request
而无需从temp中读取/写入客户端的文件。 Any ideas? 有任何想法吗?
Java stack trace: Java堆栈跟踪:
Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server has received a request on thread qtp1631904921-24
3 > POST http://localhost:8080/api/data
3 > Connection: keep-alive
3 > Content-Length: 199
3 > Content-Type: multipart/form-data; boundary=--------------------------331473417509479560313628
3 > Host: localhost:8080
Apr 27, 2016 5:07:12 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server responded with a response on thread qtp1631904921-24
3 < 400
17:07:13.003 [qtp1631904921-24] WARN org.eclipse.jetty.http.HttpParser parseNext - bad HTTP parsed: 400 No URI for HttpChannelOverHttp@425137da{r=1,c=false,a=IDLE,uri=null}
Rahat's recommended change : 拉哈特建议的改变 :
31 var reqBody = {
32 url: server.baseURL + 'data',
33 headers: {
34 'Connection': 'keep-alive',
35 'Content-Type': 'multipart/form-data'
36 }
37 };
38
39 req.pipe(req.busboy.pipe(request.post(reqBody)));
Threw error: 扔错误:
Error: Cannot pipe. Not readable.
at Busboy.Writable.pipe (_stream_writable.js:154:22)
The problem here is that you need to provide 'Content-Length' for the multipart upload manually, because request
(and underlying form-data
) can't figure it out by themselves. 这里的问题是您需要手动为分段上传提供“内容长度”,因为request
(和底层form-data
)无法自行解决。 So request sends invalid Content-Length: 199 (the same for any incoming file size), which breaks the java multipart parser. 因此请求发送无效的Content-Length:199(对于任何传入的文件大小都相同),这会破坏java多部分解析器。
There are multiple workarounds: 有多种解决方法:
1) Use incoming request 'Content-Length' 1)使用传入请求'Content-Length'
request.post({
url: server.baseURL + 'api/data',
formData: {
file: {
value: fileStream,
options: {
knownLength: req.headers['content-length']
}
}
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
This will produce a bit incorrect request though, because incoming length includes other upload fields and boundaries, but busboy was able to parse it w/o any complaints 这会产生一些不正确的请求,因为传入长度包括其他上传字段和边界,但busboy能够解析它没有任何投诉
2) Wait until file is completely buffered by the node app then send it to java 2)等待文件被节点应用程序完全缓冲,然后将其发送到java
var concat = require('concat-stream')
req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) {
fileStream.pipe(concat(function (fileBuffer) {
request.post({
url: server.baseURL + 'api/data',
formData: {
file: fileBuffer
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
}))
})
This will increase app memory consumption, so you needed to be careful and consider using busboy limits 这会增加应用程序内存消耗,因此您需要小心并考虑使用busboy限制
3) Buffer file to disk before uploading (just for the reference) 3)上传前缓冲文件到磁盘(仅供参考)
If possible, send a custom header with exactly size file (bytes). 如果可能,发送一个具有确切大小文件(字节)的自定义标头。 The header always can be read before handle payload stream. 始终可以在处理有效负载流之前读取标头。 Use this instead of content-length header of previous answer, because that sometimes doesn't works (with small files, i guess, but i cannot ensure that works with large files). 使用此代替以前答案的内容长度标题,因为这有时不起作用(我想,小文件,但我无法确保适用于大文件)。
For the answer of Afanasii Kurakin 对于Afanasii Kurakin的答案
request.post({
url: server.baseURL + 'api/data',
formData: {
file: {
value: fileStream,
options: {
knownLength: req.headers['content-length']
}
}
}
}, function (err, r, body) {
// Do rendering stuff, handle callback
})
You should change from req.headers['content-length']
to the real file size, normally the content-length from header is bigger than the file size. 您应该从req.headers['content-length']
更改为实际文件大小,通常来自标头的内容长度大于文件大小。 I got pain because of the content-length and after using the file size, everything worked perfectly. 我因内容长度而感到痛苦,使用文件大小后,一切都很完美。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.