[英]Is it possible to upload stream on amazon s3 from browser?
I want to capture webcam video stream, and directly stream it to S3 storage. 我想捕获网络摄像头视频流,并直接将其流式传输到S3存储。
I've learned that you can upload via stream to s3: https://aws.amazon.com/blogs/aws/amazon-s3-multipart-upload/ 我了解到你可以通过流上传到s3: https : //aws.amazon.com/blogs/aws/amazon-s3-multipart-upload/
I've learned that you can upload via browser: http://docs.aws.amazon.com/AmazonS3/latest/dev/HTTPPOSTExamples.html#HTTPPOSTExamplesFileUpload 我了解到你可以通过浏览器上传: http : //docs.aws.amazon.com/AmazonS3/latest/dev/HTTPPOSTExamples.html#HTTPPOSTExamplesFileUpload
But Im still lost on how to actually do it. 但我仍然失去了如何实际做到这一点。
I need an example of someone uploadin getusermediastream to S3 like above. 我需要一个上传getinmedia流到S3的例子。
Buffer, Binary data, multipart upload, stream... this is all beyond my knowledge. 缓冲区,二进制数据,分段上传,流...这是我所不知道的。 Stuff I wish I knew, but don't even now where to learn. 我希望我知道的东西,但现在甚至不在哪里学习。
Currently, you cannot simply pass the media stream to any S3 method to do the multipart upload automatically. 目前,您不能简单地将媒体流传递给任何S3方法以自动执行分段上传。
But still, there is an event called dataavailable
which produces the chunks of video each given time interval. 但是,仍有一个名为dataavailable
的事件,它在每个给定的时间间隔内产生视频块。 So we can subscribe to dataavailable
and do the S3 Multipart Upload manually. 因此,我们可以订阅dataavailable
并手动执行S3 Multipart Upload。
This approach brings some complications: say chunks of video are generated each 1 second, but we don't know how long does it take to upload the chunk to S3. 这种方法带来了一些复杂性:说每1秒生成一大块视频,但我们不知道将块上传到S3需要多长时间。 Eg the upload can take 3 times longer due to the connection speed. 例如,由于连接速度,上传可能需要3倍的时间。 So we can get stuck trying to make multiple PUT requests at the same time. 因此,我们可能会在尝试同时发出多个PUT请求时陷入困境。
The potential solution would be to upload the chunks one by one and don't start uploading the next chunk until the prev. 可能的解决方案是逐个上传块,并且不要开始上传下一个块直到prev。 one is uploaded. 一个上传。 Here is a snippet of how this can be handled using Rx.js and AWS SDK. 以下是使用Rx.js和AWS SDK如何处理此问题的片段。 Please see my comments. 请看我的评论。
// Configure the AWS. In this case for the simplicity I'm using access key and secret. AWS.config.update({ credentials: { accessKeyId: "YOUR_ACCESS_KEY", secretAccessKey: "YOUR_SECRET_KEY", region: "us-east-1" } }); const s3 = new AWS.S3(); const BUCKET_NAME = "video-uploads-123"; let videoStream; // We want to see what camera is recording so attach the stream to video element. navigator.mediaDevices .getUserMedia({ audio: true, video: { width: 1280, height: 720 } }) .then(stream => { console.log("Successfully received user media."); const $mirrorVideo = document.querySelector("video#mirror"); $mirrorVideo.srcObject = stream; // Saving the stream to create the MediaRecorder later. videoStream = stream; }) .catch(error => console.error("navigator.getUserMedia error: ", error)); let mediaRecorder; const $startButton = document.querySelector("button#start"); $startButton.onclick = () => { // Getting the MediaRecorder instance. // I took the snippet from here: https://github.com/webrtc/samples/blob/gh-pages/src/content/getusermedia/record/js/main.js let options = { mimeType: "video/webm;codecs=vp9" }; if (!MediaRecorder.isTypeSupported(options.mimeType)) { console.log(options.mimeType + " is not Supported"); options = { mimeType: "video/webm;codecs=vp8" }; if (!MediaRecorder.isTypeSupported(options.mimeType)) { console.log(options.mimeType + " is not Supported"); options = { mimeType: "video/webm" }; if (!MediaRecorder.isTypeSupported(options.mimeType)) { console.log(options.mimeType + " is not Supported"); options = { mimeType: "" }; } } } try { mediaRecorder = new MediaRecorder(videoStream, options); } catch (e) { console.error("Exception while creating MediaRecorder: " + e); return; } //Generate the file name to upload. For the simplicity we're going to use the current date. const s3Key = `video-file-${new Date().toISOString()}.webm`; const params = { Bucket: BUCKET_NAME, Key: s3Key }; let uploadId; // We are going to handle everything as a chain of Observable operators. Rx.Observable // First create the multipart upload and wait until it's created. .fromPromise(s3.createMultipartUpload(params).promise()) .switchMap(data => { // Save the uploadId as we'll need it to complete the multipart upload. uploadId = data.UploadId; mediaRecorder.start(15000); // Then track all 'dataavailable' events. Each event brings a blob (binary data) with a part of video. return Rx.Observable.fromEvent(mediaRecorder, "dataavailable"); }) // Track the dataavailable event until the 'stop' event is fired. // MediaRecorder emits the "stop" when it was stopped AND have emitted all "dataavailable" events. // So we are not losing data. See the docs here: https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/stop .takeUntil(Rx.Observable.fromEvent(mediaRecorder, "stop")) .map((event, index) => { // Show how much binary data we have recorded. const $bytesRecorded = document.querySelector("span#bytesRecorded"); $bytesRecorded.textContent = parseInt($bytesRecorded.textContent) + event.data.size; // Use frameworks in prod. This is just an example. // Take the blob and it's number and pass down. return { blob: event.data, partNumber: index + 1 }; }) // This operator means the following: when you receive a blob - start uploading it. // Don't accept any other uploads until you finish uploading: http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-concatMap .concatMap(({ blob, partNumber }) => { return ( s3 .uploadPart({ Body: blob, Bucket: BUCKET_NAME, Key: s3Key, PartNumber: partNumber, UploadId: uploadId, ContentLength: blob.size }) .promise() // Save the ETag as we'll need it to complete the multipart upload .then(({ ETag }) => { // How how much bytes we have uploaded. const $bytesUploaded = document.querySelector("span#bytesUploaded"); $bytesUploaded.textContent = parseInt($bytesUploaded.textContent) + blob.size; return { ETag, PartNumber: partNumber }; }) ); }) // Wait until all uploads are completed, then convert the results into an array. .toArray() // Call the complete multipart upload and pass the part numbers and ETags to it. .switchMap(parts => { return s3 .completeMultipartUpload({ Bucket: BUCKET_NAME, Key: s3Key, UploadId: uploadId, MultipartUpload: { Parts: parts } }) .promise(); }) .subscribe( ({ Location }) => { // completeMultipartUpload returns the location, so show it. const $location = document.querySelector("span#location"); $location.textContent = Location; console.log("Uploaded successfully."); }, err => { console.error(err); if (uploadId) { // Aborting the Multipart Upload in case of any failure. // Not to get charged because of keeping it pending. s3 .abortMultipartUpload({ Bucket: BUCKET_NAME, UploadId: uploadId, Key: s3Key }) .promise() .then(() => console.log("Multipart upload aborted")) .catch(e => console.error(e)); } } ); }; const $stopButton = document.querySelector("button#stop"); $stopButton.onclick = () => { // After we call .stop() MediaRecorder is going to emit all the data it has via 'dataavailable'. // And then finish our stream by emitting 'stop' event. mediaRecorder.stop(); };
button { margin: 0 3px 10px 0; padding-left: 2px; padding-right: 2px; width: 99px; } button:last-of-type { margin: 0; } p.borderBelow { margin: 0 0 20px 0; padding: 0 0 20px 0; } video { height: 232px; margin: 0 12px 20px 0; vertical-align: top; width: calc(20em - 10px); } video:last-of-type { margin: 0 0 20px 0; }
<div id="container"> <video id="mirror" autoplay muted></video> <div> <button id="start">Start Streaming</button> <button id="stop">Stop Streaming</button> </div> <div> <span>Recorded: <span id="bytesRecorded">0</span> bytes</span>; <span>Uploaded: <span id="bytesUploaded">0</span> bytes</span> </div> <div> <span id="location"></span> </div> </div> <!-- include adapter for srcObject shim --> <script src="https://webrtc.github.io/adapter/adapter-latest.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/aws-sdk/2.175.0/aws-sdk.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.6/Rx.js"></script>
Caveats: 注意事项:
s3:PutObject
permission. 在实例化SDK时,请确保存在具有s3:PutObject
权限的策略。 <?xml version="1.0" encoding="UTF-8"?> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <CORSRule> <AllowedOrigin>*</AllowedOrigin> <AllowedMethod>GET</AllowedMethod> <AllowedMethod>POST</AllowedMethod> <AllowedMethod>PUT</AllowedMethod> <ExposeHeader>ETag</ExposeHeader> <AllowedHeader>*</AllowedHeader> </CORSRule> </CORSConfiguration>
Limitations: 限制:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.