[英]How do you upload a zip file and unzip to s3?
I am working on an application where I have to upload a zip file. 我正在处理必须上传zip文件的应用程序。 The zip file is basically a static website so it has many files and a couple subdirectories.
该zip文件基本上是一个静态网站,因此它具有许多文件和几个子目录。 I have been playing with the rubyzip gem for a while now and can not figure out how to simply extract the files from it.
我已经使用rubyzip gem有一段时间了,无法弄清楚如何从其中提取文件。 Any pointers on where I can read up on some examples?
关于一些示例,我可以从哪里获得任何指导? I am sure someone has ran in to this problem before.
我确信以前有人遇到过这个问题。 the documentation for rubyzip is not very good so I am hoping someone can give me some pointers.
rubyzip的文档不是很好,所以我希望有人可以给我一些指导。
Here you go, one super magical multithreaded zip-to-S3 uploader which I haven't tested at all - go nuts! 在这里,您可以找到一个我从未测试过的超级神奇的多线程zip至S3上传器-疯了! Looks like I'm three years too late though.
看来我已经晚了三年。
class S3ZipUploader
require 'thread'
require 'thwait'
require 'find'
attr_reader *%i{ bucket s3 zip failed_uploads }
def initialize(zipfilepath, mys3creds)
# next 4 lines are important
@s3 = AWS::S3.new(access_key_id: mys3creds[Rails.env]['aws_access_key'],
secret_access_key: mys3creds[Rails.env]['aws_secret_access_key'],
region: 'us-west-2')
@bucket = @s3.buckets[ mys3creds[Rails.env]['bucket'] ]
@failed_uploads = []
@zip = Zip::File.open(zipfilepath)
end
def upload_zip_contents
rootpath = "mypath/"
desired_threads = 10
total_entries = zip.entries.count
slice_size = (total_entries / desired_threats).ceil
threads = []
zip.entries.each_slice(slice_size) do |e_arr|
threads << Thread.new do |et|
e_arr.each do |e|
result = upload_to_s3(rootpath + e.name, e.get_input_stream.read)
if !result
@failed_uploads << {name: e.name, entry: e, error: err}
end
end
end
end
ThreadsWait.all_waits(*threads)
end
def upload_file_to_s3(filedata,path, rewrite_basepath)
retries = 0
success = false
while !success && retries < 3
success = begin
obj = bucket.objects[path]
obj.write(Pathname.new(outputhtml))
obj.acl = :public_read
success = true
rescue
retries += 1
success = false
end
end
return success
end
end
uploader = S3ZipUploader.new("/path/to/myzip.zip", MYS3CREDS)
uploader.upload_zip_contents
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.