MIDIutil 通过 Flask 将 MIDI 文件写入 boto3 API 服务器

Question

在连接到 s3 服务器的 Flask 应用程序中尝试编写 MIDIUtil 文件时遇到了困难。

在本地实例中，这并不费力：

LOCAL_UPLOAD_FOLDER = './_static/uploads/MIDI_files/'
file_name = "NAME.mid"
file_path = f'{LOCAL_UPLOAD_FOLDER}{file_name}'
MyMIDI = MIDIFile(1)

with open(file_path, "wb") as output_file:
     MyMIDI.writeFile(output_file)

但是，我不确定如何将其应用于 s3 资源，这是我的实例化...

def get_upload_folder(UPLOAD_FOLDER=None, UPLOAD_FOLDER_KEY=None,
                      client_resource=None, client=None):
    """ Determines How to Upload / Send File for Download """
    # Flask Cloud Settings - Upload Folder
    if os.getenv('CONTEXT') == 'CLOUD':

        # Client Side
        UPLOAD_FOLDER_TYPE = 'CLOUD'
        session = boto3.session.Session()
        client = session.client(
            's3', endpoint_url=os.getenv('ENDPOINT_URL'),
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name=os.getenv('REGION_NAME'), aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

        # Resource Side
        client_resource = boto3.resource(
            's3', endpoint_url='https://nyc3.digitaloceanspaces.com',
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name='nyc3', aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

    UPLOAD_FOLDER, UPLOAD_FOLDER_KEY = 'MY_BUCKET', 'uploads/MIDI_files/'

   return UPLOAD_FOLDER_TYPE, UPLOAD_FOLDER, UPLOAD_FOLDER_KEY, client_resource, client

到目前为止，我已经尝试过：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.download_fileobj(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

以及大量其他.put_object与 client 和 client_resource boto3 对象的组合......

我在想我的问题在于：

• MIDIUtil.Midifile 的writeFile(filehandler) MIDIUtil.Midifile )

也许这个 function 正在关闭 MIDI 二进制文件put_object DATA，然后我才能将对象放入 s3 BODY=中？ 也许我需要通过 Bytes(IO) / stream object.. 来解析二进制数据？

或者

• 尝试使用我的 s3 object 实现可写目录。

也许我可以更好地分配 s3 UPLOAD_FOLDER ...我只是不确定如何在 FLASK 中建立此连接...

app.config['UPLOAD_FOLDER'] = client.Object(
    Bucket=UPLOAD_FOLDER, Key=UPLOAD_FOLDER_KEY,
    ACL='private'
)

任何帮助表示赞赏。 感觉我可能已经更接近这种方法了..，它确实写入了 s3 Bucket，所以我可能会担心获取可用的 URL，但是 MIDI 文件已损坏并且空白 =(

file_path = f'{UPLOAD_FOLDER_KEY}{file_name}'
            response = client.generate_presigned_post(UPLOAD_FOLDER,
                                                      file_name,
                                                      ExpiresIn=3600)
            post_url = response['url']
            data = response['fields']
            key = data['key']
            with open(file_name, 'wb') as f:
                http_response = requests.post(url=post_url, data=data,
                                              files={file_name: MyMIDI.writeFile(f)})

print(response)产生：

{'url': 'ENDPOINT_URL', 'fields': {'key': 'files(from above)', 'x-amz-algorithm': 'STUFF', 'x-amz-credential': 'STUFF', 'x-amz-date': 'STUFF', 'policy': 'STUFF', 'x-amz-signature': 'STUFF'}}```

如果我可以从中拉出 URL 重定向到... 试图解散这篇关于 S3 文件上传的文章以获得答案，那就不是肯定的了。

Answer 1

我的初步解决方案：我无法准确解释原因； 但我认为问题伴随着MyMIDI.writefile(filehandler)和boto3函数之间的交互。 我认为它在.write()和.close()中与MIDIUtil package 的.writefile()嵌套有关，同时必须同时为s3's Body参数生成字节数据。 所以这是我的解决方法......

# Working Version on S3 Deployment
# Assign generate_presigned_post to variable 
response = client.generate_presigned_post(UPLOAD_FOLDER,
                                          file_name,
                                          ExpiresIn=3600)

# Have MIDIUtil write / close the file within writefile       
with open(file_name, 'wb') as file:
    MIDI_FILE = MyMIDI.writeFile(file)

# Read the written binary contents for the s3 Body; assign to a variable (content)
f = open(file_name, 'rb')  
content = f.read()
       
# Stage the object with its file_name and s3 Bucket(UPLOAD_FOLDER)
MIDI_Object = client_resource.Object(UPLOAD_FOLDER, file_name)

# Write to the s3 Bucket using put
MIDI_Object.put(Body=content)

瞧：它在我的 S3 存储桶中不再是空白的，可以下载了！！ :D

更好的解决方案：（感谢@jarmod）更少的代码行和做同样的事情:)

with open(MIDI_file_name, 'wb') as file:
    MIDI_FILE = MIDI_DATA.writeFile(file)

object_name = os.path.basename(file_name)
client.upload_file(MIDI_file_name, UPLOAD_FOLDER, object_name)

Answer 2

此代码将无法按预期工作：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

此代码的问题在于您正在使用上下文管理器 ( with )，并且在您退出上下文管理器之前，上下文管理器不会关闭 output 文件。 因此，在您尝试将文件上传到 S3 时，文件内容不会刷新到磁盘。

需要这样写：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)

client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)