MIDIutil 通過 Flask 將 MIDI 文件寫入 boto3 API 服務器

Question

在連接到 s3 服務器的 Flask 應用程序中嘗試編寫 MIDIUtil 文件時遇到了困難。

在本地實例中，這並不費力：

LOCAL_UPLOAD_FOLDER = './_static/uploads/MIDI_files/'
file_name = "NAME.mid"
file_path = f'{LOCAL_UPLOAD_FOLDER}{file_name}'
MyMIDI = MIDIFile(1)

with open(file_path, "wb") as output_file:
     MyMIDI.writeFile(output_file)

但是，我不確定如何將其應用於 s3 資源，這是我的實例化...

def get_upload_folder(UPLOAD_FOLDER=None, UPLOAD_FOLDER_KEY=None,
                      client_resource=None, client=None):
    """ Determines How to Upload / Send File for Download """
    # Flask Cloud Settings - Upload Folder
    if os.getenv('CONTEXT') == 'CLOUD':

        # Client Side
        UPLOAD_FOLDER_TYPE = 'CLOUD'
        session = boto3.session.Session()
        client = session.client(
            's3', endpoint_url=os.getenv('ENDPOINT_URL'),
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name=os.getenv('REGION_NAME'), aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

        # Resource Side
        client_resource = boto3.resource(
            's3', endpoint_url='https://nyc3.digitaloceanspaces.com',
            config=botocore.config.Config(s3={'addressing_style': 'virtual'}),
            region_name='nyc3', aws_access_key_id=os.getenv('SECRET_ID'),
            aws_secret_access_key=os.getenv('SECRET_KEY')
        )

    UPLOAD_FOLDER, UPLOAD_FOLDER_KEY = 'MY_BUCKET', 'uploads/MIDI_files/'

   return UPLOAD_FOLDER_TYPE, UPLOAD_FOLDER, UPLOAD_FOLDER_KEY, client_resource, client

到目前為止，我已經嘗試過：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.download_fileobj(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

以及大量其他.put_object與 client 和 client_resource boto3 對象的組合......

我在想我的問題在於：

• MIDIUtil.Midifile 的writeFile(filehandler) MIDIUtil.Midifile )

也許這個 function 正在關閉 MIDI 二進制文件put_object DATA，然后我才能將對象放入 s3 BODY=中？ 也許我需要通過 Bytes(IO) / stream object.. 來解析二進制數據？

或者

• 嘗試使用我的 s3 object 實現可寫目錄。

也許我可以更好地分配 s3 UPLOAD_FOLDER ...我只是不確定如何在 FLASK 中建立此連接...

app.config['UPLOAD_FOLDER'] = client.Object(
    Bucket=UPLOAD_FOLDER, Key=UPLOAD_FOLDER_KEY,
    ACL='private'
)

任何幫助表示贊賞。 感覺我可能已經更接近這種方法了..，它確實寫入了 s3 Bucket，所以我可能會擔心獲取可用的 URL，但是 MIDI 文件已損壞並且空白 =(

file_path = f'{UPLOAD_FOLDER_KEY}{file_name}'
            response = client.generate_presigned_post(UPLOAD_FOLDER,
                                                      file_name,
                                                      ExpiresIn=3600)
            post_url = response['url']
            data = response['fields']
            key = data['key']
            with open(file_name, 'wb') as f:
                http_response = requests.post(url=post_url, data=data,
                                              files={file_name: MyMIDI.writeFile(f)})

print(response)產生：

{'url': 'ENDPOINT_URL', 'fields': {'key': 'files(from above)', 'x-amz-algorithm': 'STUFF', 'x-amz-credential': 'STUFF', 'x-amz-date': 'STUFF', 'policy': 'STUFF', 'x-amz-signature': 'STUFF'}}```

如果我可以從中拉出 URL 重定向到... 試圖解散這篇關於 S3 文件上傳的文章以獲得答案，那就不是肯定的了。

Answer 1

我的初步解決方案：我無法准確解釋原因； 但我認為問題伴隨着MyMIDI.writefile(filehandler)和boto3函數之間的交互。 我認為它在.write()和.close()中與MIDIUtil package 的.writefile()嵌套有關，同時必須同時為s3's Body參數生成字節數據。 所以這是我的解決方法......

# Working Version on S3 Deployment
# Assign generate_presigned_post to variable 
response = client.generate_presigned_post(UPLOAD_FOLDER,
                                          file_name,
                                          ExpiresIn=3600)

# Have MIDIUtil write / close the file within writefile       
with open(file_name, 'wb') as file:
    MIDI_FILE = MyMIDI.writeFile(file)

# Read the written binary contents for the s3 Body; assign to a variable (content)
f = open(file_name, 'rb')  
content = f.read()
       
# Stage the object with its file_name and s3 Bucket(UPLOAD_FOLDER)
MIDI_Object = client_resource.Object(UPLOAD_FOLDER, file_name)

# Write to the s3 Bucket using put
MIDI_Object.put(Body=content)

瞧：它在我的 S3 存儲桶中不再是空白的，可以下載了！！ :D

更好的解決方案：（感謝@jarmod）更少的代碼行和做同樣的事情:)

with open(MIDI_file_name, 'wb') as file:
    MIDI_FILE = MIDI_DATA.writeFile(file)

object_name = os.path.basename(file_name)
client.upload_file(MIDI_file_name, UPLOAD_FOLDER, object_name)

Answer 2

此代碼將無法按預期工作：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)
    client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)

此代碼的問題在於您正在使用上下文管理器 ( with )，並且在您退出上下文管理器之前，上下文管理器不會關閉 output 文件。 因此，在您嘗試將文件上傳到 S3 時，文件內容不會刷新到磁盤。

需要這樣寫：

with open(file_path, 'wb') as output_file:
    MyMIDI.writeFile(output_file)

client.upload_file(UPLOAD_FOLDER, 'OBJECT_NAME', output_file)