執行多個 SFTP 操作時出現“無法執行請求的操作，因為正在進行文件傳輸”錯誤

Question

我們嘗試從 sftp 下載文件，然后刪除文件（通過 Airflow），我們在其中一個 sftp 服務器上保留了這個錯誤，它在其他 sftp 服務器上工作正常。

[Errno 13] 無法執行請求的操作，因為正在進行文件傳輸

        with ssh_hook_SFTP.get_conn() as ssh_client:
            sftp_client = ssh_client.open_sftp()

            bucket = storage_bucket_client.get_bucket(target_bucket)
            proxy_obj = sftp_client.file(sftp_file_path)
            blob = Blob(gcp_bucket_path, bucket)
            blob.upload_from_file(proxy_obj)

            log.info(f'Executing delete of {sftp_file_path}')
            sftp_client.remove(sftp_file_path)

這是堆棧跟蹤

Traceback (most recent call last):
  File "/home/airflow/gcs/plugins/operators/PcsSftpToGcpOperator.py", line 148, in execute
    self.SFTPAgent(storage_bucket_client, sftp_client, source_folder, file.filename, target_bucket, post_pull_action, rename_prefix, files_list)
  File "/home/airflow/gcs/plugins/operators/PcsSftpToGcpOperator.py", line 262, in SFTPAgent
    sftp_client.remove(sftp_path)
  File "/opt/python3.8/lib/python3.8/site-packages/paramiko/sftp_client.py", line 398, in remove
    self._request(CMD_REMOVE, path)
  File "/opt/python3.8/lib/python3.8/site-packages/paramiko/sftp_client.py", line 813, in _request
    return self._read_response(num)
  File "/opt/python3.8/lib/python3.8/site-packages/paramiko/sftp_client.py", line 865, in _read_response
    self._convert_status(msg)
  File "/opt/python3.8/lib/python3.8/site-packages/paramiko/sftp_client.py", line 896, in _convert_status
    raise IOError(errno.EACCES, text)
PermissionError: [Errno 13] The requested operation cannot be performed because there is a file transfer in progress.

任何幫助表示贊賞

我確認我們可以使用相同的憑據從 sftp 手動下載和刪除文件，所以這肯定不是權限問題

Answer 1

我找到了答案以防有人遇到同樣的問題，我們需要關閉文件

        with ssh_hook_SFTP.get_conn() as ssh_client:
            sftp_client = ssh_client.open_sftp()

            bucket = storage_bucket_client.get_bucket(target_bucket)
            try:
              proxy_obj = sftp_client.file(sftp_file_path)
              blob = Blob(gcp_bucket_path, bucket)
              blob.upload_from_file(proxy_obj)
            finally:
              proxy_obj.close()

            log.info(f'Executing delete of {sftp_file_path}')
            sftp_client.remove(sftp_file_path)