简体   繁体   English

使用 Python 驱动器 API 无法上传 > 200 MB csv 文件

[英]Unable to upload > 200 MB csv file using Python Drive API

I am trying to automate my daily tasks using drive API and I came across an error when I try to upload the larger files (> 300 MB CSV) whereas the following code is working for uploading the smaller files (<100 MB).我正在尝试使用驱动器 API 自动化我的日常任务,当我尝试上传较大的文件(> 300 MB CSV)时遇到错误,而以下代码适用于上传较小的文件(<100 MB)。 Added that I am running the snippet in my Debian Linux VM and no issues from my VM end.补充说我正在我的 Debian Linux VM 中运行该片段,并且我的 VM 端没有问题。

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
import os



gauth = GoogleAuth()
# Try to load saved client credentials if already done
gauth.LoadCredentialsFile("mycreds.txt")


drive_path = '<drive_id>'

data_to_be_uploaded = "my_big_file.csv"
    
# Declared the variable so that it will be uploaded as per the last date
file1 = drive.CreateFile({'title': my_big_csv_data+'.csv',
                          'parents': [{'id': drive_path}]
                          })
file1.SetContentFile(data_to_be_uploaded)
file1.Upload()

The error as following错误如下

  File "test_code.py", line 72, in <module>
    file1.Upload()
  File "/usr/share/hunch/lib/python3.6/site-packages/pydrive/files.py", line 285, in Upload
    self._FilesInsert(param=param)
  File "/usr/share/hunch/lib/python3.6/site-packages/pydrive/auth.py", line 75, in _decorated
    return decoratee(self, *args, **kwargs)
  File "/usr/share/hunch/lib/python3.6/site-packages/pydrive/files.py", line 369, in _FilesInsert
    http=self.http)
  File "/usr/share/hunch/lib/python3.6/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/share/hunch/lib/python3.6/site-packages/googleapiclient/http.py", line 810, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "/usr/share/hunch/lib/python3.6/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/share/hunch/lib/python3.6/site-packages/googleapiclient/http.py", line 970, in next_chunk
    headers=headers)
  File "/usr/share/hunch/lib/python3.6/site-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/share/hunch/lib/python3.6/site-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/share/hunch/lib/python3.6/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/usr/share/hunch/lib/python3.6/site-packages/httplib2/__init__.py", line 1690, in _request
    content,
httplib2.RedirectMissingLocation: Redirected but the response is missing a Location: header.```

There are 100MB limits on some types of files某些类型的文件有 100MB 的限制

https://support.google.com/drive/answer/37603?hl=en https://support.google.com/drive/answer/37603?hl=en

Solved by downgrading my httplib2 package to 0.13.1通过将我的 httplib2 package 降级到 0.13.1 来解决

Reference used here 此处使用的参考

Thanks !谢谢 ! @balderman @balderman

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM