Uploading Large Files to Dropbox using Dropbox API and Python Results in Error

Question

I am trying to upload a large zipped file to Dropbox (about 2-3GB) using Python and the Dropbox API v2. I am using the "chunked method" found here: dropbox API v2 upload large files using python . Here is the code again for convenience:

f = open(file_path)
file_size = os.path.getsize(file_path)

CHUNK_SIZE = 4 * 1024 * 1024

if file_size <= CHUNK_SIZE:

    print dbx.files_upload(f, dest_path)

else:

    upload_session_start_result = dbx.files_upload_session_start(f.read(CHUNK_SIZE))
    cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
                                               offset=f.tell())
    commit = dropbox.files.CommitInfo(path=dest_path)

    while f.tell() < file_size:
        if ((file_size - f.tell()) <= CHUNK_SIZE):
            print dbx.files_upload_session_finish(f.read(CHUNK_SIZE),
                                            cursor,
                                            commit)
        else:
            dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                            cursor.session_id,
                                            cursor.offset)
            cursor.offset = f.tell()

However, when I run it I am getting a

dropbox.exceptions.APIError

as well as

UploadSessoionLookupError

and a

UploadSessionOffsetError

I think the error might be occurring at this line specifically:

 dbx.files_upload_session_append(f.read(CHUNK_SIZE),
                                       cursor.session_id,
                                       cursor.offset)

I've tried swapping that for

dbx.files_upload_session_append_v2(
                f.read(self.CHUNK_SIZE), cursor)

but that didn't work either. Any suggestions?

Answer 1

On windows, be sure to open using binary mode

f = open(file_path, 'rb')

f.read(chunk_size) and f.tell() were off.

from python docs

On Windows, tell() can return illegal values (after an fgets() ) when reading files with Unix-style line-endings. Use binary mode ( 'rb' ) to circumvent this problem.