Python，App Engine：HTTP多部分POST到Vk.Ads API

Question

I tried to sole this problem myself for a few days searching on examples and documentations, also it wasn't solved on ruSO . So, I hope that solution will be found here, on enSO.

I develop a service for automatic creation of ads at Vk social network using Python and Google App Engine. Initially, pictures for ads are loaded to my server ( part 1 ), then they are uploaded to Vk server at some time ( parts 2.1 and 2.2 ). It seems that pictures are loaded and stored on my server correctly (I downloaded them and compared with original ones — every byte is the same). But I attach the part 1 code just in case.

To upload a picture to Vk.Ads firstly I need to get a URL — this is simple, so skip it. Secondly, I need to send a POST request to this link with field file with binary content of the photo ( API documentation ). I created two ways for that ( 2.1 and 2.2 ), but both of them returns errcode: 2 which means corrupted file . To my mind, the problem is about the requests, but I don't exclude the possibility of it files uploading/storage on my server, or some strange work of the API. I'll appreciate any answers and comments.

1. Uploading to my server

import webapp2
from google.appengine.ext import ndb

# stores pictures on the server
class Photo(ndb.Model):
    name = ndb.StringProperty()
    img = ndb.BlobProperty()

    @staticmethod
    def get(name):
        retval = Photo.query(Photo.name == name).get()
        return retval

    @staticmethod
    def create(name, blob):
        retval = Photo()
        retval.name = name
        retval.img = blob
        return retval

class PhotosPage(webapp2.RequestHandler):
    def get(self):
        # general content of the page:
        html = '''<form action="/photos" method="post" enctype="multipart/form-data">
            <input type="file" name="flimg"/>
            <input value="new_pic" name="flname"/>
            <input type="submit" value="Upload"/> </form>'''

    def post(self):
        n = str(self.request.get('flname'))
        f = self.request.get('flimg')
        p = Photo.get(n)
        if p:
            p.img = f
        else:
            p = Photo.create(n, f)
        p.put()

2.1. POST to API, approach #1, using urlfetch и poster :

from poster.encode import multipart_encode, MultipartParam
from google.appengine.api import urlfetch

name = 'file'
content = ... # file binary content
where = ... # gotten URL

options = {
    'file': MultipartParam(
        name=name,
        value=content,
        filename=name,
        filetype='image/png',
        filesize=len(content))
}

data, headers = multipart_encode(options)
pocket = "".join(data)

result = urlfetch.fetch(
    url=where,
    payload=pocket,
    method=urlfetch.POST,
    headers=headers)

2.2. POST to API, approach #2, using requests :

import requests

name = 'file'
content = ... # file binary content
where = ... # gotten URL

# I also tried without this dict; is it necessary?
data = {
    'fileName': name,
    'fileSize': len(content),
    'description': 'undefined',
}

result = requests.post(where, files={name: StringIO(content)}, data=data)

In addition, for the second approach I extracted the content of my request:

POST
https://pu.vk.com/c.../upload.php?act=ads_add&mid=...&size=m&rdsn=1&hash_time=...&hash=...&rhash=...&api=1

Content-Length: 15946
Content-Type: multipart/form-data; boundary=b4b260eace4e4a7082a99753b74cf51f

--b4b260eace4e4a7082a99753b74cf51f
Content-Disposition: form-data; name="description"
undefined

--b4b260eace4e4a7082a99753b74cf51f
Content-Disposition: form-data; name="fileSize"
15518

--b4b260eace4e4a7082a99753b74cf51f
Content-Disposition: form-data; name="fileName"
file

--b4b260eace4e4a7082a99753b74cf51f
Content-Disposition: form-data; name="file"; filename="file" 
< File binary content >

--b4b260eace4e4a7082a99753b74cf51f-- 

---

UPDATE.

Thanks to SwiftStudier , I found the origin of the problem: StringIO and BytesIO don't behave identical to file open . If I use just open the code works well, but it doesn't with virtual file. How it can be solved?

import requests
from io import BytesIO

with open('path.to/file.png', 'rb') as fin:
    content = BytesIO(fin.read())

token = '...'
url = 'https://api.vk.com/method/ads.getUploadURL?access_token=' + token + '&ad_format=2'
upload_url = requests.get(url).json()['response']

post_fields = {
    'access_token': token
}

data_fields = {
    # This works:
    # 'file': open('path.to/file.png', 'rb')

    # But this does not:
    'file': content
}

response = requests.post(upload_url, data=post_fields, files=data_fields)
print(response.text)

Answer 1

Not sure if it can help, but I'll post it anyway

I used requests to upload an image to ads

import requests

token = '***'
url = f'https://api.vk.com/method/ads.getUploadURL?access_token={token}&ad_format=1' # I set add_format randomly just to avoid an error of this parameter was missing
upload_url = requests.get(url).json()['response']

post_fields = {
    'access_token': token
}

data_fields = {
    'file': open('/path/to/image.png', 'rb')
}

response = requests.post(upload_url, data=post_fields, files=data_fields)
print(response.text)

The result looks like valid photo upload, the data received can be used in further actions with ad API.

Answer 2

After a lot of experiments and investigating of different HTTP requests content I found out the only difference between wrong and working code. It was about 4 bytes only: file name MUST contain the extension. Vk API even ignores Content-Type: image/png , but needs .png or similar in filename. So, this doesn't work:

requests.post(upload_url, files={
    'file': BytesIO('<binary file content>')
})

But this option works properly:

requests.post(upload_url, files={
    'file': ('file.png', BytesIO('<binary file content>'), 'image/png')
})

Just like this one, which is not available for GAE:

requests.post(upload_url, files={
    'file': open('/path/to/image.png', 'rb')
})

---

Both StringIO and StringIO are appropriate for that task. As mentioned, Content-Type does not matter, it can be just multipart/form-data .