发布请求 tensorflow 服务：要解包的值太多（预期 2）

Question

I set up a tensorflow running service with my model, but when I try to do a post request it returns me the following error (get request work):

[nltk_data] Downloading package punkt to /home/viktor/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))
Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.

(3, 20)
Traceback (most recent call last):
  File "bert_api.py", line 99, in <module>
    res , res2= requests.post('http://3.143.108.46:8501/v1/models/colbert:predict', 
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 516, in request
    prep = self.prepare_request(req)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 449, in prepare_request
    p.prepare(
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 317, in prepare
    self.prepare_body(data, files, json)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 508, in prepare_body
    body = self._encode_params(data)
  File "/home/viktor/documents/miniconda3/lib/python3.8/site-packages/requests/models.py", line 97, in _encode_params
    for k, vs in to_key_val_list(data):
ValueError: too many values to unpack (expected 2)

This is my code:

res, res2 = requests.post('http://url:port/v1/models/colbert:predict', 
    data=sentence_input)
print(res.status_code, res.reason)

my data_sentence is an array of arrays:

 [array([[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
       dtype=int32),
 array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=int32]

Answer 1

There are two separate issues in your code. One pertains to the payload, the other to the way you are using requests.post .

Requests usage

requests.post , just as requests.request and other similar functions, returns a single instance of Response class ( source ). For this reason, to fix your error you need to change from

res, res2 = requests.post(...

to

res = requests.post(

---

Now, typically if you try to take one thing and unpack it into two variables, you would get a more clear error saying that there are not enough values to unpack:

>>> a,b = [1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not enough values to unpack (expected 2, got 1)

The error you are getting actually says the opposite - that you have too many values to unpack. And here's why.

The Response class returned by requests.post defines an __iter__(self) method ( source ), which will allow you to iterate over the response in chunks instead of reading it all at once. So when you do res, res2 = response , Python will try to unpack the response for you by using the __iter__ implementation. If you have more than 2 chunks in your response, you won't have enough variables to handle the other chunks, so you get the error "too many values to unpack".

A simple demonstration of this behaviour:

>>> class X():
...     def __iter__(self):
...             for i in range(5):
...                     yield i
...
>>> x = X()
>>> a,b = x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
>>> list(x)
[0, 1, 2, 3, 4]

Payload

When you pass in the data keyword argument to requests.post , it assumes that you want to form-encode it ( docs ). You should either pass a dictionary or a list of tuples, but what you have is a list of arrays, which isn't a form at all, so you don't want to form-encode it.

So when you are passing (data=list_of_arrays) , the library expects the first element to be a tuple with two elements. Instead you have an array with much more elements, which again results in the same error: too many values to unpack .

What you want to do instead is to send it as JSON or just as a string. Typically, you would have to follow the docs and do

import json

payload = [np.random.randint(2, size=30).reshape(3,10), np.random.randint(2, size=30).reshape(3,10)]

res = requests.post('/', data=json.dumps(payload)) # pass it as a string

res = requests.post('/', json=payload) # or let the library serialize it

Unfortunately, you will then run into the issue where np.array isn't serializable, so you have to prepare it for serialization first. As per this answer , you can just call .tolist() on np.array which will serialize it to a list of integers for you.

Thus the final solution

import json

payload = [np.random.randint(2, size=30).reshape(3,10), np.random.randint(2, size=30).reshape(3,10)]

# convert both np.arrays to list
payload = [payload[0].tolist(), payload[1].tolist()]
# or this way
payload = [arr.tolist() for arr in data]

# now the payload is serializable
res = requests.post('/', data=json.dumps(payload)) # pass it as a string

res = requests.post('/', json=payload) # or let the library serialize it

Answer 2

Looking at the error message i see that you are assigning two variables res, res2 to requests.post response that has more than 2 returning values

#for example 
var1, var2 = [1, 2, 3]
#this will raise the same error as your's 

To solve this problem you need to index your response like this below

res, res2 = requests.post('http://url:port/v1/models/colbert:predict', 
    data=sentence_input)[:2]
print(res.status_code, res.reason)

If response to your post request return a list, indexing will solve your problem