如何使用请求发布发送 BytesIO

Question

I am trying to send msgpack encoded values to server using requests post to an insert rest endpoint which takes the serialized byte stream as input. But it seems like the values are not getting there properly as I see no values getting inserted in the table. I have never tried this before, so please pardon my ignorance. Here's what I am doing:

buf = io.BytesIO()
for rows in dataframe:
    buf.write(msgpack.packb(rows))
    response = requests.post(url, data=buf, verify=False)
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as err:
        print('Error fetching response using requests')

Answer 1

extending your answer, using a with block is advised in-order to fix memory leak in case of exception.

for rows in dataframe:
    with io.BytesIO(msgpack.packb(rows)) as buf:
        buf.seek(0)
        response = requests.post(url, data=buf, verify=False)
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as err:
        print('Error fetching response using requests')

Answer 2

You can also add initial data when you create the BytesIO object so you don't have to call seek :

>>> import io
>>> buf = io.BytesIO(b"test")
>>> buf.tell()
0
>>> buf.read()
b'test'
>>> buf.tell()
4

Answer 3

I am trying to send msgpack encoded values to server using requests post to an insert rest endpoint which takes the serialized byte stream as input. But it seems like the values are not getting there properly as I see no values getting inserted in the table. I have never tried this before, so please pardon my ignorance. Here's what I am doing:

buf = io.BytesIO()
for rows in dataframe:
    buf.write(msgpack.packb(rows))
    response = requests.post(url, data=buf, verify=False)
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as err:
        print('Error fetching response using requests')

Reading other similar kind of questions/answers, I even tried passing buf.seek(0) and buf.getvalue() . Not sure if that's appropriate to assign it to the data argument, but those didn't work either.

Answer 4

Whenever I write to the buffer, the pointer will always point to the end of the buffer and waiting for a new write. So, when I pass data=buf , that is going to make an insertion with no values. The solution is to do buf.seek(0) and then pass data=buf as a parameter or simply data=buf.seek(0)

So, the following works:

for rows in dataframe:
    buf = io.BytesIO()
    buf.write(msgpack.packb(rows))
    buf.seek(0)
    response = requests.post(url, data=buf, verify=False)
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as err:
        print('Error fetching response using requests')