Python requests send text with binary data

Question

I'm trying to send both binary data and a text string (or any other normal variable) between servers using python's requests library.

I have two servers talking to each other. Server A sends a GET request via python's requests library. It looks something like this:

color, file_data = requests.get('https://www.serverb.com/testing', params = {...})

Server B then does something like:

var1 = request.args.get('var1')
var2 = request.args.get('var2')

# This function creates a binary file, saves it to disk, and returns the file name.
res = some_function(var1, var2)

file_data = codecs.open(res, 'rb').read()

return 'Success!', file_data

However, when I attempt this, Server B throws a 500 error.

Is what I'm attempting to do impossible? If I can only send back the binary data, but with custom headers, that would work as well, and help on how to set those up would be appreciated.

Thanks in advance.

Answer 1

An HTTP response looks like this:

HTTP/1.1 200 OK
<headers>
<blank line>
<response body>

The <blank line> generally contains these characters - \\r\\n , also known as CRLF .

So, when a server sends a response, the client (or browser) can differentiate from the headers and the response body by looking at the <blank line> . So, anything after <blank line> is treated as the response body.

What that means is, there can be at most 1 response body only . So, even if you think you're sending color and file_data separately like you're doing, they're still going in one response and the client will treat them as a single body.

---

What you can do to get around this?

You'll need a way to know which part of the response body is the color and which is file_data . You can do that by sending a custom header along with the response.

Let's say, your response looks like this:

HTTP/1.1 200 OK
<headers>
<blank line>
<color><file_data>

Now, you can set a custom header on your response which will tell you how many bytes are there of the color variable.

For example, if the value of color is yellow and it's 6 characters, that means it has 6 bytes. So, you can set a custom header that would tell you the total length of color :

HTTP/1.1 200 OK
<headers>
X-Color-Length: 6
<blank line>
<color><file_data>

Then, in your client side, you can get the total bytes from the X-Color-Length header and read that number of bytes from the body to get the value of the color . Example:

response = requests.get(...)

color_len = response.headers['X-Color-Length']

color = response.content[:color_len]
file_data = response.content[color_len:]

---

UPDATE: Now that I think about it, I've made this a little more complex than it should be. You can just send the color's value directly in a header like X-Color: <color> , instead of the number of bytes it has.