python sending http response

Question

I am trying to write a very simple HTTP server which sends a video being streamed server side. When a client connects, the get_video program (fictional) is started in another process and its stdout is being piped to us (under the assumption get_video sends/streams video to stdout). I am using subprocess.Popen() for this.

import subprocess, socket

def send_video(sock, programme_id):
    p = subprocess.Popen(["get_video","--pid",programme_id], stdout=subprocess.PIPE)
    sock.send("HTTP/1.1 200 OK\nContent-type: application/octet-stream\n\n")
    while True:
        chunk = p.stdout.read(1024)
        if chunk:
            try:
                sock.send(chunk)
            except Exception:
                pass
        else:
            break

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('',8080))
    s.listen(5)

    while True:
        client, address = s.accept()
        data = client.recv(1024)
        send_video(client, "123456")
        client.close()

if __name__ == "__main__":
    main()

This barely works. If I issue an HTTP request using wget http://localhost:8080/blah.mp4 the whole thing does as expected - the video is streamed to the client's socket and appended to the new file, blah.mp4.

However, if I create a dummy HTML page with <a href="http://localhost:8080/blah/mp4">download</a> and then try 'save target/link as...' on that link, the get_video program is invoked twice. The second time being when the video is actually sent to the client's socket. There must be some sort of buffering going on.

Notice the try/except block in send_video() . I was getting a 'broken pipe' error before (with the dummy HTML page method), indicating the client's socket wasn't there to be written to. I put the try/except there to try and ignore it.

I am fairly confused. The HTTP requests look the same, I'm not sure what my browser (Firefox) is doing differently to cause this.

Any ideas?

Edit:

The header from wget:

GET /blah.mp4 HTTP/1.0
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: 192.168.1.2:8080
Connection: Keep-Alive

The header from html dummy page:

GET /blah.mp4 HTTP/1.1
Host: 192.168.1.2:8080
User-Agent: Mozilla/5.0 (blah) Gecko/blah Firefox/3.6.16
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

The python server reported only one of each, but Wireshark reported 2 GET requests for the dummy html link method. I don't see anything obvious here...

Answer 1

It dawned on me that, because the browser sends a GET followed by a FIN,ACK after the 'save target as...' has been clicked on the link, then that'll be what is causing the broken pipe error. It is when you choose where you want to save the file and hit 'OK' button in the browser that another GET is issued. What was happening was I was still stuck in that while True loop once I had detected the broken pipe. So I was continuing to read from the Popen object and failing to send it to the client every time. Once this finished, the next GET was able to continue and the transfer was then successful. Phew.

In short, the working code:

import subprocess, socket

def send_video(sock, programme_id):
    p = subprocess.Popen(["get_video","--pid",programme_id], stdout=subprocess.PIPE)
    sock.send("HTTP/1.1 200 OK\nContent-type: application/octet-stream\n\n")
    while True:
        chunk = p.stdout.read(1024)
        if chunk:
            try:
                sock.send(chunk)
            except socket.error, e:
                sock.close()
                break
        else:
            break

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('',8080))
    s.listen(5)

    while True:
        client, address = s.accept()
        data = client.recv(1024)
        send_video(client, "123456")
        client.close()

if __name__ == "__main__":
    main()

Thanks.