TCP client/server with sockets, server sending files to clients, client hangs, Python

Question

I want to write a simple TCP server using sockets in Python. The server should send the image to the connected client. The client should receive the image. But, for now on, the client receives only the portion of the image, and I can't even open it.

Server is multi-client using select, but its not the problem here. I think that the problem is with sending image.

I wanted "the protocol" to be very simple here.

SERVER                   CLIENT
               GET
       <----------------
              IMAGE
       ----------------->
      END OF COMMUNICATION

So the client can only send the "GET" message to server, and the server, after getting the "GET" string, should immediately send the whole image to client. That's it, communication is over.

server.py

#!/usr/bin/env python

import random
import socket, select
from time import gmtime, strftime

image = 'image.png'

HOST = '127.0.0.1'
PORT = 6666

connected_clients_sockets = []

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((HOST, PORT))
server_socket.listen(10)

connected_clients_sockets.append(server_socket)

while True:

    read_sockets, write_sockets, error_sockets = select.select(connected_clients_sockets, [], [])

    for sock in read_sockets:

        if sock == server_socket:

            sockfd, client_address = server_socket.accept()
            connected_clients_sockets.append(sockfd)

        else:
            try:
                data = sock.recv(4096)
                bytes = open(image).read()

                if data:
                    sock.send(bytes)

            except:
                sock.close()
                connected_clients_sockets.remove(sock)
                continue

server_socket.close()

client.py

#!/usr/bin/env python

import socket
import sys

HOST = '127.0.0.1'
PORT = 6666

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (HOST, PORT)
sock.connect(server_address)

try:

    sock.sendall("GET")

    while True:

        myfile = open('imagefromserv.png', 'w')

        while True:
            data = sock.recv(4096)
            if not data:
                break
            myfile.write(data)
        myfile.close()

finally:
    sock.close()

I'm using Python 2.7 on newest Ubuntu.

------------------------------------------------------------------------------------------------------------------------------------ EDIT ------------------------------------------------------------------------------------------------------------------------------------

Following advices given by one of the users in comments, I TRIED to implement a simple protocol:

CLIENT                                      SERVER
                      GET\r\n   
       ----------------------------------->
                      OK\r\n
       <----------------------------------- 
                   GET_SIZE\r\n
       ----------------------------------->
                    SIZE 1024\r\n
       <-----------------------------------
                   GET_IMG\r\n
       ----------------------------------->
                  IMG_DATA\r\r
       <-----------------------------------

Everything seems to work, but after the image transfer, my CPU is 100% busy, as top says. And ....

Server's output:

--GET--
--GET_SIZE--
--24518--
--GET_IMG--

Client's output:

--OK--
--SIZE 24518--
--24518--
4096
8192
12288
16384
20480
24523
Image received successfully

Indicates that the client received the image successfully. Is it ok now? I mean, I got the image from server, but I do not know, if I implemented the protocol correctly. Maybe something can be improved here?

client.py:

#!/usr/bin/env python

import socket
import sys

HOST = '127.0.0.1'
PORT = 6666

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (HOST, PORT)
sock.connect(server_address)
fname = 'fromserver.png'

try:

    sock.sendall("GET\r\n")
    data = sock.recv(4096)

    if data:
        txt = data.strip()
        print '--%s--' % txt

        if txt == 'OK':

            sock.sendall("GET_SIZE\r\n")
            data = sock.recv(4096)

            if data:
                txt = data.strip()
                print '--%s--' % txt

                if txt.startswith('SIZE'):

                    tmp = txt.split()
                    size = int(tmp[1])

                    print '--%s--' % size

                    sock.sendall("GET_IMG\r\n")

                    myfile = open(fname, 'wb')

                    amount_received = 0
                    while amount_received < size:
                        data = sock.recv(4096)
                        if not data :
                            break
                        amount_received += len(data)
                        print amount_received

                        txt = data.strip('\r\n')

                        if 'EOF' in str(txt) :
                            print 'Image received successfully'
                            myfile.write(data)
                            myfile.close()
                        else :
                            myfile.write(data)
finally:
    sock.close()

server.py:

    #!/usr/bin/env python

import random
import socket, select
from time import gmtime, strftime

image = 'tux.png'

HOST = '127.0.0.1'
PORT = 6666

connected_clients_sockets = []

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((HOST, PORT))
server_socket.listen(10)

connected_clients_sockets.append(server_socket)

while True:

    read_sockets, write_sockets, error_sockets = select.select(connected_clients_sockets, [], [])

    for sock in read_sockets:

        if sock == server_socket:

            sockfd, client_address = server_socket.accept()
            connected_clients_sockets.append(sockfd)

        else:
            try:
                data = sock.recv(4096)

                if data :

                    txt = data.strip()
                    print '--%s--'%txt

                    if txt == 'GET' :
                        sock.sendall('OK\r\n')

                    elif txt == 'GET_SIZE' :

                        with open ('tux.png','rb') as f1:
                            file_size = len(f1.read())
                            f1.seek(0)

                        print '--%s--'%file_size

                        file_size = '%s' % file_size
                        sock.sendall('SIZE %s\r\n' % file_size)

                    elif txt == 'GET_IMG' :
                        with open(image, 'rb') as fp:
                            image_data = fp.read()

                        msg = '%sEOF\r\r' % image_data
                        sock.sendall(msg)
                        print msg

            except:
                sock.close()
                connected_clients_sockets.remove(sock)
                continue

server_socket.close()

Or maybe I should rather do:

sock.sendall(image_data)
sock.sendall('EOF\r\n')

instead of:

msg = '%sEOF\r\n' % image_data
sock.sendall(msg)

in client?

Answer 1

You accidentally forgot to just use HTTP and Twisted.

Server:

from twisted.web.static import File
from twisted.web.resource import Resource
def resource():
    resource = Resource()
    resource.putChild(b"", File(u"xkcd/sandwich.png"))
    return resource

Client:

from filepath import FilePath
from twisted.internet.task import react
from treq import get, content

def main(reactor):
    d = get(b"http://localhost:8080/")
    d.addCallback(content)
    d.addCallback(FilePath(u"image.png").setContent)
    return d

react(main, [])

Server demo:

(everything) exarkun@baryon:/tmp/demo$ twist web --class server.resource
2017-02-23T21:32:14-0500 [-] Site starting on 8080
2017-02-23T21:32:14-0500 [twisted.web.server.Site#info] Starting factory <twisted.web.server.Site instance at 0x7fd1ef81a8c0>
2017-02-23T21:32:14-0500 [twisted.application.runner._runner.Runner#info] Starting reactor...
2017-02-23T21:33:01-0500 [twisted.python.log#info] "127.0.0.1" - - [24/Feb/2017:02:33:01 +0000] "GET / HTTP/1.1" 200 21489 "-" "-"
^C
2017-02-23T21:33:05-0500 [-] Received SIGINT, shutting down.
2017-02-23T21:33:05-0500 [-] (TCP Port 8080 Closed)
2017-02-23T21:33:05-0500 [twisted.web.server.Site#info] Stopping factory <twisted.web.server.Site instance at 0x7fd1ef81a8c0>
2017-02-23T21:33:05-0500 [-] Main loop terminated.
(everything) exarkun@baryon:/tmp/demo$ 

Client demo:

(everything) exarkun@baryon:/tmp/demo$ ls -l image.png
ls: cannot access 'image.png': No such file or directory
(everything) exarkun@baryon:/tmp/demo$ python client.py 
(everything) exarkun@baryon:/tmp/demo$ ls -l image.png 
-rwxr-xr-x 1 exarkun exarkun 21489 Feb 23 21:33 image.png
(everything) exarkun@baryon:/tmp/demo$

If you want to learn more about how select-loop driven networking is done, you can peruse the Twisted implementation .

Answer 2

Your client sends the string "GET". You only want to send and receive image data and "GET" is not image data.

You may have other bugs, it's hard to tell without understanding your protocol. For example, how does one side know what it has gotten all the image data?

Answer 3

您可以阅读以下内容： https : //picamera.readthedocs.io/en/release-1.12/recipes1.html并学到了很多有关如何通过插槽发送图片的知识。“基本食谱”为您提供了所需的一切。

Answer 4

The code below the EDIT section seems good to me. I completely agree with @David Schwartz. If you're looking to implement a protocol you have to take into consideration a whole lot of things regarding the protocol design. Ex:

• if you send a command, how the server reacts
• which commands are allowed
• implement error codes for server responses, etc

For that you can read "TCP/IP and Linux Protocol Implementation" it is a great book for such matter.

In the other hand, you can stick with the basics and continue using TCP sockets,my humble suggestion is that you implement some mechanism for the client to know if it has the complete information, like:

• send a hash of the data, check the hash in the client
• cut the info in chunks and send the chunk quantity, then in the client check how many chunks were received, and re-request the image in case of failure
• or all of the above

Answer 5

This is really strange. I tried two different images, and the code worked. The problem was with the image then.