Transferring multiple files from one host to another

Question

I'm new to python socket programming. I want to transfer 5 files (photos) from one host in mininet to another. The name of these files are numbered respectively (I mean 1.jpg, 2.jpg and ...). The problem is that when I run these codes, the first photo is transferred correctly but others become corrupted. What's the problem:

sender.py

import socket
import sys

buf = 1024

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('10.0.0.1',12345))

for i in range(1,6):
    with open("%d.jpg" % (i),'rb') as f:
        data = f.read(buf)
        while 1:
            if not data:
                break
            s.sendall(data)
            data = f.read(buf)
s.close()

receiver.py:

import socket
import sys

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('10.0.0.1', 12345))

buf = 1024

s.listen(1)
conn , addr = s.accept()
for i in range(6,11):
    with open("%d.jpg" % (i),'wb') as f:
        while 1:
            data = conn.recv(buf)
            #print(data[:10])
            #print "PACKAGE RECEIVED..."
            f.write(data)
            if not data: break
#conn.close()
#s.close()

Answer 1

The simple solution to your problem is to create a new connection for each file. The code below does that.

Sender

from __future__ import print_function
import socket

HOST = 'localhost'
PORT = 12345
BUFSIZE = 4096

def send_file(fname):
    with open(fname, 'rb') as f:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        try:
            sock.connect((HOST, PORT))
        except socket.error as err:
            print(err, HOST, PORT)
            sock.close()
            return

        while True:
            data = f.read(BUFSIZE)
            if not data:
                break
            while data:
                sent = sock.send(data)
                data = data[sent:]

    sock.close()

fnames = [
    'test0.jpg',
    'test1.jpg',
    'test2.jpg',
    'test3.jpg',
]

def main():
    for fname in fnames:
        send_file(fname)

if __name__ == '__main__':
    main()
  

Receiver

from __future__ import print_function
import socket

HOST = 'localhost'
PORT = 12345
BUFSIZE = 4096
        
def main():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    try:
        sock.bind((HOST, PORT))
    except socket.error as err:
        print('Bind failed', err)
        return

    sock.listen(1)
    print('Socket now listening at', HOST, PORT)
    file_number = 0
    try:
        while True:
            conn, addr = sock.accept()
            print('Connected with', *addr)

            fname = 'image%d.jpg' % file_number
            with open(fname, 'wb') as f:
                while True:
                    data = conn.recv(BUFSIZE)
                    if not data:
                        break
                    f.write(data)
            conn.close()
            print(fname, 'saved\n')
            file_number += 1

    # Hit Break / Ctrl-C to exit
    except KeyboardInterrupt:
        print('\nClosing')

    sock.close()

if __name__ == '__main__':
    main()
     

You need to hit Ctrl C or Break (depending on your OS) to exit the receiver.

---

But using those numeric file names at the receiver isn't very satisfactory , so I decided to make it a little more complicated. :) In the following version we send the file name before the file data. That's slightly tricky because the receiver needs to separate the file name from the actual file data. If each socket.recv call corresponded to a socket.send call that would be easy, but that's not guaranteed to happen: the received bytes may be split differently from how they were sent. The receiver needs to buffer the bytes so it can break them up correctly. See the Socket Programming HOWTO for details.

So that the receiver knows where the file name ends we first send a single byte that encodes the length of the file name. A byte can hold a number from 0 to 255, so this code can't handle file names longer than that. After the length byte, we send the file name itself, encoded using UTF-8. And then we send the actual file contents.

The receiver uses a class named Receiver to handle the buffering. This class has a .get method which we can use to get a specified number of bytes. We use that method to get the file name length and the file name. And then we use Receiver's .save method to save the received file contents to a new file.

This code is a little bit messy because it's designed to run on Python 2 and Python 3, in any combination. It'd be somewhat neater if it was for Python 3 only. I hard-coded 'localhost' as the host name, since I only have one computer, so I couldn't test it over a network, but I'm confident that it will work correctly on a network.

Here's the sender:

from __future__ import print_function
import socket
from struct import pack

HOST = 'localhost'
PORT = 12345
BUFSIZE = 4096

def send(sock, data):
    while data:
        sent = sock.send(data)
        data = data[sent:]

def send_file(fname):
    with open(fname, 'rb') as f:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        try:
            sock.connect((HOST, PORT))
        except socket.error as err:
            print(err, HOST, PORT)
            sock.close()
            return

        # Send the file name length & the filename itself in one packet          
        send(sock, pack('B', len(fname)) + fname.encode())
        while True:
            data = f.read(BUFSIZE)
            if not data:
                break
            send(sock, data)

    sock.close()

fnames = [
    'test1.gif',
    'test2.jpg',
    'test3.png',
    'test4.pdf',
]

def main():
    for fname in fnames:
        send_file(fname)

if __name__ == '__main__':
    main()

And here's the receiver:

from __future__ import print_function
import socket
from struct import unpack

HOST = 'localhost'
PORT = 12345
BUFSIZE = 4096

class Receiver:
    ''' Buffer binary data from socket conn '''
    def __init__(self, conn):
        self.conn = conn
        self.buff = bytearray()

    def get(self, size):
        ''' Get size bytes from the buffer, reading
            from conn when necessary 
        '''
        while len(self.buff) < size:
            data = self.conn.recv(BUFSIZE)
            if not data:
                break
            self.buff.extend(data)
        # Extract the desired bytes
        result = self.buff[:size]
        # and remove them from the buffer
        del self.buff[:size]
        return bytes(result)

    def save(self, fname):
        ''' Save the remaining bytes to file fname '''
        with open(fname, 'wb') as f:
            if self.buff:
                f.write(bytes(self.buff))
            while True:
                data = self.conn.recv(BUFSIZE)
                if not data:
                    break
                f.write(data)

def main():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    try:
        sock.bind((HOST, PORT))
    except socket.error as err:
        print('Bind failed', err)
        return

    sock.listen(1)
    print('Socket now listening at', HOST, PORT)
    try:
        while True:
            conn, addr = sock.accept()
            print('Connected with', *addr)
            # Create a buffer for this connection
            receiver = Receiver(conn)
            # Get the length of the file name
            name_size = unpack('B', receiver.get(1))[0] 
            # Get the file name itself
            name = receiver.get(name_size).decode()
            print('name', name)
            # Save the file
            receiver.save(name)
            conn.close()
            print('saved\n')
    # Hit Break / Ctrl-C to exit
    except KeyboardInterrupt:
        print('\nClosing')

    sock.close()

if __name__ == '__main__':
    main()

Answer 2

You should use it like:

   with open("%d.jpg" % (i),'wb') as f:
        while 1:
            data = conn.recv(buf)
            #print(data[:10])
            #print "PACKAGE RECEIVED..."
            f.write(data)
            if not data: break

this way f gets closed automatically when you leave the with block when you exit the while .

I guess you need to extend the protocol so that the receiver knows what it gets and that you can let it act on that. Something like a 'begin-of-transfer' ... sending the filename, sending the data, sending 'end-of-transfer' that makes the receiver close the file