Files not being received correctly when sent through sockets over LAN

Question

I am trying to build an application that will send files over wireless LAN through the Python 3.4 socket module. It appears to be able to send files within one workstation, but when attempting to send files between two computers on the same network, the computer only appears to receive the first few thousand bytes, the amount dependent on the total size of the file when sent.

For example, a 127 byte file was sent with no issues, but of a 34,846 byte file, only 1,431 or 4,327 bytes were received (only these two numbers of bytes seemed to be received, seemingly randomly switching between the two) and of a 65,182 byte file, only 5,772 or 4,324 bytes were received (the same situation). From looking at the contents of the received files, it appeared that it was the first few bytes that were received.

Both systems have free, accessible RAM exceeding 2GB and sufficient storage space. The server is being run on Windows 8.1, Python 3.4.2, and the client is Ubuntu 14.04 Linux, Python 3.4.0.

My code may be piecemeal and generally poorly written, as I am a beginner without a formal computer science education or any notable experience, especially in network programming. However, I have rooted through the code and racked my brain with no clear solution presenting itself.

Server (Host):

import os
import socket
import struct
import time
import hashlib

def md5(fname):
    hash_md5 = hashlib.md5()
    with open(fname, "rb") as readFile:
        for chunk in iter(lambda: readFile.read(4096), b""):
            hash_md5.update(chunk)
    return hash_md5.hexdigest()

try:
    s = socket.socket()
    host = socket.gethostname()
    port = 26
    s.bind(("0.0.0.0", port))

    filename = input("File to send? ")

    fileLength = os.stat(filename).st_size
    print("Length:", fileLength, "bytes")
    fileLengthBytes = struct.pack(">L", fileLength)

    filenameBytes = bytes(filename, "ascii")
    filenameLength = struct.pack(">L", len(filenameBytes))

    checksum = md5(filename)
    checksumBytes = bytes(bytearray.fromhex(checksum))

    s.listen(5)
    while True:
        c, addr = s.accept()
        print("Connection from", addr)
        c.send(checksumBytes)
        time.sleep(0.1)
        c.send(filenameLength + fileLengthBytes)
        time.sleep(0.1)
        with open(filename, "rb") as f:
            c.send(filenameBytes + f.read())
        c.close()
finally:
    try:
        c.close()
    except NameError:
        pass
    except Exception as e:
        print(type(e), e.args, e)
    try:
        f.close()
    except NameError:
        pass
    except Exception as e:
        print(type(e), e.args, e)

Client (Recipient):

import socket
import struct
import hashlib

def md5(fname):
    hash_md5 = hashlib.md5()
    with open(fname, "rb") as readFile:
        for chunk in iter(lambda: readFile.read(4096), b""):
            hash_md5.update(chunk)
    return hash_md5.hexdigest()


CHECKSUM_LENGTH = 16

try:
    s = socket.socket()
    host = socket.gethostname()
    port = 26

    ip = input("Connect to IP: ")

    s.connect((ip, port))

    initialChecksum = s.recv(CHECKSUM_LENGTH)

    received = s.recv(8)
    # 1st 4 bytes: filename length
    # 2nd 4 bytes: file contents length

    filenameLength = struct.unpack(">L", received[0:4])[0]
    fileLength = struct.unpack(">L", received[4:])[0]
    print("Length:", fileLength)

    bytesToReceive = filenameLength + fileLength
    receivedBytes = s.recv(bytesToReceive)

    filename = str(receivedBytes[0:filenameLength], "ascii")
    f = open(filename, "wb")
    f.write(receivedBytes[filenameLength:]) # Write file contents
    actualChecksum = bytes(bytearray.fromhex(md5(filename)))

    if initialChecksum == actualChecksum:
        print("Fully received", filename)
    else:
        print(filename, "not received correctly")
finally:
    try:
        f.close()
    except NameError:
        pass
    except Exception as e:
        print(type(e), e.args, e)
    try:
        s.close()
    except NameError:
        pass
    except Exception as e:
        print(type(e), e.args, e)

I am aware that the md5 function that I use doesn't appear to work for small files below, I assume, 4 kilobytes.

Where is the problem? How can it be solved? Am I missing something important?

Answer 1

You are receiving only the first few packages of the transmission. This is because the s.recv() will receive the bytes that have reached the client already, but at most the number given as argument. On a local machine the transmission is fast, you will receive more.

To get all parts of the transmission you should collect all bytes until the expected length has been reached.

A very simplistic approach would be something like:

buffer = ''
while len(buffer) < bytes_to_receive:
    buffer += s.recv(1500)

1500 is the typical LAN MTU and a good value to get at least a full package. The code given is only a simple example for Python 2 to explain the concept. It needs to be refined and optimized and, if you are using Python 3, adapted to bytes.