I can't receive more than 64 bytes on custom USB CDC class based STM32 device

Question

currently I try to sent 720 bytes from Windows application to custom STM32 device (now for testing purposes I use Blue Pill - STM32F103xxx). Ah, I forgot to point that I am totally newbie into programming:). So on device side I have 1000 bytes buffers for receiving and sending (Thanks to STMCube for this). Testing device with terminal program ( packets < than 64 bytes) works. Then I rework one of Microsoft examples to be able to sent more data to device. Used device driver on Windows is "usbser.sys". In short my console program do following:

1. Calculate SINE weave (360) samples - 16 bytes size
2. Sent them to USB Device as 720 bytes (byte size protocol for COM port) My problem is that no more than 64 bytes comes into device. Somewhere I read that reason for this can be into built in Rx,Tx Windows buffers (64 bytes long by mention somewhere on internet) and for this into code below I insert:
   • SetupComm(hCom,1000,1000) in hope that this will solve my troubles but nope. Below is "my" code, any ideas how I can fix this?

    #include <windows.h>
    #include <tchar.h>
    #include <stdio.h>
    #include <math.h>  
    
    #define PI 3.14159265
  
    void PrintCommState(DCB dcb)
    {
        //  Print some of the DCB structure values
        _tprintf(TEXT("\nBaudRate = %d, ByteSize = %d, Parity = %d, StopBits = %d\n"),
            dcb.BaudRate,
            dcb.ByteSize,
            dcb.Parity,
            dcb.StopBits);
    }
    
    
    int _tmain(int argc, TCHAR* argv[])
    {
        DCB dcb;
        HANDLE hCom;
        BOOL fSuccess;
        const TCHAR* pcCommPort = TEXT("COM3"); //  Most systems have a COM1 port
        unsigned __int8 aOutputBuffer[720];// Data that will sent to device
        unsigned __int16 aCalculatedWave[360];// Data that will sent to device
        int iCnt; // temp counter to use everywhere 
    
        for (iCnt = 0; iCnt < 360; iCnt = iCnt + 1)
        {
            aCalculatedWave[iCnt] = (unsigned short)(0xFFFF * sin(iCnt * PI / 180));
            if (iCnt > 180) aCalculatedWave[iCnt] = 0 - aCalculatedWave[iCnt];
        }
    
        // 16 bit aCalculatedWaveto to 8 bit aOutputBuffer
        for (int i = 0, j = 0; i < 720; i += 2, ++j)
        {
            aOutputBuffer[i] = aCalculatedWave[j] >> 8; // Hi byte
            aOutputBuffer[i + 1] = aCalculatedWave[j] & 0xFF; // Lo byte
        }
    
        //  Open a handle to the specified com port.
        hCom = CreateFile(pcCommPort,
            GENERIC_READ | GENERIC_WRITE,
            0,      //  must be opened with exclusive-access
            NULL,   //  default security attributes
            OPEN_EXISTING, //  must use OPEN_EXISTING
            0,      //  not overlapped I/O
            NULL); //  hTemplate must be NULL for comm devices
    
        if (hCom == INVALID_HANDLE_VALUE)
        {
            //  Handle the error.
            printf("CreateFile failed with error %d.\n", GetLastError());
            return (1);
        }
        if (SetupComm(hCom,1000,1000) !=0)
            printf("Windows In/Out serial buffers changed to 1000 bytes\n");
        else
            printf("Buffers not changed with error %d.\n", GetLastError());
    
        //  Initialize the DCB structure.
        SecureZeroMemory(&dcb, sizeof(DCB));
        dcb.DCBlength = sizeof(DCB);
    
        //  Build on the current configuration by first retrieving all current
        //  settings.
        fSuccess = GetCommState(hCom, &dcb);
    
        if (!fSuccess)
        {
            //  Handle the error.
            printf("GetCommState failed with error %d.\n", GetLastError());
            return (2);
        }
    
        PrintCommState(dcb);       //  Output to console
    
        //  Fill in some DCB values and set the com state: 
        //  57,600 bps, 8 data bits, no parity, and 1 stop bit.
        dcb.BaudRate = CBR_9600;     //  baud rate
        dcb.ByteSize = 8;             //  data size, xmit and rcv
        dcb.Parity = NOPARITY;      //  parity bit
        dcb.StopBits = ONESTOPBIT;    //  stop bit
    
        fSuccess = SetCommState(hCom, &dcb);
    
        if (!fSuccess)
        {
            //  Handle the error.
            printf("SetCommState failed with error %d.\n", GetLastError());
            return (3);
        }
    
        //  Get the comm config again.
        fSuccess = GetCommState(hCom, &dcb);
    
        if (!fSuccess)
        {
            //  Handle the error.
            printf("GetCommState failed with error %d.\n", GetLastError());
            return (2);
        }
    
        PrintCommState(dcb);       //  Output to console
    
        _tprintf(TEXT("Serial port %s successfully reconfigured.\n"), pcCommPort);
        if (WriteFile(hCom, aOutputBuffer, 720, NULL, 0) != 0)
            _tprintf(TEXT("720 bytes successfully writed to Serial port %s \n"), pcCommPort);
        else
            _tprintf(TEXT("Fail on write 720 bytes to Serial port %s \n"), pcCommPort);
        return (0);
    }

Answer 1

USB bulk endpoints implement a stream-based protocol, ie an endless stream of bytes. This is in contrast to a message-based protocol. So USB bulk endpoints have no concept of messages, message start or end. This also applies to USB CDC as it is based on bulk endpoints.

At the lower USB level, the stream of bytes is split into packets of at most 64 bytes. As per USB full-speed standard, packets cannot be larger than 64 bytes.

If the host sends small chunks of data that are more than 1ms apart, they will be sent and received in separate packets and it looks as if USB is a message-based protocol. However, for chunks of more than 64 bytes, they are split into smaller packets. And if small chunks are sent with less than 1ms in-between, the host will merge them into bigger packets.

Your design seems to require that data is grouped, eg the group of 720 bytes mentioned in the question. If this is a requirement, the grouping must be implemented, eg by first sending the size of the group and then the data.

Since larger groups are split into chunks of 64 bytes and the receive callback is called for every packet, the packets must be joined until the full group is available.

Also note a few problems in your current code (see usbd_cdc_if.c, line 264 ):

  USBD_CDC_SetRxBuffer(&hUsbDeviceFS, &Buf[0]);
  USBD_CDC_ReceivePacket(&hUsbDeviceFS);
  NewDataFromUsb = *Len;

USBD_CDC_SetRxBuffer sets the buffer for the next packet to be received. If you always use the same buffer – as in this case – it's not needed. The initial setup is sufficient. However, it could be used to set a new buffer if the current packet does not contain a full group.

Despite its name, USBD_CDC_ReceivePacket does not receive a packet. Instead, it gives the OK to receive the next package. It should only be called if the data in the buffer has been processed and the buffer is ready to receive the next packet. Your current implementation runs the risk that the buffer is overwritten before it is processed, in particular if you send a group of more than 64 bytes, which will likely result in a quick succession of packets.

Note that Windows hasn't been mentioned here. The Windows code seems to be okay. And changing to Winusb.sys will just make your life harder but not get you packets bigger than 64 bytes.