SOCK_SEQPACKET Unix套接字上的空数据包

Question

I'm playing with SOCK_SEQPACKET type on Unix sockets.

The code I'm using for reading is the classic

ssize_t recv_size = recv(sd, buffer, sizeof(buffer), 0);
if (recv_size < 0) {
    handle_error("recv", errno);
} else if (recv_size > 0) {
    handle_packet(buffer, recv_size);
} else {
    // recv_size == 0 => peer closed socket.
    handle_end_of_stream();
}

While this works just fine, I noticed it is not able to distinguish between a socket closure and a message of size 0. In other words, if on the other end I issue a sequence of calls like this:

send(sd, "hello", strlen("hello"), 0);
send(sd, "", 0, 0);
send(sd, "world", strlen("world"), 0);

…the reader will only receive "hello" and react to the second message with a socket closure, missing the "world" message entirely.

I was wondering if there's some way to disambiguate between the two situations.

Answer 1

What if you make some sort of "confirmation" function on both ends. For example instead of handle_end_of_stream, make something like this:

->send(xx, "UNIQUE_MESSAGE", strlen("UNIQUE_MESSAGE"), 0);

< you receive "UNIQUE_RESPONSE" if connection is still up, if you don't, you know for sure that the other end is closed. Just filter out some sort of "UNIQUE_MESSAGE" and "UNIQUE_RESPONSE" inf your "confirmation" function.

Answer 2

As I mentioned in a comment, zero-length seqpackets (as well as zero-length datagrams) can behave oddly, typically mistaken for disconnects; and for that reason, I definitely recommend against using zero-length seqpackets or datagrams for any purpose.

To illustrate the main problem, and explore the details, I created two test programs. First is receive.c , which listens on an Unix domain seqpacket socket, accepts one connection, and describes what it receives:

#define  _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <signal.h>
#include <string.h>
#include <poll.h>
#include <time.h>
#include <stdio.h>
#include <errno.h>

static volatile sig_atomic_t  done = 0;

static void handle_done(int signum)
{
    done = 1;
}

static int install_done(int signum)
{
    struct sigaction  act;

    memset(&act, 0, sizeof act);
    sigemptyset(&act.sa_mask);
    act.sa_handler = handle_done;
    act.sa_flags = 0;
    if (sigaction(signum, &act, NULL) == -1)
        return errno;

    return 0;
}

static inline unsigned int digit(const int c)
{
    switch (c) {
    case '0': return 0;
    case '1': return 1;
    case '2': return 2;
    case '3': return 3;
    case '4': return 4;
    case '5': return 5;
    case '6': return 6;
    case '7': return 7;
    case '8': return 8;
    case '9': return 9;
    case 'A': case 'a': return 10;
    case 'B': case 'b': return 11;
    case 'C': case 'c': return 12;
    case 'D': case 'd': return 13;
    case 'E': case 'e': return 14;
    case 'F': case 'f': return 15;
    default:  return 16;
    }
}

static inline unsigned int octbyte(const char *src)
{
    if (src) {
        const unsigned int  o0 = digit(src[0]);
        if (o0 < 4) {
            const unsigned int  o1 = digit(src[1]);
            if (o1 < 8) {
                const unsigned int  o2 = digit(src[2]);
                if (o2 < 8)
                    return o0*64 + o1*8 + o2;
            }
        }
    }
    return 256;
}

static inline unsigned int hexbyte(const char *src)
{
    if (src) {
        const unsigned int  hi = digit(src[0]);
        if (hi < 16) {
            const unsigned int  lo = digit(src[1]);
            if (lo < 16)
                return 16*hi + lo;
        }
    }
    return 256;
}

size_t set_unix_path(const char *src, struct sockaddr_un *addr)
{
    char         *dst = addr->sun_path;
    char *const   end = addr->sun_path + sizeof (addr->sun_path) - 1;
    unsigned int  byte;

    if (!src || !addr)
        return 0;

    memset(addr, 0, sizeof *addr);
    addr->sun_family = AF_UNIX;
    while (*src && dst < end)
        if (*src == '\\')
            switch (*(++src)) {
            case '0':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else {
                    *(dst++) = '\0';
                    src++;
                }
                break;
            case '1':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case '2':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case '3':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case 'x':
                byte = hexbyte(src + 1);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case 'a':  *(dst++) = '\a'; src++; break;
            case 'b':  *(dst++) = '\b'; src++; break;
            case 't':  *(dst++) = '\t'; src++; break;
            case 'n':  *(dst++) = '\n'; src++; break;
            case 'v':  *(dst++) = '\v'; src++; break;
            case 'f':  *(dst++) = '\f'; src++; break;
            case 'r':  *(dst++) = '\r'; src++; break;
            case '\\': *(dst++) = '\\'; src++; break;
            default:   *(dst++) = '\\';
            }
        else
            *(dst++) = *(src++);

    *(dst++) = '\0';

    return (size_t)(dst - (char *)addr);
}


int main(int argc, char *argv[])
{
    struct sockaddr_un  addr, conn;
    socklen_t           addrlen, connlen;
    int                 socketfd, connfd;

    if (argc != 2 || !argv[1][0] || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
        fprintf(stderr, "       %s SOCKET_PATH\n", argv[0]);
        fprintf(stderr, "\n");
        return EXIT_FAILURE;
    }

    if (install_done(SIGINT) ||
        install_done(SIGHUP) ||
        install_done(SIGTERM)) {
        fprintf(stderr, "Cannot install signal handlers: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    socketfd = socket(AF_UNIX, SOCK_SEQPACKET, 0);
    if (socketfd == -1) {
        fprintf(stderr, "Cannot create an Unix domain seqpacket socket: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    addrlen = set_unix_path(argv[1], &addr);
    if (bind(socketfd, (const struct sockaddr *)&addr, addrlen) == -1) {
        fprintf(stderr, "Cannot bind to %s: %s.\n", argv[1], strerror(errno));
        close(socketfd);
        return EXIT_FAILURE;
    }

    if (listen(socketfd, 1) == -1) {
        fprintf(stderr, "Cannot listen for incoming connections: %s.\n", strerror(errno));
        close(socketfd);
        return EXIT_FAILURE;
    }

    memset(&conn, 0, sizeof conn);
    connlen = sizeof conn;
    connfd = accept(socketfd, (struct sockaddr *)&conn, &connlen);
    if (connfd == -1) {
        close(socketfd);
        fprintf(stderr, "Canceled.\n");
        return EXIT_SUCCESS;
    }

    if (connlen > 0)
        fprintf(stderr, "Connected, peer address size is %d.\n", (int)connlen);
    else
        fprintf(stderr, "Connected; no peer address.\n");

    while (!done) {
        char     buffer[65536];
        ssize_t  n;
        int      r;

        n = recv(connfd, buffer, sizeof buffer, 0);
        if (n > 0)
            fprintf(stderr, "Received %zd bytes.\n", n);
        else
        if (n == 0) {
            struct pollfd  fds[1];

            fds[0].fd = connfd;
            fds[0].events = 0;
            fds[0].revents = 0;
            r = poll(fds, 1, 0);
            if (r > 0 && (fds[0].revents & POLLHUP)) {
                fprintf(stderr, "Disconnected (revents = %d).\n", fds[0].revents);
                break;
            } else
            if (r > 0)
                fprintf(stderr, "recv() == 0, poll() == %d, revents == %d\n", r, fds[0].revents); 
            else
            if (r == 0)
                fprintf(stderr, "Received a zero-byte seqpacket.\n");
            else
                fprintf(stderr, "recv() == 0, poll() == %d, revents == %d\n", r, fds[0].revents);
        }
    }

    close(connfd);
    close(socketfd);
    return EXIT_SUCCESS;
}

You can compile the above using eg gcc -Wall -O2 receive.c -o receive . To run, give it the Unix domain address to listen on. In Linux, you can use the abstract namespace by prepending \\0 to the address; for example, by running ./receive '\\0example' . Otherwise, the socket address will be visible in the filesystem, and you'll need to remove it (as if it was a file, using rm ) before running ./receive again with the same socket address.

We also need an utility to send seqpackets. The following send.c is very similar (reuses much of the same code). You specify the Unix domain address to connect to, and one or more seqpacket lengths. You can also specify delays in milliseconds (just prepend a - ; ie, negative integers are delays in milliseconds):

#define  _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <signal.h>
#include <string.h>
#include <poll.h>
#include <time.h>
#include <stdio.h>
#include <errno.h>

static volatile sig_atomic_t  done = 0;

static void handle_done(int signum)
{
    done = 1;
}

static int install_done(int signum)
{
    struct sigaction  act;

    memset(&act, 0, sizeof act);
    sigemptyset(&act.sa_mask);
    act.sa_handler = handle_done;
    act.sa_flags = 0;
    if (sigaction(signum, &act, NULL) == -1)
        return errno;

    return 0;
}

static inline unsigned int digit(const int c)
{
    switch (c) {
    case '0': return 0;
    case '1': return 1;
    case '2': return 2;
    case '3': return 3;
    case '4': return 4;
    case '5': return 5;
    case '6': return 6;
    case '7': return 7;
    case '8': return 8;
    case '9': return 9;
    case 'A': case 'a': return 10;
    case 'B': case 'b': return 11;
    case 'C': case 'c': return 12;
    case 'D': case 'd': return 13;
    case 'E': case 'e': return 14;
    case 'F': case 'f': return 15;
    default:  return 16;
    }
}

static inline unsigned int octbyte(const char *src)
{
    if (src) {
        const unsigned int  o0 = digit(src[0]);
        if (o0 < 4) {
            const unsigned int  o1 = digit(src[1]);
            if (o1 < 8) {
                const unsigned int  o2 = digit(src[2]);
                if (o2 < 8)
                    return o0*64 + o1*8 + o2;
            }
        }
    }
    return 256;
}

static inline unsigned int hexbyte(const char *src)
{
    if (src) {
        const unsigned int  hi = digit(src[0]);
        if (hi < 16) {
            const unsigned int  lo = digit(src[1]);
            if (lo < 16)
                return 16*hi + lo;
        }
    }
    return 256;
}

size_t set_unix_path(const char *src, struct sockaddr_un *addr)
{
    char         *dst = addr->sun_path;
    char *const   end = addr->sun_path + sizeof (addr->sun_path) - 1;
    unsigned int  byte;

    if (!src || !addr)
        return 0;

    memset(addr, 0, sizeof *addr);
    addr->sun_family = AF_UNIX;
    while (*src && dst < end)
        if (*src == '\\')
            switch (*(++src)) {
            case '0':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else {
                    *(dst++) = '\0';
                    src++;
                }
                break;
            case '1':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case '2':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case '3':
                byte = octbyte(src);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case 'x':
                byte = hexbyte(src + 1);
                if (byte < 256) {
                    *(dst++) = byte;
                    src += 3;
                } else
                    *(dst++) = '\\';
                break;
            case 'a':  *(dst++) = '\a'; src++; break;
            case 'b':  *(dst++) = '\b'; src++; break;
            case 't':  *(dst++) = '\t'; src++; break;
            case 'n':  *(dst++) = '\n'; src++; break;
            case 'v':  *(dst++) = '\v'; src++; break;
            case 'f':  *(dst++) = '\f'; src++; break;
            case 'r':  *(dst++) = '\r'; src++; break;
            case '\\': *(dst++) = '\\'; src++; break;
            default:   *(dst++) = '\\';
            }
        else
            *(dst++) = *(src++);

    *(dst++) = '\0';

    return (size_t)(dst - (char *)addr);
}

static inline long sleep_ms(const long ms)
{
    struct timespec  t;

    if (ms > 0) {
        t.tv_sec = ms / 1000;
        t.tv_nsec = (ms % 1000) * 1000000;
        if (nanosleep(&t, &t) == -1 && errno == EINTR)
            return 1000 * (unsigned long)(t.tv_sec)
                 + (unsigned long)(t.tv_nsec / 1000000);
        return 0;
    } else
        return ms;
}

static int parse_long(const char *src, long *dst)
{
    const char *end = src;
    long        val;

    if (!src || !*src)
        return errno = EINVAL;

    errno = 0;
    val = strtol(src, (char **)&end, 0);
    if (errno)
        return errno;

    if (end == src)
        return errno = EINVAL;

    while (*end == '\t' || *end == '\n' || *end == '\v' ||
           *end == '\f' || *end == '\r' || *end == ' ')
        end++;

    if (*end)
        return errno = EINVAL;

    if (dst)
        *dst = val;

    return 0;
}

int main(int argc, char *argv[])
{
    char                buffer[65536];
    struct sockaddr_un  conn;
    socklen_t           connlen;
    int                 connfd, arg;
    ssize_t             n;
    long                val, left;

    if (argc < 3 || !argv[1][0] || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
        fprintf(stderr, "       %s SOCKET_PATH [ LEN | -MS ] ...\n", argv[0]);
        fprintf(stderr, "\n");
        fprintf(stderr, "All arguments except the first one, SOCKET_PATH, are integers.\n");
        fprintf(stderr, "A positive integer causes a seqpacket of that length to be sent,\n");
        fprintf(stderr, "a negative value causes a delay (magnitude in milliseconds).\n");
        fprintf(stderr, "\n");
        return EXIT_FAILURE;
    }

    if (install_done(SIGINT) ||
        install_done(SIGHUP) ||
        install_done(SIGTERM)) {
        fprintf(stderr, "Cannot install signal handlers: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    /* Fill buffer with some data. Anything works. */
    {
        size_t  i = sizeof buffer;
        while (i-->0)
            buffer[i] = (i*i) ^ i;
    }

    connfd = socket(AF_UNIX, SOCK_SEQPACKET, 0);
    if (connfd == -1) {
        fprintf(stderr, "Cannot create an Unix domain seqpacket socket: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    connlen = set_unix_path(argv[1], &conn);
    if (connect(connfd, (const struct sockaddr *)&conn, connlen) == -1) {
        fprintf(stderr, "Cannot connect to %s: %s.\n", argv[1], strerror(errno));
        close(connfd);
        return EXIT_FAILURE;
    }

    /* To avoid output affecting the timing, fully buffer stdout. */
    setvbuf(stdout, NULL, _IOFBF, 65536);

    for (arg = 2; arg < argc; arg++)
        if (parse_long(argv[arg], &val)) {
            fprintf(stderr, "%s: Not an integer.\n", argv[arg]);
            close(connfd);
            return EXIT_FAILURE;
        } else
        if (val > (long)sizeof buffer) {
            fprintf(stderr, "%s: Seqpacket size too large. Current limit is %zu.\n", argv[arg], sizeof buffer);
            close(connfd);
            return EXIT_FAILURE;
        } else
        if (val >= 0) {
            n = send(connfd, buffer, (size_t)val, 0);
            if (n == (ssize_t)val)
                printf("Sent %ld-byte seqpacket successfully.\n", val);
            else
            if (n != (ssize_t)val && n >= 0)
                fprintf(stderr, "Sent %zd bytes of a %ld-byte seqpacket.\n", n, val);
            else
            if (n < -1) {
                fprintf(stderr, "C library bug: send() returned %zd.\n", n);
                close(connfd);
                return EXIT_FAILURE;
            } else
            if (n == -1) {
                fprintf(stderr, "Send failed: %s.\n", strerror(errno));
                close(connfd);
                return EXIT_FAILURE;
            }
        } else {
            left = sleep_ms(-val);
            if (left)
                fprintf(stderr, "Slept %ld milliseconds (out of %ld ms).\n", -val-left, -val);
            else
                printf("Slept %ld milliseconds.\n", -val);
        }

    if (close(connfd) == -1) {
        fprintf(stderr, "Error closing connection: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    printf("All done, connection closed.\n"); 
    fflush(stdout);

    return EXIT_SUCCESS;
}

Compile this using eg gcc -Wall -O2 send.c -o send .

For testing, I recommend you use two terminal windows. Run send in one, and receive in the other. For simplicity, I'll show the corresponding commands and outputs side-by-side. The machine this runs on is a Core i5 7200U laptop (HP EliteBook 830), running Ubuntu 16.04.4 LTS, 64-bit Linux kernel version 4.15.0-24-generic, and binaried compiled using GCC-5.4.0 20160609 (5.4.0-6ubuntu1~16.04.10) and the abovementioned commands ( gcc -Wall -O2 ).

When we use a small delay before the final send, everything seems to work just fine:

$ ./send '\0example' 1 0 3 0 0 -1 6   │   $ ./receive '\0example'
                                      │   Connected, peer address size is 2.
Sent 1-byte seqpacket successfully.   │   Received 1 bytes.
Sent 0-byte seqpacket successfully.   │   Received a zero-byte seqpacket.
Sent 3-byte seqpacket successfully.   │   Received 3 bytes.
Sent 0-byte seqpacket successfully.   │   Received a zero-byte seqpacket.
Sent 0-byte seqpacket successfully.   │   Received a zero-byte seqpacket.
Slept 1 milliseconds.                 │  
Sent 6-byte seqpacket successfully.   │   Received 6 bytes.
All done, connection closed.          │   Disconnected (revents = 16).

However, when the sender sends the final few seqpackets (starting with a zero-length one) without any delays in between, I observe this:

$ ./send '\0example' 1 0 3 0 0 6      │   ./receive '\0example'
                                      │   Connected, peer address size is 2.
Sent 1-byte seqpacket successfully.   │   Received 1 bytes.
Sent 0-byte seqpacket successfully.   │   Received a zero-byte seqpacket.
Sent 3-byte seqpacket successfully.   │   Received 3 bytes.
Sent 0-byte seqpacket successfully.   │   
Sent 0-byte seqpacket successfully.   │   
Sent 6-byte seqpacket successfully.   │   
All done, connection closed.          │  Disconnected (revents = 16).

See how the two zero-byte seqpackets and the 6-byte seqpacket are missed (because poll() returned revents == POLLHUP . ( POLLHUP == 0x0010 == 16, so there were no other flags set either time.)

I am personally not sure if this is a bug or not. In my opinion, it is just and indication that using zero-length seqpackets is problematic, and should be avoided.

(The peer address length is 2 above, because the sender does not bind to any address, and therefore uses an unnamed Unix domain socket address (as described in the man unix man page). I don't think it is important, but I left it in just in case.)

There was a discussion on one possible solution to the problem at hand, via MSG_EOR (since recvmsg() should add MSG_EOR to the msg_flags field in the msghdr structure; and since it "should be set for all seqpackets", even zero-length ones, it would be a reliable way to detect zero-length seqpackets from end-of-input/read-side shutdown/disconnect) in the Linux Kernel Mailing List (and linux-netdev list) in May 2007. (The archived thread at Marc.info is here ) However, in Linux Unix domain seqpacket sockets, the MSG_EOR is not set nor passed, according to the initial poster, Sam Kumar. The discussion did not lead anywhere; as I read it, nobody was sure what the expected behaviour even should be.

Looking at the Linux kernel changelog for Unix domain sockets , there have been no related changes since that thread either (as of 23 July 2018).

---

The above programs were written in one sitting, without review; so, they could easily have bugs or thinkos in them. If you notice any, or obtain very different results (but do note timing-based effects are sometimes hard to replicate), do let me know in a comment, so I can check, and fix if necessary.