Does total packets length in tcp buffer can exceed allocated buffer size?

Question

It is known that getsockopt(sock, SOL_SOCKET, SO_SNDBUF, &opt_val, &opt_len); returns the twice of the size of the tcp buffer that was allocated before in setsockopt() .

(As written in man 7 tcp :

Note that TCP actually allocates twice the size of the buffer requested in the setsockopt(2) call, and so a succeeding getsockopt(2) call will not return the same size of buffer as requested in the setsockopt(2) call. TCP uses the extra space for administrative purposes and internal kernel structures, Note that TCP actually allocates twice the size of the buffer requested in the setsockopt(2) call, and so a succeeding getsockopt(2) call will not return the same size of buffer as requested in the setsockopt(2) call. TCP uses the extra space for administrative purposes and internal kernel structures, )

So if I do setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (int *)&buf_size, sizeof(buf_size)) for buf_size = 256K , 512K will be allocated and returned in getsockopt() .

I want to count the current amount of bytes in tcp buffer . for that I'm counting the length of each packet in the queue ( sk->sk_write_queue->len ) while sk is struct sock *sk .

It happens that there are times which the length returned is bigger than 256K . (for instance, I got 294879 bytes that is 32735 bytes more than 256K ).

Why does it happens? Does it includes "extra space for administrative purposes and internal kernel structures" as the getsockopt(.., SOL_SOCKET, SO_SNDBUF, ..) ?

Thanks.

Answer 1

Since 512K actually gets allocated, buffering 294879 bytes is not that surprising. On linux, when you set the SO_SNDBUF, the kernel simply doubles that amount.

If you want the socket buffer to be 256k, call setsockopt() with 128k.

Now, the buffers used are not purely for your data, the skb's and other data structures the kernel needs are allocated in these buffers, which is the "administrative overhead", how much overhead this amounts to depends on how the kernel slices up the data you give it - so reserving exactly 256k of buffer space for only the data you send from userspace is a rather hard if not practically impossible task.