How to pass a buffer to write() without allocating memory for it

Question

It's hard to phrase the question, but I'll try. I want to test performance of my usb connection, therefor I'm using gadgetfs on an ambedded device so I can just do a write to a filedescriptor and the data is sent to the host. But when I want to send large amounts of data, I cannot allocate as much data as I want to.

write(int fildes, const void *buf, size_t nbyte);

Is there a way to pass a pointer to somewhere in memory where it can actually read nbyte Bytes? I can't allocate an array or a vector of the size that I want to send. cause I get an "Cannot allocate memory" error.

Answer 1

If your OS has MAP_ANONYMOUS or /dev/zero then you could mmap that into your virtual address space up to the maximum possible mapping and read from there.

#include <sys/mman.h>

void * zeroed_memory = mmap(NULL, nbytes, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

This will allocate a continuous mapping of length nbytes to a single zeroed out read-only page and thus does not actually use any physical memory. You can read the whole array (and will only get zeroes), but you can not write to it.

Most systems will limit your maximum continuous virtual mapping though due to physical hardware restrictions. Current x86_64 Linux generally supports 64TB long mappings.

Note: some systems (OS X for instance, maybe other BSD based have MAP_ANON instead of MAP_ANONYMOUS , consult man 2 mmap to find out the exact one).

If MAP_ANONYMOUS is not supported on your system, but /dev/zero is then an equivalent method is:

void * zeroed_memory;
int fd = open("/dev/zero", O_RDONLY);
if (fd > 0) {
    zeroed_memory = mmap(NULL, nbytes, PROT_READ, MAP_PRIVATE, fd, 0);
}

Answer 2

The write function will take whatever address you give it, and (attempt to) write nbyte bytes from that address onwards. As long as the memory is possible to read, this should work fine.

If you want to send more than a few kilobytes, it is often better to do that as multiple writes of a relatively small size (say 4KB), and use a loop to complete the write.