简体繁体 English

创建大于 2GB 的文件

[英]How can I write/create a file larger than 2GB by using C/C++

原文 2012-04-06 10:23:01 9 3 c++/ c/ large-files

I tried to use write() function to write a large piece of memory into a file (more than 2GB) but never succeed.我尝试使用 write() function 将一大块 memory 写入文件（超过 2GB）但从未成功。 Can somebody be nice and tell me what to do?有人可以告诉我该怎么做吗？

3 个解决方案

Assuming Linux:)假设 Linux:)

https://users.suse.com/~aj/linux_lfs.html https://users.suse.com/~aj/linux_lfs.html

Define _FILE_OFFSET_BITS to 64. ( gcc -D_FILE_OFFSET_BITS=64 )将_FILE_OFFSET_BITS定义为 64。 ( gcc -D_FILE_OFFSET_BITS=64 )
Define _LARGEFILE_SOURCE and _LARGEFILE_SOURCE64 .定义_LARGEFILE_SOURCE和_LARGEFILE_SOURCE64 。
Use the O_LARGEFILE flag with open to operate on large file使用带有 open 的O_LARGEFILE标志对大文件进行操作

Also some information there: http://www.gnu.org/software/libc/manual/html_node/Opening-Streams.html#index-fopen64-931还有一些信息： http://www.gnu.org/software/libc/manual/html_node/Opening-Streams.html#index-fopen64-931

These days the file systems you have on your system will support large file out of the box.如今，您系统上的文件系统将支持开箱即用的大文件。

Add -D_FILE_OFFSET_BITS=64 to your compiler command line (aka CFLAGS ) and it will work.将-D_FILE_OFFSET_BITS=64添加到您的编译器命令行（又名CFLAGS ），它将起作用。 This is not necessary on any 64-bit system and it's also unnecessary on some (but not all) 32-bit systems these days.这在任何 64 位系统上都不是必需的，而且现在在某些（但不是全部）32 位系统上也没有必要。

Avoid advice to go writing O_LARGEFILE or open64 etc. all over your source.避免向 go 建议在整个源代码中写入O_LARGEFILE或open64等。 This is non-portable and, quite simply, ugly.这是不可移植的，而且很简单，很丑。

Edit: Actually, I think we've all misread your issue.编辑：实际上，我认为我们都误读了您的问题。 If you're on a 32-bit system, the largest possible single object in memory is 2GB-1byte (ie SSIZE_MAX ).如果您使用的是 32 位系统，则 memory 中最大可能的单个 object 是 2GB-1byte（即SSIZE_MAX ）。 Per POSIX, the behavior of write for size arguments that cannot fit in ssize_t is not defined;根据 POSIX，未定义无法放入 ssize_t 的大小为ssize_t的write行为； this is because write returns type ssize_t and could not represent the number of bytes written if it were larger.这是因为write返回类型ssize_t并且如果它更大则不能表示写入的字节数。 Perhaps more importantly, it's dangerous for an implementation to allow objects so large that their size does not fit in ptrdiff_t , since that would create a situation where pointer arithmetic could invoke integer overflow and thus undefined behavior.或许更重要的是，实现允许对象太大以至于它们的大小不适合ptrdiff_t是危险的，因为这会造成指针算法可能调用 integer 溢出并因此导致未定义行为的情况。 You may have been able to create such a large object with mmap (I would consider this a bug in your system's kernel or libc), but it's a very bad idea and will lead to all sorts of bugs.您可能已经能够使用mmap创建如此大的 object（我认为这是您系统的 kernel 或 libc 中的错误），但这是一个非常糟糕的主意，并且会导致各种错误。 If you need single objects larger than 2GB, you really need to run on a 64-bit machine to have it be safe and bug-free.如果您需要大于 2GB 的单个对象，您确实需要在 64 位机器上运行以确保安全且无错误。

It depends upon the operating system, the processor, the file system.它取决于操作系统、处理器和文件系统。 On Linux/x86-64 systems (with ext3 file systems) it is trivial to do.在 Linux/x86-64 系统（带有 ext3 文件系统）上，这很容易做到。 Just use the usual library functions (in C++ std::ofstream , in C <stdio.h> , fopen && fprintf etc.) or the underlying system calls (open, write).只需使用常用的库函数（在 C++ std::ofstream中，在 C <stdio.h>中， fopen && fprintf等）或底层系统调用（打开、写入）。