简体   繁体   English

在磁盘上分配文件而不清零

[英]Allocate file on disk without zeroing

I need to allocate huge file without zeroing it's content. 我需要分配大文件而不将其内容清零。 I'm producing this steps fopen => ftruncate => fclose => mmap => (...work...) => munmap with huge file sizes (hundreds of gigabytes). 我正在执行以下步骤: fopen => ftruncate => fclose => mmap => (...work...) => munmap具有大文件大小(数百GB)。 App hangs on termination for a few minutes while system is trying zeroing file bytes – IMHO because of ftruncate usage. 系统尝试将文件字节清零时,应用程序会在终止时挂起几分钟-恕我直言,因为使用率ftruncate

ftruncate(ofd, 0);

#ifdef HAVE_FALLOCATE

    int ret = fallocate(ofd, 0, 0, cache_size);
    if (ret == -1) {
        printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
        exit(-1);
    }

#elif defined(HAVE_POSIX_FALLOCATE)

    int ret = posix_fallocate(ofd, 0, cache_size);
    if (ret == -1) {
        printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
        exit(-1);
    }

#elif defined(__APPLE__)

    fstore_t store = {F_ALLOCATECONTIG, F_PEOFPOSMODE, 0, cache_size, 0};
    int ret = fcntl(ofd, F_PREALLOCATE, &store);
    if (ret == -1) {
        store.fst_flags = F_ALLOCATEALL;
        ret = fcntl(ofd, F_PREALLOCATE, &store);
    }
    if (ret == -1) { // read fcntl docs - must test against -1
        printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
        exit(-1);
    }
    struct stat sb;
    ret = fstat(ofd, &sb);
    if (ret != 0) {
        printf("Failed to write to file to establish the size.\n");
        exit(-1);
    }
    //ftruncate(ofd, cache_size); <-- [1]

#endif

It seems it does not work with commented line [1] . 似乎在注释行[1]不起作用。 But uncommenting this line produces file zeroing which I am trying to avoid. 但是,取消注释此行会产生文件清零,而我试图避免这种情况。 I really don't care dirty file content before writing. 在写之前,我真的不在乎脏文件的内容。 I just wanna avoid hang on app termination. 我只是想避免挂断应用程序。

SOLUTION: 解:

According to @torfo 's answer , replaced all my Apple-related code with this few lines: 根据@torfo回答 ,用以下几行替换了我所有与Apple相关的代码:

unsigned long long result_size = cache_size;
int ret = fcntl(ofd, F_SETSIZE, &result_size);
if(ret == -1) {
    printf("Failed set size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
    exit(-1);
}

But only works for superuser! 但仅适用于超级用户!

This is MacOS X, apparently. 显然,这是MacOSX。

You can try to replace the ftruncate call with 您可以尝试将ftruncate呼叫替换为

fcntl(ofd, F_SETSIZE, &size);

(note needs root privilege and might create a security hole because it might provide access to old file contents that was previously there, so must be handled with extreme caution. The "dirty file content" you don't care about might actually be the user's bank account passwords he deleted a week ago...) (注意需要root特权,并且可能会创建一个安全漏洞,因为它可能提供对以前存在的旧文件内容的访问权限,因此必须格外小心。您不关心的“脏文件内容”实际上可能是用户的他一周前删除的银行帐户密码...)

MacOS X doesn't really support sparse files - It does create and maintain them, but its file system driver is very eager to fill the holes as soon as possible. MacOS X并不真正支持稀疏文件-它确实创建和维护它们,但是其文件系统驱动程序非常渴望尽快填补漏洞。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM