[英]How to prevent flushing to disk of a memory map opened on a windows temporary delete-on-close file
UPDATE 2 / TL;DR 更新2 / TL; DR
Is there some way to prevent dirty pages from a windows temporary delete-on-close file being flushed as a result of closing memory maps opened on these files.
是否有一些方法可以防止由于关闭在这些文件上打开的内存映射而刷新Windows临时删除关闭文件的脏页。
Yes. 是。 If you do not need to do anything with the files themselves after their initial creation and you implement some naming conventions, this is possible through the strategy explained in this answer .
如果您在初始创建后不需要对文件本身执行任何操作,并且实现了一些命名约定,则可以通过本答案中说明的策略实现。
Note: I am still quite interested in finding out the reasons for why there is so much difference in the behavior depending on how maps are created and the order of disposal / unmapping. 注意:我仍然非常有兴趣找出导致行为有这么大差异的原因,具体取决于地图的创建方式和处理/取消映射的顺序。
I have been looking into some strategies for an inter-process shared memory data structure that allows growing and shrinking its committed capacity on windows by using a chain of "memory chunks". 我一直在研究一些进程间共享内存数据结构的策略,它允许通过使用一系列“内存块”来增加和缩小它在Windows上的承诺容量。
One possible way is to use pagefile backed named memory maps as the chunk memory. 一种可能的方法是使用页面文件支持的命名内存映射作为块内存。 An advantage of this strategy is the possibility to use
SEC_RESERVE
to reserve a big chunk of memory address space and incrementally allocate it using VirtualAlloc
with MEM_COMMIT
. 此策略的一个优点是可以使用
SEC_RESERVE
保留大块内存地址空间,并使用VirtualAlloc
和MEM_COMMIT
逐步分配它。 Disadvantages appear to be (a) the requirement to have SeCreateGlobalPrivilege
permissions to allow using a shareable name in the Global\\
namespace and (b) the fact that all committed memory contributes to the system commit charge. 缺点似乎是(a)要求具有
SeCreateGlobalPrivilege
权限以允许在Global\\
namespace中使用可共享名称,以及(b)所有提交的内存都有助于系统提交费用。
To circumvent these disadvantages, I started investigating the use of temporary file backed memory maps . 为了克服这些缺点,我开始研究使用临时文件支持的内存映射 。 Ie memory maps over files created using the
FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY
即使用
FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY
创建的文件的内存映射 FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY
flags combination. FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY
标志组合。 This appears to be a recommended strategy that according to eg this blog post should prevent flushing the mapped memory to disk (unless memory pressure causes dirty mapped pages to be paged out). 这似乎是一种推荐的策略,根据例如此博客文章应该防止将映射的内存刷新到磁盘(除非内存压力导致脏映射页面被分页)。
I am however observing that closing the map/file handle before the owning process exits, causes dirty pages to be flushed to disk. 然而,我观察到在拥有进程退出之前关闭映射/文件句柄会导致脏页被刷新到磁盘。 This occurs even if the view/file handle is not the one through which the dirty pages were created and when these views/file handles were opened after the pages were 'dirtied' in a different view.
即使视图/文件句柄不是创建脏页面的那个句柄以及在不同视图中页面被“弄脏”之后打开这些视图/文件句柄时,也会发生这种情况。
It appears that changing the order of disposal (ie unmapping the view first or closing the file handle first) has some impact on when the disk flush is initiated, but not on the fact that flushing takes place. 似乎更改处理顺序(即首先取消映射视图或首先关闭文件句柄)对启动磁盘刷新的时间有一些影响,但不会影响发生刷新的事实。
So my questions are: 所以我的问题是:
- Is there some way to use temporary file backed memory maps and prevent them from flushing dirty pages when the map/file is closed, taking into account that multiple threads within a process/multiple processes may have open handles/views to such a file?
有没有办法使用临时文件支持的内存映射,并防止它们在关闭映射/文件时刷新脏页,考虑到进程/多个进程中的多个线程可能有这样一个文件的打开句柄/视图?
- If not, what is/could be the reason for the observed behavior?
如果不是,观察到的行为的原因是什么?
- Do you know of an alternative strategy that I may have overlooked?
你知道我可能忽略的另一种策略吗?
Refer to the (c++) sample code below that allows reproducing the problem on my system (x64, Win7): 请参阅下面的(c ++)示例代码,该代码允许在我的系统上重现问题(x64,Win7):
static uint64_t start_ts; static uint64_t elapsed() { return ::GetTickCount64() - start_ts; } class PageArena { public: typedef uint8_t* pointer; PageArena(int id, const char* base_name, size_t page_sz, size_t chunk_sz, size_t n_chunks, bool dispose_handle_first) : id_(id), base_name_(base_name), pg_sz_(page_sz), dispose_handle_first_(dispose_handle_first) { for (size_t i = 0; i < n_chunks; i++) chunks_.push_back(new Chunk(i, base_name_, chunk_sz, dispose_handle_first_)); } ~PageArena() { for (auto i = 0; i < chunks_.size(); ++i) { if (chunks_[i]) release_chunk(i); } std::cout << "[" << ::elapsed() << "] arena " << id_ << " destructed" << std::endl; } pointer alloc() { auto ptr = chunks_.back()->alloc(pg_sz_); if (!ptr) { chunks_.push_back(new Chunk(chunks_.size(), base_name_, chunks_.back()->capacity(), dispose_handle_first_)); ptr = chunks_.back()->alloc(pg_sz_); } return ptr; } size_t num_chunks() { return chunks_.size(); } void release_chunk(size_t ndx) { delete chunks_[ndx]; chunks_[ndx] = nullptr; std::cout << "[" << ::elapsed() << "] chunk " << ndx << " released from arena " << id_ << std::endl; } private: struct Chunk { public: Chunk(size_t ndx, const std::string& base_name, size_t size, bool dispose_handle_first) : map_ptr_(nullptr), tail_(nullptr), handle_(INVALID_HANDLE_VALUE), size_(0), dispose_handle_first_(dispose_handle_first) { name_ = name_for(base_name, ndx); if ((handle_ = create_temp_file(name_, size)) == INVALID_HANDLE_VALUE) handle_ = open_temp_file(name_, size); if (handle_ != INVALID_HANDLE_VALUE) { size_ = size; auto map_handle = ::CreateFileMappingA(handle_, nullptr, PAGE_READWRITE, 0, 0, nullptr); tail_ = map_ptr_ = (pointer)::MapViewOfFile(map_handle, FILE_MAP_ALL_ACCESS, 0, 0, size); ::CloseHandle(map_handle); // no longer needed. } } ~Chunk() { if (dispose_handle_first_) { close_file(); unmap_view(); } else { unmap_view(); close_file(); } } size_t capacity() const { return size_; } pointer alloc(size_t sz) { pointer result = nullptr; if (tail_ + sz <= map_ptr_ + size_) { result = tail_; tail_ += sz; } return result; } private: static const DWORD kReadWrite = GENERIC_READ | GENERIC_WRITE; static const DWORD kFileSharing = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE; static const DWORD kTempFlags = FILE_ATTRIBUTE_NOT_CONTENT_INDEXED | FILE_FLAG_DELETE_ON_CLOSE | FILE_ATTRIBUTE_TEMPORARY; static std::string name_for(const std::string& base_file_path, size_t ndx) { std::stringstream ss; ss << base_file_path << "." << ndx << ".chunk"; return ss.str(); } static HANDLE create_temp_file(const std::string& name, size_t& size) { auto h = CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, CREATE_NEW, kTempFlags, 0); if (h != INVALID_HANDLE_VALUE) { LARGE_INTEGER newpos; newpos.QuadPart = size; ::SetFilePointerEx(h, newpos, 0, FILE_BEGIN); ::SetEndOfFile(h); } return h; } static HANDLE open_temp_file(const std::string& name, size_t& size) { auto h = CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, OPEN_EXISTING, kTempFlags, 0); if (h != INVALID_HANDLE_VALUE) { LARGE_INTEGER sz; ::GetFileSizeEx(h, &sz); size = sz.QuadPart; } return h; } void close_file() { if (handle_ != INVALID_HANDLE_VALUE) { std::cout << "[" << ::elapsed() << "] " << name_ << " file handle closing" << std::endl; ::CloseHandle(handle_); std::cout << "[" << ::elapsed() << "] " << name_ << " file handle closed" << std::endl; } } void unmap_view() { if (map_ptr_) { std::cout << "[" << ::elapsed() << "] " << name_ << " view closing" << std::endl; ::UnmapViewOfFile(map_ptr_); std::cout << "[" << ::elapsed() << "] " << name_ << " view closed" << std::endl; } } HANDLE handle_; std::string name_; pointer map_ptr_; size_t size_; pointer tail_; bool dispose_handle_first_; }; int id_; size_t pg_sz_; std::string base_name_; std::vector<Chunk*> chunks_; bool dispose_handle_first_; }; static void TempFileMapping(bool dispose_handle_first) { const size_t chunk_size = 256 * 1024 * 1024; const size_t pg_size = 8192; const size_t n_pages = 100 * 1000; const char* base_path = "data/page_pool"; start_ts = ::GetTickCount64(); if (dispose_handle_first) std::cout << "Mapping with 2 arenas and closing file handles before unmapping views." << std::endl; else std::cout << "Mapping with 2 arenas and unmapping views before closing file handles." << std::endl; { std::cout << "[" << ::elapsed() << "] " << "allocating " << n_pages << " pages through arena 1." << std::endl; PageArena arena1(1, base_path, pg_size, chunk_size, 1, dispose_handle_first); for (size_t i = 0; i < n_pages; i++) { auto ptr = arena1.alloc(); memset(ptr, (i + 1) % 256, pg_size); // ensure pages are dirty. } std::cout << "[" << elapsed() << "] " << arena1.num_chunks() << " chunks created." << std::endl; { PageArena arena2(2, base_path, pg_size, chunk_size, arena1.num_chunks(), dispose_handle_first); std::cout << "[" << ::elapsed() << "] arena 2 loaded, going to release chunks 1 and 2 from arena 1" << std::endl; arena1.release_chunk(1); arena1.release_chunk(2); } } }
Please refer to this gist that contains the output of running the above code and links to screen captures of system free memory and disk activity when running TempFileMapping(false)
and TempFileMapping(true)
respectively. 请参阅此gist ,其中包含运行上述代码的输出,并分别在运行
TempFileMapping(false)
和TempFileMapping(true)
时链接到系统可用内存和磁盘活动的屏幕截图。
After the bounty period expired without any answers that provided more insight or solved the mentioned problem, I decided to dig a little deeper and experiment some more with several combinations and sequences of operations. 在赏金期结束后,没有任何答案提供更多洞察力或解决了上述问题,我决定深入挖掘一下,并通过几种组合和操作序列进行更多实验。
As a result, I believe I have found a way to achieve memory maps shared between processes over temporary, delete-on-close files, that are not being flushed to disk when they are closed. 因此,我相信我已经找到了一种方法来实现在进程之间通过临时的关闭时删除文件共享的内存映射,这些文件在关闭时不会刷新到磁盘。
The basic idea involves creating the memory map when a temp file is newly created with a map name that can be used in a call to OpenFileMapping
: 基本思想包括在新创建临时文件时使用可在
OpenFileMapping
调用中使用的映射名称创建内存映射:
// build a unique map name from the file name.
auto map_name = make_map_name(file_name);
// Open or create the mapped file.
auto mh = ::OpenFileMappingA(FILE_MAP_ALL_ACCESS, false, map_name.c_str());
if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
// existing map could not be opened, create the file.
auto fh = ::CreateFileA(name.c_str(), kReadWrite, kFileSharing, nullptr, CREATE_NEW, kTempFlags, 0);
if (fh != INVALID_HANDLE_VALUE) {
// set its size.
LARGE_INTEGER newpos;
newpos.QuadPart = desired_size;
::SetFilePointerEx(fh, newpos, 0, FILE_BEGIN);
::SetEndOfFile(fh);
// create the map
mh = ::CreateFileMappingA(mh, nullptr, PAGE_READWRITE, 0, 0, map_name.c_str());
// close the file handle
// from now on there will be no accesses using file handles.
::CloseHandle(fh);
}
}
Thus, the file handle is only used when the file is newly created, and closed immediately after the map is created, while the map handle itself remains open, to allow opening the mapping without requiring access to a file handle. 因此,文件句柄仅在新创建文件时使用,并在创建映射后立即关闭,而映射句柄本身保持打开状态,以允许打开映射而无需访问文件句柄。 Note that a race condition exists here, that we would need to deal with in any "real code" (as well as adding decent error checking and handling).
请注意,此处存在竞争条件,我们需要在任何“实际代码”中处理(以及添加适当的错误检查和处理)。
So if we got a valid map handle, we can create the view : 因此,如果我们获得了有效的地图句柄,我们就可以创建视图 :
auto map_ptr = MapViewOfFile(mh, FILE_MAP_ALL_ACCESS, 0, 0, 0);
if (map_ptr) {
// determine its size.
MEMORY_BASIC_INFORMATION mbi;
if (::VirtualQuery(map_ptr, &mbi, sizeof(MEMORY_BASIC_INFORMATION)) > 0)
map_size = mbi.RegionSize;
}
When, some time later closing a mapped file: close the map handle before unmapping the view: 以后关闭映射文件的时间:在取消映射视图之前关闭映射句柄:
if (mh == 0 || mh == INVALID_HANDLE_VALUE) {
::CloseHandle(mh);
mh = INVALID_HANDLE_VALUE;
}
if (map_ptr) {
::UnmapViewOfFile(map_ptr);
map_ptr = 0;
map_size = 0;
}
And, according to the test I have performed so far, this does not cause flushing dirty pages to disk on close, problem solved . 并且,根据我到目前为止所执行的测试,这不会导致在关闭时将脏页刷到磁盘, 问题已解决 。 Well partially anyway, there may still be a cross-session map name sharing issue.
无论如何,部分地,可能仍存在跨会话地图名称共享问题。
If I take it correctly, commenting out Arena2
part of code shall reproduce the issue without the need for second process. 如果我认为正确,注释掉
Arena2
部分代码将重现问题而无需第二个过程。 I have tried this: 我试过这个:
I edited base_path
as follows for convenience: 为方便起见,我按如下方式编辑了
base_path
:
char base_path[MAX_PATH]; GetTempPathA(MAX_PATH, base_path); strcat_s(base_path, MAX_PATH, "page_pool");
n_pages = 1536 * 128
to bring the used memory to 1.5GB, compared to your ~800mb. n_pages = 1536 * 128
,使用的内存为1.5GB,而你的内存为~800mb。 TempFileMapping(false)
and TempFileMapping(true)
, one at a time, for the same results. TempFileMapping(false)
和TempFileMapping(true)
,以获得相同的结果。 Arena2
commented out and intact, for the same results. Arena2
注释完整,但结果相同。 So, the question is, what are you observing? 所以,问题是,你在观察什么? I'd suggest that you:
我建议你:
Update I have tested on yet another Win7 x64, and times are 890ms full, 430ms spent on dealloc. 更新我已经测试了另一个Win7 x64,时间是890毫秒满,430分钟花在dealloc上。 I have looked into your results, and what is VERY suspicious is that almost exactly 4000ms is spent in freeze each time on your machine.
我已经查看了你的结果, 非常可疑的是,每次在你的机器上花费大约4000毫秒。 That can't be a mere coincidence, I believe.
我相信,这不仅仅是巧合。 Also, it's rather obvious now the the problem is somehow bound to a specific machine you're using.
此外,现在相当明显的问题是以某种方式绑定到您正在使用的特定机器上。 So my suggestions are:
所以我的建议是:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.