在 C++ 中寻找 Windows 基于 RAM 的共享 memory 解决方案

Question

I'm facing a situation where I need to pass up to several hundreds of megabytes of memory from one process to another. Right now I'm doing it via files and it's too slow. I guess that to make it quicker, those files should be written directly to RAM and be accessible from another process. No fancy synchronization required. One process would create shared memory objects and would fill them with data. The other process would read and remove them. However I've done a quick research and it seems like you can't share memory in RAM in Windows - the shared memory is backed by either a file or paging file. The docs of boost::interprocess confirm this. Where is the speed up then if the shared memory implementation still uses disk? Is there any C++ library that uses RAM-based shared memory?

EDIT: I made some further reading: 1. from boost::interprocess docs: "as the operating system has to synchronize the file contents with the memory contents, memory-mapped files are not as fast as shared memory. " 2. from http://msdn.microsoft.com/en-us/library/ms810613.aspx : "A memory-mapped file can also be mapped by more than one application simultaneously. This represents the only mechanism for two or more processes to directly share data in Windows NT."

Answer 1

I think that here is a fundamental misunderstanding: you think that, if you create a file mapping backed by the paging file, it will be as slow as actually writing stuff on disk.

This is definitely not the case: the meaning of "backed by the paging file" in the documentation means that the shared memory in general resides in memory, but it has a reserved place in the paging file to write such data if there's not enough free physical memory and the virtual memory manager needs to swap out memory pages.

This is not really clear from the documentation but the File Mapping page on MSDN confirms:

[...] It is backed by the file on disk. This means that when the system swaps out pages of the file mapping object , any changes made to the file mapping object are written to the file. When the pages of the file mapping object are swapped back in, they are restored from the file.

Notice that this applies to shared memory backed by the paging file as well as memory backed by regular files (the VMM guarantees that the various views are kept coherent).

Incidentally, this is how "regular" (=virtual) memory in user processes works: every bit of allocated memory can be swapped out to the paging file if it's not currently used and the system need to use physical memory for other stuff (eg making memory pages that are used at the moment available to your/another application).

Answer 2

There's nothing wrong with being backed by a file -- under memory pressure, the data has to go somewhere, and your choices are:

• treat the memory as sacred data that cannot be paged or dropped

  Likely to only create much worse memory pressure problems, but good choice for some embedded systems where the entire runtime environment of the system can be very well controlled.

• drop the memory

  Obviously not suitable to all data. Cached contents? Maybe. Original photos? Probably not.

• page the memory to disk

  Good generic choice!

Are you seeing memory pressure when using a shared-memory tool? Add more RAM or figure out how to shrink your systems. :)

Answer 3

As far as I know you have essentially 2 Options here.

1)You create a DLL and use the data_seg pragma and load the DLL in both of your processes. This has HUGE drawbacks which are explained in detail here: http://msdn.microsoft.com/en-us/library/h90dkhs0(v=vs.80).aspx

The most important drawbacks are: The space has to be initialized statically and are stored in the data segment of the compiled DLL, meaning that if you want to share hundreds of MBs using this method then your DLL is going to be hundreds of MBs big.

2)There's nothing wrong with using regular Memory-mapped files though as they are cached anyways. You can even use the Systems pagefile to store the data as described in this article: http://msdn.microsoft.com/en-us/library/ms810613.aspx

I actually tested this example [1] of Inter-process communication With a 1 GiB memory-mapped file and can confirm that nothing was written to disk even after filling the whole GiB with data.

[1] http://msdn.microsoft.com/en-us/library/aa366551(v=vs.85).aspx

Answer 4

I would still suggested memory mapped files instead of the approach I will mention.

If you really want to read from another process memory , use Win32 API ReadProcessMemory() .

If you are paranoid on keeping data in RAM, there are Unix mlock() equivalents in MS Windows VirtualLock()