如何在python和节点进程之间共享mmap

Question

I'm trying to share memory between a python process and a nodejs process started from the python process using an anonymous mmap. Essentially, the python process begins, initializes the mmap and starts a subprocess using either call or Popen to launch a child that runs some node code. This nodejs code uses mmap to try to access the same area in memory. However I get two different mappings and no data is shared between them. Why is this?

import mmap, math, os
from subprocess import call

mm = mmap.mmap( -1, 1024,
                flags=mmap.MAP_SHARED | mmap.MAP_ANONYMOUS,
                prot= mmap.PROT_READ | mmap.PROT_WRITE )


mm.seek(0)
mm.write('hello world!\n'.encode('utf-8'))



call([
    'node', '-e',
    """
const mmap = require('mmap.js');

const fileBuf = mmap.alloc(
    1024,
    mmap.PROT_READ | mmap.PROT_WRITE,
    mmap.MAP_SHARED| mmap.MAP_ANONYMOUS,
    -1,
    0
)

console.log(fileBuf.toString('utf-8'));

    """ 
])

The mmap.js that I am using is a NAPI of the original mmap c function. This is the github for this library.

EDIT:

Thanks to 'that other guy' for his answer. It was the correct one. Here's some sample code that works out of the box!: test_mmap.py

import os, ctypes, posix_ipc, sys, mmap
from subprocess import call
SHARED_MEMORY_NAME = "/shared_memory"



memory = posix_ipc.SharedMemory(SHARED_MEMORY_NAME, posix_ipc.O_CREX,
                                size=1024)

mapFile = mmap.mmap(memory.fd, memory.size)
memory.close_fd()


mapFile.seek(0)
mapFile.write("Hello world!\n".encode('utf-8'))
mapFile.seek(0)

print("FROM PYTHON MAIN PROCESS: ", mapFile.readline().decode('utf-8'))
mapFile.seek(0)



call([
    "node", "./test_mmap.js", SHARED_MEMORY_NAME
])





mapFile.close()


posix_ipc.unlink_shared_memory(SHARED_MEMORY_NAME)

test_mmap.js

const args = process.argv;
const mmap = require('mmap.js');
const shm  = require('nodeshm');
const SHM_FILE_NAME=args[args.length-1];


let fd = shm.shm_open(SHM_FILE_NAME, shm.O_RDWR, 0600);
if (fd == -1){
    console.log("FD COULD NOT BE OPENED!");
    throw "here";
}



let mm = mmap.alloc(1024, mmap.PROT_READ | mmap.PROT_WRITE, mmap.MAP_SHARED, fd, 0);


console.log("FROM NODE: ", mm.slice(0, mm.indexOf('\n')).toString('utf-8'));

Sample output:

FROM PYTHON MAIN PROCESS:  Hello world!

FROM NODE:  Hello world!

Answer 1

Fortunately this doesn't work: imagine how confusing if all of the system's MAP_ANONYMOUS mappings were against the same area and kept overwriting each other.

Instead, use shm_open to create a new handle you can mmap in both processes. This is a portable wrapper around the equally valid but less portable strategy of creating and mmap'ing a file in /dev/shm/ .