MapViewOfFile返回错误5（ERROR_ACCESS_DENIED）

Question

I'm trying to develop a system that uses shared memory for communication between two processes. One process is a 32 bit application and uses a 32bit dll for the communication. The other process is a 64 bit application and uses a 64 bit dll which contains the exact same code for creating/opening the shared memory. I made it so whenever one process opens a handle to the memory and cannot open it, it automatically tries to create the memory. The other process will then try the same so whichever process runs the code first will create the memory while the other process will open a handle to the already existing memory.

The code works absolutely fine when the shared memory is created by the 64 bit process/dll but whenever the 32 bit dll creates it, I get error 5 (ERROR_ACCESS_DENIED) returned when calling MapViewOfFile.

I already checked if the size that I pass to any of the functions is different, for example because one of the structures has a different size depending on if it was compiled for 32 bit or 64 bit. However, this is not the case, the size is always the same in both processes.

I also tried the code that was suggested in this answer without success. Does anyone have an idea why the code sometimes fails with error 5?

Here is my code:

static LPVOID lpMappedInputData = nullptr,
    lpMappedOutputData = nullptr;

static HANDLE hInputFileMapping = NULL,
    hOutputFileMapping = NULL;

HANDLE createOrOpenFileMapping(DWORD size, LPCSTR lpName)
{
    HANDLE hMapFile = OpenFileMappingA(FILE_MAP_READ | FILE_MAP_WRITE, FALSE, lpName);

    if (!hMapFile)
        DEBUG_LOG("OpenFileMapping failed! Error code: %i - Attempting to create the file mapping instead...\n", GetLastError());

    if (!hMapFile)
    {
        hMapFile = CreateFileMappingA(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, size, lpName);

        if (!hMapFile)
            DEBUG_LOG("CreateFileMapping failed! Error code: %i\n", GetLastError());
    }

    return hMapFile;
}

LPVOID mapViewOfFile(HANDLE hFileMappingObject, DWORD size)
{
    LPVOID lpMappedData = MapViewOfFile(hFileMappingObject, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, size);

    if (!lpMappedData)
    {
        CloseHandle(hFileMappingObject);
        DEBUG_LOG("MapViewOfFile failed! Error code: %i\n", GetLastError());
    }

    return lpMappedData;
}

bool Initialize()
{
    DEBUG_LOG("Initializing the input file mapping...\n");

    hInputFileMapping = createOrOpenFileMapping(sizeof(InputData), "Local\\InputData");

    if (!hInputFileMapping)
        return false;

    lpMappedInputData = mapViewOfFile(hInputFileMapping, sizeof(InputData));

    if (!lpMappedInputData)
        return false;

    DEBUG_LOG("Initializing the output file mapping...\n");

    hOutputFileMapping = createOrOpenFileMapping(sizeof(OutputData), "Local\\OutputData");

    if (!hOutputFileMapping)
        return false;

    lpMappedOutputData = mapViewOfFile(hOutputFileMapping, sizeof(OutputData));

    if (!lpMappedOutputData)
        return false;

    return true;
}

void Deinitialize()
{
    DEBUG_LOG("Deinitializing the file mappings...\n");

    if (lpMappedInputData)
        UnmapViewOfFile(lpMappedInputData);

    if (hInputFileMapping)
        CloseHandle(hInputFileMapping);

    if (lpMappedOutputData)
        UnmapViewOfFile(lpMappedOutputData);

    if (hOutputFileMapping)
        CloseHandle(hOutputFileMapping);

    DEBUG_LOG("Successfully deinitialized the file mappings!\n");
}

Answer 1

win32 errors - this is in most cast translated from original NTSTATUS native api call result. but this translation is not injective - sometimes many different status converted to the same win32 error value. for example to the same ERROR_ACCESS_DENIED converted at least 18(!) different ntstatus. so original error can be not STATUS_ACCESS_DENIED but absolute another, having nothing common to real access denied. because this, if win32 api call is shell over native call - always better call RtlGetLastNtStatus for get original error status code or call native api direct ( ZwMapViewOfSection instead MapViewOfFile )

unfortunately I don't know original status returned by ZwMapViewOfSection , but I guess that this is STATUS_INVALID_VIEW_SIZE ( An attempt was made to create a view for a section which is bigger than the section. ).

in create and map section used sizeof(InputData) and sizeof(OutputData) - no definitions of this structures in code, bat based on

The code works absolutely fine when the shared memory is created by the 64 bit process/dll but whenever the 32 bit dll creates it, I get error 5 ( ERROR_ACCESS_DENIED ) returned when calling MapViewOfFile .

I guess next - in 32bit code InputData or OutputData have smaller size )compare 64 bit code). this just explain all. if 64 bit code first create section and then 32 bit code try map smaller size - this is ok. but if 32 bit code first create section and then 64 bit code try map size bigger that section size - we got STATUS_INVALID_VIEW_SIZE which translated to ERROR_ACCESS_DENIED

but how about

I already checked if the size .. the size is always the same in both processes.

not believe. I be recheck.

as separate note - the CreateFileMapping

Creates or opens a named or unnamed file mapping object for a specified file.

so we not need special function createOrOpenFileMapping and call OpenFileMappingA first - just call CreateFileMappingW (for what use A api ?). and any way code of createOrOpenFileMapping have raise condition - both processes can call OpenFileMappingA in concurrent and both fail here. that both anyway call CreateFileMappingA

but main GetLastError() some times not the best and ERROR_ACCESS_DENIED not always is STATUS_ACCESS_DENIED and related to real access denied error