Memory std::condition_variable 的位置可能导致 futex 错误

Question

We had a bug in our software that ended in the dreaded:

The futex facility returned an unexpected error code.

We traced it down to a problem where the location of std::condition_variable within a malloc'd region of memory causes a futex error. If the std::condition_variable is not aligned on a 16 byte word - then it causes the futex error when you try to wait . In the example the first two wait_for calls work, but the last one aborts the program with the futex error.

void futex_error()
{
    /* init */
    std::mutex mtx;

    /* Normal one works  */
    std::cout << "Doing normal" << "\n";
    std::condition_variable* con_var = (std::condition_variable*)malloc(sizeof(std::condition_variable));
    new (con_var) std::condition_variable{};

    {
        std::unique_lock<std::mutex> lck(mtx);
        con_var->wait_for(lck, std::chrono::seconds(1));
    }

    /* Clean */
    con_var->std::condition_variable::~condition_variable();
    free(con_var);

    std::cout << "Doing 16 bytes" << "\n";
    /* Works on 16 byte alignment  */
    uint8_t* ptr_16 = (uint8_t*)malloc(sizeof(std::condition_variable) + 16);
    std::condition_variable* con_var_16 = new (ptr_16 + 16) std::condition_variable{};

    {
        std::unique_lock<std::mutex> lck(mtx);
        con_var_16->wait_for(lck, std::chrono::seconds(1));
    }

    /* Clean */
    con_var_16->std::condition_variable::~condition_variable();
    free(ptr_16);

    std::cout << "Doing 1 byte" << "\n";
    /* Futex error */
    uint8_t* bad_ptr = (uint8_t*)malloc(sizeof(std::condition_variable) + 1);
    std::condition_variable* bad = new (bad_ptr + 1) std::condition_variable{};

    {
        std::unique_lock<std::mutex> lck(mtx);
        bad->wait_for(lck, std::chrono::seconds(1)); //<--- error here?
    }

    /* Clean */
    bad->std::condition_variable::~condition_variable();
    free(con_var);
}

I cant seem to find documentation on futex errors and why the alignment would cause this. Does anyone know why this would occur? This is on linux (Arch and Ubuntu) whilst using gcc 9.3.

Answer 1

why the alignment would cause this

From C++ draft Alignment p1 :

Object types have alignment requirements ([basic.fundamental], [basic.compound]) which place restrictions on the addresses at which an object of that type may be allocated.

The expression:

new (bad_ptr + 1) std::condition_variable{};

invokes undefined behavior on systems where bad_ptr + 1 is not aligned to alignof(std::condition_variable) . Testing on godbolt with gcc10 the alignof(std::confition_variable) is equal to 8 .

Both bad-> accesses are unaligned accesses and both are undefined behavior.

Does anyone know why this would occur?

Inspecting strace output on execution of the executable, we can see that:

futex(0x557da3e262e9, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=2439, tv_nsec=619296657}, FUTEX_BITSET_MATCH_ANY) = -1 EINVAL (Invalid argument)

Because uaddr first argument which should be a pointer to int of futex call is not aligned to _Alignof(int) , kernel detects it here and futex return EINVAL . The standard library just exits the application then, which is a perfectly fine behavior for undefined behavior.

Answer 2

Found out the source of my version of this "The futex facility returned an unexpected error code." error message.

It was simply caused by #pragma pack(1) without returning to normal packing in some other unrelated source code. Then std::condition_variable was initialized and it errored on use.