C++ unordered_map SIGSEGV on find() or insert()

Question

Currently I am trying to implement some kind of basic File-Access layer for a Database System we need to build for University. To do this I am asked to cache pages in Memory. To quickly decide which pages are loaded yet, I store all of them in an unordered_map.

Now the Problem is, that after some calls to this function a SIGSEGV appears within _M_find_before_node of the unordered_map. I tried debugging it using gdb, but I have no clue where to look for it, as the error happens on the call to the unordered_map:

#0  0x0000000000402ab0 in std::_Hashtable<unsigned long, std::pair<unsigned long const, BufferFrame>, std::allocator<std::pair<unsigned long const, BufferFrame> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_find_before_node (this=0x608038, __n=2, __k=@0x7fffffffe050: 2, __code=2) at /usr/include/c++/4.9/bits/hashtable.h:1442
#1  0x0000000000402286 in std::_Hashtable<unsigned long, std::pair<unsigned long const, BufferFrame>, std::allocator<std::pair<unsigned long const, BufferFrame> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_find_node (this=0x608038, __bkt=2, __key=@0x7fffffffe050: 2, __c=2) at /usr/include/c++/4.9/bits/hashtable.h:625
#2  0x0000000000401cd6 in std::_Hashtable<unsigned long, std::pair<unsigned long const, BufferFrame>, std::allocator<std::pair<unsigned long const, BufferFrame> >, std::__detail::_Select1st, std::equal_to<unsigned long>, std::hash<unsigned long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find (this=0x608038, __k=@0x7fffffffe050: 2) at /usr/include/c++/4.9/bits/hashtable.h:1304
#3  0x000000000040198f in std::unordered_map<unsigned long, BufferFrame, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<std::pair<unsigned long const, BufferFrame> > >::find (
this=0x608038, __x=@0x7fffffffe050: 2) at /usr/include/c++/4.9/bits/unordered_map.h:574
#4  0x00000000004013dd in BufferManager::fixPage (this=0x608030, pageId=2, exclusive=true) at BufferManager.cpp:36
#5  0x0000000000404268 in main (argc=4, argv=0x7fffffffe2c8) at buffertest.cpp:87

I also tried to debug it using valgrind, as i thought about I might get a unnoticed memory corrupution at some point in time before the error appears. This gave me the following output:

==27031== Memcheck, a memory error detector
==27031== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==27031== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==27031== Command: ./buffer-test 11 3 1
==27031== 
==27031== Invalid write of size 8
==27031==    at 0x401642: std::__detail::_Hash_node_base::_Hash_node_base() (hashtable_policy.h:234)
==27031==    by 0x4020A0: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::__detail::_Mod_range_hashing const&, std::__detail::_Default_ranged_hash const&, std::equal_to<std::string> const&, std::__detail::_Select1st const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:799)
==27031==    by 0x401BB5: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:420)
==27031==    by 0x4018DC: std::unordered_map<std::string, int, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, int> > >::unordered_map(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (unordered_map.h:143)
==27031==    by 0x401257: BufferManager::BufferManager(unsigned int) (BufferManager.cpp:21)
==27031==    by 0x4041B6: main (buffertest.cpp:79)
==27031==  Address 0x5c1d0e0 is 16 bytes after a block of size 64 alloc'd
==27031==    at 0x4C29180: operator new(unsigned long) (vg_replace_malloc.c:324)
==27031==    by 0x4041A3: main (buffertest.cpp:79)
==27031== 
==27031== Invalid write of size 8
==27031==    at 0x4020A5: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::__detail::_Mod_range_hashing const&, std::__detail::_Default_ranged_hash const&, std::equal_to<std::string> const&, std::__detail::_Select1st const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:799)
==27031==    by 0x401BB5: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:420)
==27031==    by 0x4018DC: std::unordered_map<std::string, int, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, int> > >::unordered_map(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (unordered_map.h:143)
==27031==    by 0x401257: BufferManager::BufferManager(unsigned int) (BufferManager.cpp:21)
==27031==    by 0x4041B6: main (buffertest.cpp:79)
==27031==  Address 0x5c1d0e8 is 24 bytes after a block of size 64 in arena "client"
==27031== 

valgrind: m_mallocfree.c:304 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 128, hi = 0.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==27031==    at 0x380A48EF: show_sched_status_wrk (m_libcassert.c:319)
==27031==    by 0x380A49E4: report_and_quit (m_libcassert.c:390)
==27031==    by 0x380A4B66: vgPlain_assert_fail (m_libcassert.c:455)
==27031==    by 0x380B170D: get_bszB_as_is (m_mallocfree.c:302)
==27031==    by 0x380B170D: get_bszB (m_mallocfree.c:312)
==27031==    by 0x380B170D: get_pszB (m_mallocfree.c:386)
==27031==    by 0x380B170D: vgPlain_describe_arena_addr (m_mallocfree.c:1532)
==27031==    by 0x3809DC93: vgPlain_describe_addr (m_addrinfo.c:188)
==27031==    by 0x3809C73B: vgMemCheck_update_Error_extra (mc_errors.c:1133)
==27031==    by 0x380A05BA: vgPlain_maybe_record_error (m_errormgr.c:818)
==27031==    by 0x3809BCB2: vgMemCheck_record_address_error (mc_errors.c:753)
==27031==    by 0x803EDF459: ???
==27031==    by 0x802D95EEF: ???
==27031==    by 0x3807295F: vgMemCheck_malloc (mc_malloc_wrappers.c:388)
==27031==    by 0x4020A0: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::__detail::_Mod_range_hashing const&, std::__detail::_Default_ranged_hash const&, std::equal_to<std::string> const&, std::__detail::_Select1st const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:799)

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable
==27031==    at 0x401682: std::__detail::_Prime_rehash_policy::_Prime_rehash_policy(float) (hashtable_policy.h:464)
==27031==    by 0x4020C4: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::__detail::_Mod_range_hashing const&, std::__detail::_Default_ranged_hash const&, std::equal_to<std::string> const&, std::__detail::_Select1st const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:799)
==27031==    by 0x401BB5: std::_Hashtable<std::string, std::pair<std::string const, int>, std::allocator<std::pair<std::string const, int> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Hashtable(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (hashtable.h:420)
==27031==    by 0x4018DC: std::unordered_map<std::string, int, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, int> > >::unordered_map(unsigned long, std::hash<std::string> const&, std::equal_to<std::string> const&, std::allocator<std::pair<std::string const, int> > const&) (unordered_map.h:143)
==27031==    by 0x401257: BufferManager::BufferManager(unsigned int) (BufferManager.cpp:21)
==27031==    by 0x4041B6: main (buffertest.cpp:79)


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

Sadly I could not figure out what these errors want to tell me. The code where they happen is not even doing anything with this unordered_map, it is simply the constructor.

I would appreciate any help or idea how to fix this issue. Below are the sources of my code (the relevant lines). If you need any more information or want me to run any test, I will be happy to give it to you.

BufferManager.hpp (updated)

#ifndef BUFFER_MANAGER_HPP
#include "BufferFrame.hpp"
#include <stdint.h>
#include <unordered_map>

class BufferManager {

public:
    BufferManager( const unsigned int pageCount );
    BufferFrame& fixPage( const uint64_t pageId, const bool exclusive );
    void unfixPage( BufferFrame& frame, const bool isDirty );
    ~BufferManager();

private:
    uint64_t pageCount;
    std::unordered_map<uint64_t, BufferFrame *> pages;
    std::unordered_map<std::string, int> files;
};


#define BUFFER_MANAGER_HPP
#endif

BufferManager.cpp (parts) (updated):

BufferManager::BufferManager( const unsigned int pageCount ) {
    this->pageCount = pageCount;
    std::cout << "BufferFrame constructor" << std::endl;
}

BufferFrame& BufferManager::fixPage( const uint64_t pageId, const bool exclusive ) {
    std::cout << "fixPage called with pageId: " << pageId << " requesting exclusive: " << exclusive<< std::endl;

    std::cout << "Pages.size(): " << pages.size() << std::endl;

    //Test if we got the frame in the buffer
    BufferFrame * frame;
    std::unordered_map<uint64_t, BufferFrame *>::const_iterator got = pages.find(pageId);
    if (got == pages.end()) {
        frame = new BufferFrame();
        frame->data = malloc(pageSize);
        frame->pageId = pageId;
        frame->isDirty = false;

        std::pair<uint64_t, BufferFrame *> element(pageId, frame);
        pages.insert(element);
        std::cout << "fixPage of page: " << pageId << " (empty page used. Load not implemented!)" << std::endl;
    }
    else {
        frame = got->second;
        std::cout << "fixPage of page: " << pageId << " (Page loaded from map)" << std::endl;
    }
    return *frame;
}

BufferFrame.hpp:

#ifndef BUFFER_FRAME_HPP
#include <stdint.h>

class BufferFrame {
    friend class BufferManager;

public:

    void* getData();

private:

    void * data;
    uint64_t pageId;
    bool isDirty;
};



#define BUFFER_FRAME_HPP
#endif

What I tried so far is replacing the BufferFrame within the unordered_map by a BufferFrame*, which did not change anything. Even replacing it with an int did not seem to have any effect.

EDIT: updated BufferManager.hpp and BufferManager.cpp to use a unordered_map.

EDIT2: First of all: Thank you to all, that tried to help me

I found the problem: It was a bug in the Makefile that caused one of my .o files not to be regenerated. This lead to different files using a different version of the BufferFrame.hpp. This obviously must lead to bizzarre memory errors (like the ones I got).

Answer 1

It seems the code is returning reference to a local variable.

BufferFrame& BufferManager::fixPage(
    ....
    BufferFrame frame;
    ....   
    return frame;

You may want to change your design to

std::unordered_map<uint64_t, BufferFrame *> pages;

or even

std::unordered_map<uint64_t, std::shared_ptr<BufferFrame> > pages;

I would also suggest to create BufferFrame as a proper C++ class with a constructor and destructor so that it manages its own memory.

Answer 2

Please look at these two aspects closely:

• Where is 'pageSize' set and to what value? I can't see any.
• Do you really need to mix 'new' and malloc, while allocating 'frame'.
• You didn't show how do you clean-up BufferFrame objects after usage.
• If you are doing new+free or malloc+delete, you are in trouble.

I'd go with a ctor and dtor for BufferFrame as Arun has suggested.