RocksDB: iterator upper bound not working as expected

Question

I need to store a bunch of values in RocksDB (using v6.20.3) with the keys in the following format: PREFIX_<INTEGER_ID> . The INTEGER_ID is a running sequence of numbers from 0 to N .

I use a serialize_uint32_t function which converts the integer IDs to lexographic ordered strings (since I need to iterate the integers in order). Since N could be large, I wanted to iterate on the RocksDB entries until a smaller integer value K by setting the iterate_upper_bound property of the iterator. However, when I enable this property, the iteration does not happen.

Here's a small, fully reproduceable program below. When I comment the upper bound setting line out, the iterator works and the values are printed correctly.

UPDATE: Noticed another strange thing: the code snippet with upper bound works fine on Debug mode, but not on Release with -O2.

#include <iostream>
#include <rocksdb/db.h>
#include <rocksdb/options.h>

using namespace std;

static std::string serialize_uint32_t(uint32_t num) {
    unsigned char bytes[4];
    bytes[0] = (unsigned char) ((num >> 24) & 0xFF);
    bytes[1] = (unsigned char) ((num >> 16) & 0xFF);
    bytes[2] = (unsigned char) ((num >> 8) & 0xFF);
    bytes[3] = (unsigned char) ((num & 0xFF));

    return std::string(bytes, bytes+4);
}

int main() {
    const std::string state_dir_path = "/tmp/local-data";
    system("rm -rf /tmp/local-data && mkdir -p /tmp/local-data");

    rocksdb::DB *db;
    rocksdb::Options options;
    rocksdb::WriteOptions write_options;
    options.create_if_missing = true;

    rocksdb::Status s = rocksdb::DB::Open(options, state_dir_path, &db);
    if(!s.ok()) {
        std::cout << "Error while initializing store: " << s.ToString() << std::endl;
        return 1;
    }

    std::string key_prefix = "PREFIX_";

    for(size_t i = 0; i < 9; i++) {
        auto key_val = key_prefix + serialize_uint32_t(i);
        db->Put(write_options, key_val, "HELLO:"+std::to_string(i));
    }

    std::cout << "Wrote 9 entries." << std::endl;

    // Try to iterate only 6 entries
    rocksdb::ReadOptions read_opts;
    rocksdb::Slice upper_bound(key_prefix + serialize_uint32_t(6));
    read_opts.iterate_upper_bound = &upper_bound;   // does NOT work when set

    rocksdb::Iterator *iter = db->NewIterator(read_opts);
    iter->Seek(key_prefix + serialize_uint32_t(0));

    while(iter->Valid() && iter->key().starts_with(key_prefix)) {
        std::cout << iter->value().ToString() << std::endl;
        iter->Next();
    }

    delete iter;
    delete db;

    return 0;
}

Answer 1

I'm not sure whether it is the root cause but the code has a bug. rocksdb::Slice is similar to std::string_view. It points to a char array but doesn't maintain it. After upper_bound is constructed, key_prefix + serialize_uint32_t(6) might get destroyed, so read_opts.iterate_upper_bound points to a destroyed memory.

The argument of iter->Seek() might not be safe either, since Seek() also accepts rocksdb::Slice, rather than std::string.