Determine member offset of struct or tuple in template

Question

I want to write a template function that writes tables to HDF5 files. The signature should look similar to

template<typename record> void writeTable(const std::vector<record>& data);

where record is a struct, or

template<typename... elements> 
    void writeTable(const std::vector<std::tuple<elements...>>& data);

The actual implementation would have more parameters to determine the destionation, etc.

To write the data I need to define a HDF5 compound type, which contains the name and the offset of the members. Usually you would use the HOFFSET macro the get the field offset, but as I don't know the struct fields beforehand I can't do that.

What I tried so far was constructing a struct type from the typename pack. The naive implementation did not have standard layout, but the implementation here does. All that's left is get the offsets of the members. I would like to expand the parameter pack into an initializer list with the offsets:

#include <vector>

template<typename... members> struct record {};

template<typename member, typename... members> struct record<member, members...> : 
    record<members...> {
  record(member m, members... ms) : record<members...>(ms...), tail(m) {}
  member tail;
};

template<typename... Args> void 
    make_table(const std::string& name, const std::vector<record<Args...>>& data) {
  using record_type = record<Args...>;
  std::vector<size_t> offsets = { get_offset(record_type,Args)... };
}

int main() {
  std::vector<record<int, float>> table = { {1, 1.0}, {2, 2.0} };
  make_table("table", table);
}

Is there a possible implementation for get_offset ? I would think not, because in the case of record<int, int> it would be ambiguous. Is there another way to do it?

Or is there any other way I could approach this problem?

Answer 1

Calculating offsets is quite simple. Given a tuple with types T0, T1 ... TN. The offset of T0 is 0 (as long as you use alignas(T0) on your char array. The offset of T1 is the sizeof(T0) rounded up to alignof(T1) .

In general, the offset of TB (which comes after TA ) is round_up(offset_of<TA>() + sizeof(TA), alignof(TB)) .

Calculating the offsets of elements in a std::tuple could be done like this:

constexpr size_t roundup(size_t num, size_t multiple) {
  const size_t mod = num % multiple;
  return mod == 0 ? num : num + multiple - mod;
}

template <size_t I, typename Tuple>
struct offset_of {
  static constexpr size_t value = roundup(
    offset_of<I - 1, Tuple>::value + sizeof(std::tuple_element_t<I - 1, Tuple>),
    alignof(std::tuple_element_t<I, Tuple>)
  );
};

template <typename Tuple>
struct offset_of<0, Tuple> {
  static constexpr size_t value = 0;
};

template <size_t I, typename Tuple>
constexpr size_t offset_of_v = offset_of<I, Tuple>::value;

Here's a test suite. As you can see from the first test, the alignment of elements is taken into account.

static_assert(offset_of_v<1, std::tuple<char, long double>> == 16);
static_assert(offset_of_v<2, std::tuple<char, char, long double>> == 16);
static_assert(offset_of_v<3, std::tuple<char, char, char, long double>> == 16);
static_assert(offset_of_v<4, std::tuple<char, char, char, char, long double>> == 16);

static_assert(offset_of_v<0, std::tuple<int, double, int, char, short, long double>> == 0);
static_assert(offset_of_v<1, std::tuple<int, double, int, char, short, long double>> == 8);
static_assert(offset_of_v<2, std::tuple<int, double, int, char, short, long double>> == 16);
static_assert(offset_of_v<3, std::tuple<int, double, int, char, short, long double>> == 20);
static_assert(offset_of_v<4, std::tuple<int, double, int, char, short, long double>> == 22);
static_assert(offset_of_v<5, std::tuple<int, double, int, char, short, long double>> == 32);

I hardcoded the offsets in the above tests. The offsets are correct if the following tests succeed.

static_assert(sizeof(char) == 1 && alignof(char) == 1);
static_assert(sizeof(short) == 2 && alignof(short) == 2);
static_assert(sizeof(int) == 4 && alignof(int) == 4);
static_assert(sizeof(double) == 8 && alignof(double) == 8);
static_assert(sizeof(long double) == 16 && alignof(long double) == 16);

std::tuple seems to store it's elements sequentially (without sorting them to optimize padding). That's proven by the following tests. I don't think the standard requires std::tuple to be implemented this way so I don't think the following tests are guaranteed to succeed.

template <size_t I, typename Tuple>
size_t real_offset(const Tuple &tup) {
  const char *base = reinterpret_cast<const char *>(&tup);
  return reinterpret_cast<const char *>(&std::get<I>(tup)) - base;
}

int main(int argc, char **argv) {
  using Tuple = std::tuple<int, double, int, char, short, long double>;
  Tuple tup;
  assert((offset_of_v<0, Tuple> == real_offset<0>(tup)));
  assert((offset_of_v<1, Tuple> == real_offset<1>(tup)));
  assert((offset_of_v<2, Tuple> == real_offset<2>(tup)));
  assert((offset_of_v<3, Tuple> == real_offset<3>(tup)));
  assert((offset_of_v<4, Tuple> == real_offset<4>(tup)));
  assert((offset_of_v<5, Tuple> == real_offset<5>(tup)));
}

Now that I've gone to all of this effort, would that real_offset function suit your needs?

---

This is a minimal implementation of a tuple that accesses a char[] with offset_of . This is undefined behavior though because of the reinterpret_cast . Even though I'm constructing the object in the same bytes and accessing the object in the same bytes, it's still UB. See this answer for all the standardese. It will work on every compiler you can find but it's UB so just use it anyway. This tuple is standard layout (unlike std::tuple ). If the elements of your tuple are all trivially copyable, you can remove the copy and move constructors and replace them with memcpy .

template <typename... Elems>
class tuple;

template <size_t I, typename Tuple>
struct tuple_element;

template <size_t I, typename... Elems>
struct tuple_element<I, tuple<Elems...>> {
  using type = std::tuple_element_t<I, std::tuple<Elems...>>;
};

template <size_t I, typename Tuple>
using tuple_element_t = typename tuple_element<I, Tuple>::type;

template <typename Tuple>
struct tuple_size;

template <typename... Elems>
struct tuple_size<tuple<Elems...>> {
  static constexpr size_t value = sizeof...(Elems);
};

template <typename Tuple>
constexpr size_t tuple_size_v = tuple_size<Tuple>::value;

constexpr size_t roundup(size_t num, size_t multiple) {
  const size_t mod = num % multiple;
  return mod == 0 ? num : num + multiple - mod;
}

template <size_t I, typename Tuple>
struct offset_of {
  static constexpr size_t value = roundup(
    offset_of<I - 1, Tuple>::value + sizeof(tuple_element_t<I - 1, Tuple>),
    alignof(tuple_element_t<I, Tuple>)
  );
};

template <typename Tuple>
struct offset_of<0, Tuple> {
  static constexpr size_t value = 0;
};

template <size_t I, typename Tuple>
constexpr size_t offset_of_v = offset_of<I, Tuple>::value;

template <size_t I, typename Tuple>
auto &get(Tuple &tuple) noexcept {
  return *reinterpret_cast<tuple_element_t<I, Tuple> *>(tuple.template addr<I>());
}

template <size_t I, typename Tuple>
const auto &get(const Tuple &tuple) noexcept {
  return *reinterpret_cast<tuple_element_t<I, Tuple> *>(tuple.template addr<I>());
}

template <typename... Elems>
class tuple {
  alignas(tuple_element_t<0, tuple>) char storage[offset_of_v<sizeof...(Elems), tuple<Elems..., char>>];
  using idx_seq = std::make_index_sequence<sizeof...(Elems)>;

  template <size_t I>
  void *addr() {
    return static_cast<void *>(&storage + offset_of_v<I, tuple>);
  }

  template <size_t I, typename Tuple>
  friend auto &get(const Tuple &) noexcept;

  template <size_t I, typename Tuple>
  friend const auto &get(Tuple &) noexcept;

  template <size_t... I>
  void default_construct(std::index_sequence<I...>) {
    (new (addr<I>()) Elems{}, ...);
  }
  template <size_t... I>
  void destroy(std::index_sequence<I...>) {
    (get<I>(*this).~Elems(), ...);
  }
  template <size_t... I>
  void move_construct(tuple &&other, std::index_sequence<I...>) {
    (new (addr<I>()) Elems{std::move(get<I>(other))}, ...);
  }
  template <size_t... I>
  void copy_construct(const tuple &other, std::index_sequence<I...>) {
    (new (addr<I>()) Elems{get<I>(other)}, ...);
  }
  template <size_t... I>
  void move_assign(tuple &&other, std::index_sequence<I...>) {
    (static_cast<void>(get<I>(*this) = std::move(get<I>(other))), ...);
  }
  template <size_t... I>
  void copy_assign(const tuple &other, std::index_sequence<I...>) {
    (static_cast<void>(get<I>(*this) = get<I>(other)), ...);
  }

public:
  tuple() noexcept((std::is_nothrow_default_constructible_v<Elems> && ...)) {
    default_construct(idx_seq{});
  }
  ~tuple() {
    destroy(idx_seq{});
  }
  tuple(tuple &&other) noexcept((std::is_nothrow_move_constructible_v<Elems> && ...)) {
    move_construct(other, idx_seq{});
  }
  tuple(const tuple &other) noexcept((std::is_nothrow_copy_constructible_v<Elems> && ...)) {
    copy_construct(other, idx_seq{});
  }
  tuple &operator=(tuple &&other) noexcept((std::is_nothrow_move_assignable_v<Elems> && ...)) {
    move_assign(other, idx_seq{});
    return *this;
  }
  tuple &operator=(const tuple &other) noexcept((std::is_nothrow_copy_assignable_v<Elems> && ...)) {
    copy_assign(other, idx_seq{});
    return *this;
  }
};

---

Alternatively, you could use this function:

template <size_t I, typename Tuple>
size_t member_offset() {
  return reinterpret_cast<size_t>(&std::get<I>(*static_cast<Tuple *>(nullptr)));
}

template <typename Member, typename Class>
size_t member_offset(Member (Class::*ptr)) {
  return reinterpret_cast<size_t>(&(static_cast<Class *>(nullptr)->*ptr));
}

template <auto MemPtr>
size_t member_offset() {
  return member_offset(MemPtr);
}

Once again, this is undefined behavior (because of the nullptr dereference and the reinterpret_cast ) but it will work as expected with every major compiler. The function cannot be constexpr (even though member offset is a compile-time calculation).

Answer 2

Not sure to understand what do you exactly want but... what about using recursion based on a index sequence (starting from C++14) something as follows?

#include <vector>
#include <utility>
#include <iostream>

template <typename... members>
struct record
 { };

template <typename member, typename... members>
struct record<member, members...> : record<members...>
 {
   record (member m, members... ms) : record<members...>(ms...), tail(m)
    { }

   member tail;
 };

template <std::size_t, typename, std::size_t = 0u>
struct get_offset;

template <std::size_t N, typename A0, typename ... As, std::size_t Off>
struct get_offset<N, record<A0, As...>, Off> 
   : public get_offset<N-1u, record<As...>, Off+sizeof(A0)>
 { };

template <typename A0, typename ... As, std::size_t Off>
struct get_offset<0u, record<A0, As...>, Off> 
   : public std::integral_constant<std::size_t, Off>
 { };

template <typename... Args, std::size_t ... Is>
auto make_table_helper (std::string const & name,
                        std::vector<record<Args...>> const & data,
                        std::index_sequence<Is...> const &)
 { return std::vector<std::size_t>{ get_offset<Is, record<Args...>>::value... }; }

template <typename... Args>
auto make_table (std::string const & name,
                 std::vector<record<Args...>> const & data)
 { return make_table_helper(name, data, std::index_sequence_for<Args...>{}); }

int main ()
 {
   std::vector<record<int, float>> table = { {1, 1.0}, {2, 2.0} };

   auto v = make_table("table", table);

   for ( auto const & o : v )
      std::cout << o << ' ';

   std::cout << std::endl;
 }

Unfortunately isn't an efficient solution because the last value is calculated n-times.