Can I capture lambda variables without std::function?

Question

Is it possible to get the captured values of a lambda without using std::function ? I'm asking because I want to place the captured copies into my own memory, which std::function cannot do as they don't support custom allocators.

(I assume allocator support missing for std::function is due to a very good reason, perhaps the logic behind capturing values in a lambda is extremely difficult to implement? But if it's possible, I'd like to try it myself.)

Background: I'm asking to learn more about lambda in C++. I'd like to place the captured values and reference pointers in over-aligned memory for a thread pool system I'm writing as an academic exercise. I also really like the brevity writing lambdas with automatic captures, it'd make for a very easy "job" writing interface I think.

class MyFunction {
public:

    // I want more than just the function pointer, 
    // I also want the captured value copies or references too
    // I'm unsure how to really accomplish this though.
    MyFunction & operator=( ??? ) {
        ???
    }

};

int main(){
    int captureThis = 1;
    MyFunction func = [=]()->void {
        printf("%i", captureThis);
    };
}

Answer 1

Checking values of captured variables outside of lambda would not look good but at least you can use a factory function to produce it with its unique "lambda" template type (also with help of auto on the final type) so that you can do extra work between what thread calls and what you initialize:

#include <iostream>
#include <thread>
#include <vector>

template <typename F>
struct Callable
{
    Callable(const F && lambda):func(std::move(lambda))
    {
        
    }

    // this is what the std::thread calls
    void operator()()
    {
        // you can check the captured variable
        int out;
        func(out,false);
        std::cout<< "Callable successfully found out the value of captured variable: "<<out <<std::endl;
        func(out,true);
    }
    const F func;
};


template<typename F>
Callable<F> CallableFactory(F&& lambda)
{
    return Callable<F>(std::forward<F>(lambda)); // or std::move(lambda)
}

int main()
{
    // variable to capture
    int a=1;
    
    auto callable = CallableFactory([&](int & outputCapturedValue, bool runNow){
        // not looking good as its not possible from outside (because they are private variables & depends on implementation of C++)
        // if checking the captured variables
        if(!runNow)
        {
            outputCapturedValue = a;
            std::cout << "inside the lambda: a=" << a <<std::endl;
        }
        else
        {
            std::cout<<"algorithm runs"<<std::endl;
        }
    });
    
    
    std::vector<std::thread> threads;
    threads.emplace_back(std::move(callable));
    threads[0].join();
    
    return 0;
}

output:

inside the lambda: a=1
Callable successfully found out the value of captured variable: 1
algorithm runs

---

If its only for having an array of lambdas processed by array of threads, you can use smart-pointers and an extra container struct to box/unbox them during work-distribution:

#include <iostream>
#include <thread>
#include <vector>
#include <memory>


struct ICallable
{
    virtual void operator()()=0;
};

template <typename F>
struct Callable:public ICallable
{
    Callable(const F && lambda):func(std::move(lambda))
    {
        
    }

    // this is what the std::thread calls
    void operator()() override
    {
        func();
    }
    const F func;
};


template<typename F>
std::shared_ptr<ICallable> CallablePtrFactory(F&& lambda)
{
    return std::shared_ptr<ICallable>(new Callable<F>(std::forward<F>(lambda)));
}

struct CallableContainer
{
    std::shared_ptr<ICallable> callable;
    void operator()()
    {
        callable.get()->operator()();
    }        
};

int main()
{
    // variable to capture
    int a=1;

    // simulating work pool
    std::vector<std::shared_ptr<ICallable>> callables;
    callables.push_back(CallablePtrFactory([&](){
        std::cout<< "a="<<a<<std::endl;
    }));

    // simulating worker pool load-balancing
    std::vector<std::thread> threads;
    threads.emplace_back(CallableContainer{ callables[0] });
    threads[0].join();
    
    return 0;
}

output:

a=1

---

If you're after a custom-allocation for the container, you can just use a second parameter for the factory function. Following example uses placement-new on a stack buffer. But still the lambda itself has something else outside of it making container's size not changed by its lambda (just like a function-pointer):

#include <iostream>
#include <thread>
#include <vector>
#include <memory>


struct ICallable
{
    virtual void operator()()=0;
};

template <typename F>
struct Callable:public ICallable
{
    Callable(const F && lambda):func(std::move(lambda))
    {
        currentSize = sizeof(*this); std::cout<<"current size = "<<currentSize <<" (useful for alignement of next element?)" <<std::endl;
    }

    // this is what the std::thread calls
    void operator()() override
    {
        func();
    }
    int currentSize;
    const F func;
    
};


template<typename F>
std::shared_ptr<ICallable> CallablePtrFactory(F&& lambda, char * buffer)
{
    return std::shared_ptr<ICallable>(
                new (buffer) Callable<F>(std::forward<F>(lambda)),
                [](ICallable *){ /* placement-new does not require a delete! */}
                );
}

struct CallableContainer
{
    std::shared_ptr<ICallable> callable;
    void operator()()
    {
        callable.get()->operator()();
    }        
};

int main()
{
    // variable to capture
    int a=1;

    char buffer[10000];

    // simulating work pool
    std::vector<std::shared_ptr<ICallable>> callables;
    
    
    callables.push_back(
        // observe the buffer for placement-new
        CallablePtrFactory([&](){
            std::cout<< "a="<<a<<std::endl;
        },buffer /* you should compute offset for next element */)
    );

  

    // simulating worker pool load-balancing
    std::vector<std::thread> threads;
    threads.emplace_back(CallableContainer{ callables[0] });
    threads[0].join();
    
    return 0;
}

output:

current size = 24 (useful for alignement of next element?)
a=1

Answer 2

Considering the thread pool system you mention in comments then you could try a polymorphic approach:

class ThreadRoutine
{
protected:
    virtual ~ThreadRoutine() { }
public:
    virtual void run() = 0;
};

template <typename Runner>
class ThreadRoutineT
{
    // variant 1:
    Runner m_runner; // store by value;
    // variant 2:
    std::reference_wrapper<Runner> m_runner;
public:
    void run() override { m_runner(); } // call operator()
};

Now you might store your thread routines in a std::vector (note: pointers to, by value would lead to object slicing; likely std::unique_ptr , possibly, depending on use case, a classic raw pointer might fit, too).

You could even implement both variants, your thread pool manager could provide an additional parameter in the thread creation function or maybe even more elegant distinguish by overloading that function (l-value reference: create the reference wrapper variant; r-value reference: create the value variant, move-construct it), eg like:

class ThreadManager
{
    template <typename Runner>
    void createThread(Runner& runner)
    {
        // assuming std::vector<std::unique_ptr<ThreadRoutine>>
        m_runners.emplace_back
        (
            // assuming appropriate constructor
            std::make_unique<ThreadRoutineRef>(runner)
        );
        // return some kind of thread handle or start thread directly?
        // thread handle: could be an iterator into a std::list
        // (instead of std::vector) as these iterators do not invalidate
        // if elements preceding in the list are removed
        // alternatively an id as key into a std::[unordered_]map
    }
    template <typename Runner>
    void createThread(Runner&& runner)
    {
        m_runners.emplace_back
        (
            // assuming appropriate constructor
            std::make_unique<ThreadRoutineVal>(std::move(runner))
        );
    }
}

About the alignment issue: The specific template instantiations would select the alignment appropriate for the template argument type, so you wouldn't have to consider anything particular.