how to use a C++ thread pool with serial execution order

Question

I am trying to use a C++ thread pool that uses priority based tasks. Depending on the priority (its a comparator object and not just a value in my case) it needs to be executed either serially rather than just dispatched to the next available thread in the thread pool.

My current implementation is based on the following code https://github.com/en4bz/ThreadPool and I having it working just fine as a normal thread pool (I am not using the priority variant of that pool though, as I don't know how to specify a custom predicate object instead of the int - if anyone can let ke know how to pass in the PriorityLevel below that would be a real plus) so instead the items are fed into the thread pool from a std::priority_queue< T > where T is an object that uses the a nexted PriorityLevel to order the items in priority order.

The type of task priorities are described per the lines below - these consist of a channel number, priority character 'AZ' and an optional sequence number (which when specified, indicates that the task must wait for all higher priority tasks to finish before the task is scheduled for execution on an available thread pool. I know how to order these things in a thread pool using the operator<() strict weak ordering predicate - but I do not know how I can put some of these elements in back to back execution queues.

So in this example of

(1) channel[1] priority[A] 
(2) channel[1] priority[A] sequenceNum[1]
(3) channel[1] priority[A] sequenceNum[2] 
(4) channel[1] priority[A] sequenceNum[3] 
(5) channel[2] priority[B] 
(6) channel[2] priority[B] sequenceNum[1] 
(7) channel[2] priority[B] sequenceNum[2]

Items 1 & 5 would have top priority and as they have no pre-requisites - they would run concurrently (if there are available threads) the other elements however would have to wait until their prerequisite channel/priority tasks completed.

Here is how I use the thread pool (note that the SLDBJob contains the PriorityLevel to handle the operator<() priority ordering.

    std::priority_queue<SLDBJob> priorityJobQueue;
    //... insert a bunch of Jobs 
    // enqueue closure objects in highest to lowest priority so that the 
    // highest ones get started ahead of the lower or equal priority jobs.  
    // these tasks will be executed in priority order using rPoolSize threads
    UtlThreadPool<> threadPool(rPoolSize);
    while (!priorityJobQueue.empty()) {
        const auto& nextJob = priorityJobQueue.top();
        threadPool.enqueue(std::bind(
            &SLDBProtocol::moduleCheckingThreadFn, 
            nextJob, std::ref(gActiveJobs)));
        gActiveJobs.insert(nextJob);
        priorityJobQueue.pop();
    }

and here is the priority class

class PriorityLevel {
public:
    // default constructor
    explicit PriorityLevel(
        const int32_t& rChannel = -1,
        const char priority = 'Z',
        const boost::optional<int32_t>& rSequenceNum =
            boost::optional<int32_t>())
        : mChannel(rChannel)
        , mPriority(priority)
        , mSequenceNum(rSequenceNum)
    {}

    // copy constructor
    PriorityLevel(const PriorityLevel& rhs)
        : mChannel(rhs.mChannel)
        , mPriority(rhs.mPriority)
        , mSequenceNum(rhs.mSequenceNum)
    {}

    // move constructor
    PriorityLevel(PriorityLevel&& rhs)
        : mChannel(std::move(rhs.mChannel))
        , mPriority(std::move(rhs.mPriority))
        , mSequenceNum(std::move(rhs.mSequenceNum))
    {}

    // non-throwing-swap idiom
    inline void swap(PriorityLevel& rhs) {
        // enable ADL (not necessary in our case, but good practice)
        using std::swap;
        // no need to swap base members - as we are topmost class
        swap(mChannel, rhs.mChannel);
        swap(mPriority, rhs.mPriority);
        swap(mSequenceNum, rhs.mSequenceNum);
    }

    // non-throwing copy-and-swap idiom unified assignment
    PriorityLevel& operator=(PriorityLevel rhs) {
        rhs.swap(*this);
        return *this;
    }

    // equality operator
    inline bool operator==(const PriorityLevel& rhs) const {
        return std::tie(mChannel, mPriority, mSequenceNum) ==
            std::tie(rhs.mChannel, rhs.mPriority, rhs.mSequenceNum);
    }

    // inequality operator
    inline bool operator!=(const PriorityLevel& rhs) const {
        return !(operator==(rhs));
    }

    /**
     * comparator that orders the elements in the priority_queue<p>
     *
     * This is implemented via a lexicographical comparison using a
     * std::tuple<T...> as a helper. Tuple compares work as follows:
     * compares the first elements, if they are equivalent, compares
     * the second elements, if those are equivalent, compares the
     * third elements, and so on. All comparison operators are short
     * - circuited; they do not access tuple elements beyond what is
     * necessary to determine the result of the comparison. note
     * that the presence of the sequence number assigns a lower
     * priority (bigger value 1) contribution to the lexicographical
     * nature of the comparison
     *
     * @param rhs    PriorityLevel to compare against
     *
     * @return true if this is lower priority than rhs
     */
    inline bool operator<(const PriorityLevel& rhs) const {
        auto prtyLen = getPriorityStr().length();
        auto rhsPrtyLen = rhs.getPriorityStr().length();
        auto sequencePrtyVal = mSequenceNum ? mSequenceNum.get() : 0;
        auto rhsSequencePrtyVal = rhs.mSequenceNum ? rhs.mSequenceNum.get() : 0;
        return std::tie(prtyLen, mPriority, mChannel, sequencePrtyVal) >
            std::tie(rhsPrtyLen, rhs.mPriority, rhs.mChannel, rhsSequencePrtyVal);
    }

    // stream friendly struct
    inline friend std::ostream& operator << (std::ostream& os, const PriorityLevel& rValue) {
        std::string sequenceInfo;
        if (rValue.mSequenceNum) {
            sequenceInfo = std::string(", sequence[") +
                std::to_string(rValue.mSequenceNum.get()) + "]";
        }
        os  << "channel[" << rValue.mChannel
            << "], priority[" << rValue.mPriority
            << "]" << sequenceInfo;
        return os;
    }

    // channel getter
    inline int32_t getChannel() const {
        return mChannel;
    }

    // string representation of the priority string
    inline std::string getPriorityStr() const {
        std::stringstream ss;
        ss << mChannel << mPriority;
        if (mSequenceNum) {
            ss << mSequenceNum.get();
        }
        return ss.str();
    }
private:
    // the 3 fields from the ModuleNameTable::szPriorityLevel
    int32_t mChannel;
    // single upper case character A=>'highest priority'
    char mPriority;
    // optional field - when present indicates start order
    boost::optional<int32_t> mSequenceNum;
};

Answer 1

I wouldn't put all of them into a priority_queue, since a priority_queue is extremely inefficient with things involving changing priority. Rather, I would add 1 and 5 to the priority queue, and put all the rest into a "followup map" of channels to a list of subsequent tasks. When channel 1 finishes, it checks to see if channel1 has anything in the followup map, and if so, pops the first item out of that list, and adds it to the priority_queue.

 using ChannelID = int32_t;
 using PriorityLevel = char;

 struct dispatcher {
      std::priority_queue<SLDBJob> Todo; //starts with items 1 and 5
      std::unordered_map<ChannelID, std::vector<SLDBJob>> FollowupMap;
          //starts with {1, {2,3,4}}, {2, {6, 7, 8}}
          //note the code is actually faster if you store the followups in reverse

      void OnTaskComplete(ChannelID id) {
          auto it = FollowupMap.find(id);
          if (it != FollowupMap.end())
              if (it->empty() == false) {
                  Todo.push_back(std::move(it->front()));
                  it->erase(it->begin());
              }
              if (it->empty() == true)
                  FollowupMap.erase(it);
           }
      }
 };

Usage would be vaguely as follows:

struct reportfinished {
    ChannelID id;
    ~reportfinished() {dispatcher.OnTaskComplete(id);} //check for exceptions? Your call.
};

UtlThreadPool<> threadPool(rPoolSize);
while (!priorityJobQueue.empty()) {
    const auto& nextJob = priorityJobQueue.top();
    auto wrapper = [&gActiveJobs, =]() 
        -> decltype(SLDBProtocol::moduleCheckingThreadFn(nextJob, gActiveJobs))
        {
            reportfinished queue_next{nextJob.channel};
            return SLDBProtocol::moduleCheckingThreadFn(nextJob, gActiveJobs);
        };
    threadPool.enqueue(std::move(wrapper));
    gActiveJobs.insert(nextJob);
    priorityJobQueue.pop();
}