translate python function with yield(string) to C++

Question

I'm translating a python function into a C++ function; this function uses a yield(string) statement that I don't know how to translate.

Here the whole story...I have a certain function that reads an input ASCII file filename , which contains lines of data, and puts the content of the file into an std::vector . When reading this input file (in another function respect with the one showed below), I need to jump a bunch of lines (5 or 6, it depends on the input file's name) and, for this purpose, I define a function, the file_struct_generator , which marks as "data" the data lines and as "unused" the data lines I don't need. I need something similar to this function in C++, and in particular, I need something similar to yield(string) (pay attention, string !), but in C++ of course. Here I show you the lines of the code I need to "translate" from python to C++. How can I rewrite yield("unused") and yield("data") in C++? Or if yield is unusable in C++, can I write a similar function in C++ using something different which works as a generator? Thanks for helping me!

def file_struct_generator(filename):
    if "more" in filename:
        bad_line_number = 5
    else:
        bad_line_number = 6

    yield("data")
    for i in range(bad_line_number):
        yield("unused")
    while True:
        yield("data")
    

EDIT: I do not use C++20, but C++11. I tried @Arthur Tacca code and it works for my purpose, thank you all.

Answer 1

translate python function with yield(string) to C++

Prior to C++20, I don't think there is a simple, exact translation.

An alternative is to write a function template that accepts a functor:

template<class Yield>
void file_struct_generator(const std::string& filename, const Yield& yield)
{
    // ...
    yield("unused")
    // ...

This way the caller can provide a functor that deals with the output in a way they want to use it. They can provide for example one that prints the values, or another that stores the values in a container.

Crucially, this alternative is not lazy, and you cannot have an infinite loop like in the python code. Whether this is a useful translation depends on how the generator is used in Python. This is good for cases where the entire sequence is consumed.

---

I think this would be a direct translation in C++20:

generator<std::string>
file_struct_generator(const std::string& filename)
{
    bad_line_number = filename.find("more") != std::string::npos
        ? 5
        : 6;

    co_yield "data";
    for (int i : std::views::iota(0, bad_line_number))
        co_yield "unused";
    for(;;)
        co_yield "data";
}

Not tested due to lack of conforming compilers. Also the standard doesn't provide the generator type.

Answer 2

This is a state machine that reproduces that generator's functionality.

#include <stdexcept>
#include <string>
#include <iostream>

class FileStructIterator {
public:
    explicit FileStructIterator(const std::string& filename): 
        m_filename(filename), m_state(STATE_START) 
    {}

    std::string getNext() {
        switch (m_state) {
        case STATE_START:
            m_state = STATE_LOOP1;
            if (m_filename.find("more") != std::string::npos) {
                m_badLineNumber = 5;
            } else {
                m_badLineNumber = 6;
            }
            m_i = 0;
            return "data";

        case STATE_LOOP1:
            ++m_i;
            if (m_i >= m_badLineNumber) {
                m_state = STATE_LOOP2;
            }            
            return "unused";

        case STATE_LOOP2:
            return "data";

        default:
            throw std::logic_error("bad state");
        }
    }

private:
    std::string m_filename;

    enum State { STATE_START, STATE_LOOP1, STATE_LOOP2 };
    State m_state;

    size_t m_badLineNumber;
    size_t m_i;
};

Here's an example of using it (in this case I limited the output to the first 10 results so it doesn't loop forever).

int main() {
    auto it = FileStructIterator("nomore.txt");
    for (int i = 0; i < 10; ++i) {
        std::string nextValue = it.getNext();
        std::cout << nextValue << "\n";
    }
}

Answer 3

From C++20 you can use co-routines, you can achieve similar functionality to the python snippet with some thread_local variables (that are static):

std::string file_struct_generator(std::string filename) 
{
  thread_local int bad_line_number = filename.find("more") != std::string::npos ? 5 : 6;

  thread_local int i = 0;

  if (i++ && i <= bad_line_number) 
    return "unused";

  return "data";
}

Note that, as pointed out by ArthurTacca in the comments, this function can't actually be called with a different file-name, or at least, it won't start a new loop if you do that.

Answer 4

In C++20 this would use co_yield . Prior to that you can write an iterator.

class file_struct_iterator : public std::iterator<std::input_iterator_tag, std::string, std::ptrdiff_t, const std::string *, std::string> {
    enum class State {
        initial,
        bad_lines,
        remainder
    }

    State state = State::initial;
    size_t bad_line_number = 0;

public:

    file_struct_iterator(std::string filename) 
        : bad_line_number((filename.find("more) != filename.end()) ? 5 : 6)
    {}

    std::string operator*() {
        switch (state) {
            case State::initial:
            case State::remainder: return "data";
            case State::bad_lines: return "unused";
        }
    }

    file_struct_iterator& operator++() {
        switch (state) {
            case State::initial: state = State::bad_lines; break;
            case State::bad_lines: if (--bad_line_number == 0) { state = remainder; } break;
            case State::remainder: break;
        }
        return *this;
    }

    file_struct_iterator operator++(int) {
        file_struct_iterator retval = *this;
        ++(*this);
        return retval;
    }

    bool operator==(file_struct_iterator other) const {
        return (state == other.state) && (bad_line_number == other.bad_line_number);
    }
    
    bool operator!=(file_struct_iterator other) const {
        return !(*this == other);
    }
};