Python sys.stdin.buffer size detection

Question

I'm trying to execute a Python script from a Qt application and to communicate with the script via standard input and output (as one would do via common Unix pipes). My calling code stub looks like this:

int main(int argc, char *argv[]) {
  QCoreApplication a(argc, argv);
  QProcess process;
  QTimer timer;
  QObject::connect(&process, &QProcess::readyReadStandardOutput, [&process]() {
    qDebug() << "<<o" << process.readAllStandardOutput();
  });
  QObject::connect(&process, &QProcess::readyReadStandardError, [&process]() {
    qDebug() << "<<e" << process.readAllStandardError();
  });
  QObject::connect(&process, &QProcess::started, [&process] {
    qDebug() << "Process name" << process.program() << process.processId();
  });
  QObject::connect(&timer, &QTimer::timeout, [&process]() {
    qDebug() << process.state();
    QByteArray ba("12345");
    qDebug() << ">>" << ba;
    process.write(ba);
    if (!process.waitForBytesWritten())
      qDebug() << process.errorString();
  });
  QObject::connect(&a, &QCoreApplication::aboutToQuit, [&]() {
    process.terminate();
    process.kill();
  });
  process.start("python3", {"main.py"});
  //  process.start("cat", QStringList{});
  timer.start(2000);
  a.exec();
  process.terminate();
  process.kill();
  return 0;
}

And my Python script is shown below:

import sys, time

def process_data(data):
    size=len(data)
    if size %2:
        print(f'Odd, {size}',data) 
    else:
        print(f'Even, {size}',data)
    sys.stdout.flush()


if __name__ == '__main__':
    while True:
        data=sys.stdin.buffer.read(5)
        if len(data):
            process_data(data)
        else:
            print('.')
        time.sleep(0.02)

The thing is that I want to have my script react on any incoming buffer, much like a cat command does. When I comment out the line calling my script and uncomment the one calling the cat command, each time I send a buffer, I receive a reply, which is what I want. But when I'm calling a Python script, I have no means of detecting incoming buffer size that I know of. Explicitly setting a value in a sys.stdin.buffer.read command allows me not to wait for an EOF, but I want to receive a buffer without knowing its size in advance. In Qt I would achieve such behavior by calling readAll() method of a QIODevice. Is there a way of doing the same in Python?

I have tried calling sys.stdin.buffer.read() without any arguments, expecting it to behave like QIODevice::readAll() - producing a buffer with all the data read so far. But obviously it produces nothing until it receives an EOF. I hope there is a kind of method that yields a size of buffer received so that I could write:

size=stdin.buffer.bytesReceived()
data=stdin.buffer.read(size)

yet such method seems to be missing.

Does anyone know of any solution to this problem?

Answer 1

The problem is solved by changing sys.stdin.buffer.read line to:

data = sys.stdin.buffer.raw.read(20000)

This also works:

data = sys.stdin.buffer.read1(20000) 

---

_{This answer was posted as edit 1 and edit 2 to the question Python sys.stdin.buffer size detection [solved] by the OP Kirill Didkovsky under CC BY-SA 4.0.}