Difference in terminal stdin handling between Python and OCaml

Question

I'm trying to do something very specific, involving sending control chars to stdout and reading from stdin.

I have a working implementation in Python and I am trying to translate it to OCaml.

I was pleasantly surprised that it was possible to translate very directly, almost line-for-line. But when I run it the behaviour is different and the OCaml one does not work.

It appears to me the problem must be some obscure difference between how OCaml and Python runtimes handle the terminal, perhaps stdin specifically.

Firstly here is the working Python code:

import os, select, sys, time, termios, tty

def query_colours():
  fp = sys.stdin
  fd = fp.fileno()
  if os.isatty(fd):
      old_settings = termios.tcgetattr(fd)
      tty.setraw(fd)
      try:
          print('\033]10;?\07\033]11;?\07')
          r, _, _ = select.select([ fp ], [], [], 0.1)
          if fp in r:
              return fp.read(48)
          else:
              print("no input available")
              return None
      finally:
          termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
  else:
      raise ValueError("Not a tty")

And my OCaml translation looks like:

let query_colours () =
  let fd = Unix.stdin in
  if Unix.isatty fd then
    let old_settings = Unix.tcgetattr fd in
    set_raw fd;
    Fun.protect
      ~finally:(fun () -> Unix.tcsetattr fd Unix.TCSADRAIN old_settings)
      (fun () ->
          print_string "\o033]10;?\o007\o033]11;?\o007";
          let r, _, _ = Unix.select [fd] [] [] 0.1 in
          let buf = Bytes.create 48 in
          Printf.printf ">> len r: %d\n" (List.length r);  (* debugging *)
          ignore @@ (
            match List.exists (fun (el) -> el == fd) r with
            | true -> Unix.read fd buf 0 48
            | false -> failwith "No input available"
          );
          Bytes.to_string buf
        )
  else
    invalid_arg "Not a tty"

Note that we had to make an OCaml implementation of tty.setraw . First, here is the source from Python stdlib:

def setraw(fd, when=TCSAFLUSH):
    """Put terminal into a raw mode."""
    mode = tcgetattr(fd)
    mode[IFLAG] = mode[IFLAG] & ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON)
    mode[OFLAG] = mode[OFLAG] & ~(OPOST)
    mode[CFLAG] = mode[CFLAG] & ~(CSIZE | PARENB)
    mode[CFLAG] = mode[CFLAG] | CS8
    mode[LFLAG] = mode[LFLAG] & ~(ECHO | ICANON | IEXTEN | ISIG)
    mode[CC][VMIN] = 1
    mode[CC][VTIME] = 0
    tcsetattr(fd, when, mode)

iflag , oflag , cflag , lflag are bit-masked integers

In OCaml side the Stdlib has provided instead of four bit-masked ints a single record with all the boolean values: https://ocaml.org/api/Unix.html#TYPEterminal_io

My OCaml translation of tty.setraw looks like:

let set_raw ?(set_when=Unix.TCSAFLUSH) fd =
  let mode : Unix.terminal_io = {
    (Unix.tcgetattr fd) with
    c_brkint = false;
    c_icrnl = false;
    c_inpck = false;
    c_istrip = false;
    c_ixon = false;
    c_opost = false;
    c_csize = 8;
    c_parenb = false;
    c_echo = false;
    c_icanon = false;
    (* c_iexten = false; ...does not exist on Unix.terminal_io  *)
    c_ixoff = false; (* IEXTEN and IXOFF appear to set the same bit *)
    c_isig = false;
    c_vmin = 1;
    c_vtime = 0;
  } in
  Unix.tcsetattr fd set_when mode

Ok, now the problem...

When I run the Python version it just returns a string like:

'\x1b]10;rgb:c7f1/c7f1/c7f1\x07\x1b]11;rgb:0000/0000/0000\x07'

which is the intended behaviour. I do not hear the BEL sound or any other content printed to screen.

When I run my OCaml version, I hear the BEL sound and I see:

╰─ dune exec -- ./bin/cli.exe
>> len r: 0
Fatal error: exception Failure("No input available")
^[]10;rgb:c7f1/c7f1/c7f1^G^[]11;rgb:0000/0000/0000^G%

╭─    ~/Documents/Dev/ *5 !4 ?4       2 ✘  18:20:26 
╰─ 10;rgb:c7f1/c7f1/c7f1

╭─    ~/Documents/Dev/ *5 !4 ?4       2 ✘  18:20:26 
╰─ 11;rgb:0000/0000/0000

We can see from the print debugging len r: 0 that the select call did not find stdin ready for reading.

Instead we see the results sent to stdin in the terminal after my program has exited.

FWIW if I run the Python script from inside an OCaml program via Unix.open_process_in then I get the same (broken) behaviour from the Python script:

utop # run "bin/query.py";;
- : string list = ["\027]10;?\007\027]11;?\007"; "no input available"]

I realise this is maybe a bit of an obscure corner but would be very grateful if anyone has any insight.

Answer 1

This is a lot of code to read but just from the description it sounds like you're returning the terminal to its old state before flushing the output.

This isn't anything particularly strange about OCaml, but OCaml does have a tendency to hold on to buffered output longer than some other languages.

You might try adding this after the print_string :

flush stdout

Like I said, there's a lot of code to read and this is just my quick take.