Erlport/Python STDOUT 捕获到 Elixir

Question

I'm trying to pipe STDOUT from Python/Erlport back to Elixir. I've got :python calls working fine, I just want to send the STDOUT stuff from Python back to Elixir for logging but I can't wrap my head around how to achieve that. I know it'spossible even though I'm using Python 2.7.

I've got a Genserver wrapper around the :python module so that my call works like so:

pid = Python.start()
Python.call(pid, :bridge, :register_handler, [self()]) 

Python.call just looks like this:

def call(pid, module, function, args \\ []) do
  :python.call(pid, module, function, args)
end

Anything from :bridge (ie, bridge.py ) is lost to STDOUT unless I explicitly return something (obviously halting the function). What can I do to capture STDOUT?

My idea was to call something like Python.call(pid, :builtins, :print, [self()]) but that results in a bunch of errors and I really don't know if that's the right direction at all.

I actually want to pipe it into a Phoenix channel, but that's the easy part (I hope). Any advice? Thanks.

Answer 1

My idea was to call something like Python.call(pid, :builtins, :print, [self()]) but that results in a bunch of errors and I really don't know if that's the right direction at all.

self() is not where the output goes--rather self() is the argument for print , ie what python will print out.

I think erlport can only handle MFA calls(module, function, argument), and because print isn't a function in python 2.7 , I think you need to wrap a function around print , eg:

myprint.py:

def print_this(str):
    print str

I just want to send the STDOUT stuff from Python back to Elixir for logging but I can't wrap my head around how to achieve that. I know it's possible even though I'm using Python 2.7

Theerlport docs say:

As a convenient feature ErlPort also supports redirection of Python`s STDOUT to Erlang...

That appears to be the default setup, so you don't have to do anything to get python stdout to be redirected to elixir stdout. The question then becomes: "How do you log elixir stdout to a file?"

I'm able to log elixir stdout to a file like this:

friends.ex:

defmodule Friends do

  use Export.Python

  def go do

    #Get path to logfile:

    priv_path = :code.priv_dir(:friends)
    logfile_path = Path.join([priv_path, "log", "mylog.log"])

    #Redirect stdout:

    {:ok, io_pid} = File.open(logfile_path, [:append])
    Process.group_leader(self(), io_pid)

    #Send output to stdout:

    IO.puts "Am I in the log file??!"

    python_path = Path.expand("lib/python") 
    {:ok, py} = Python.start(
                 python: "python2.7",
                 python_path: python_path
               )

    Python.call(py, "myprint", "print_this", ["hello world!"])
    Python.call(py, "myprint", "print_this", ["goodbye..."])

    Python.stop(py)
  end

end

This is my directory structure:

friends
    /lib
      /friends
      /python
          myprint.py
      friends.ex
   /priv
      /log
          mylog.log

In iex:

~/elixir_programs/friends$ iex -S mix
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Compiling 1 file (.ex)
Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)> Friends.go
iex(2)>

In the log file:

Am I in the log file??!
hello world!
goodbye...
[33m[36m:ok[0m[33m[0m

(I don't know what that junk is on the last line. Edit: Hmmm...it's the atom :ok surrounded by some other stuff.)

If I comment out everything inside go() above the python_path line, then I get:

iex(1)> Friends.go
hello world!
goodbye...
:ok

In erlang/elixir file I/O is handled by starting a process to which requests are sent to either write to the file or read the file. I think stdout gets sent to whatever process is the group_leader , and if the process handling file I/O is the group_leader, then stdout gets sent to the file.

I don't know if messing with the group_leader will screw things up when you are using a GenServer . There is a warning in the erlang docs :

The group leader should be rarely changed in applications with a supervision tree, because OTP assumes the group leader of their processes is their application master.

.

Answer 2

For anyone else getting stuck with this kinda thing: since I've got a GenServer around the :python instance, I've just leveraged handle_info :

def handle_info({:python, message}, session) do
  message |> String.split("\n", trim: true)
  SomeWeb.Endpoint.broadcast("log", "update", %{body: message})

  {:stop, :normal,  session}
end

Detail

To more fully outline my solution as @7stud advised, I'll include the wider approach based on erlport and this great post . Accordingly, I've got a Python module that looks like this:

defmodule App.Python do
   @doc """
      Python instance pointing to priv/python.
    """
   def start() do
      path = [
         :code.priv_dir(:prefect),
         "python"
      ]|> Path.join()

      {:ok, pid} = :python.start([
         {:python_path, to_charlist(path)}
      ])
      pid
   end

   def call(pid, module, function, args \\ []) do
      :python.call(pid, module, function, args)
   end

   def cast(pid, message) do
      :python.cast(pid, message)
   end

   def stop(pid) do
      :python.stop(pid)
   end
end

It's called from a GenServer that handles its spawning and termination:

defmodule App.PythonServer do
   @doc """
      Receives async. messages from Python instance.
    """
   use GenServer

   alias App.Python

   def start_link() do
      GenServer.start_link(__MODULE__, [])
   end

   def init(_args) do
      pid = Python.start()
      Python.call(pid, :bridge, :register_handler, [self()])
      App.Application.broadcast_change

      {:ok, pid}
   end

   def cast_draw(id) do
      {:ok, pid} = start_link()

      GenServer.cast(pid, {:id, id})
   end

   def call_draw(id) do
      {:ok, pid} = start_link()

      GenServer.call(pid, {:id, id}, 10_000)
   end

   def handle_call({:id, id}, _from, session) do
      result = Python.call(session, :bridge, :draw, [id])

      {:reply, result, session}
   end

   def handle_cast({:id, id}, session) do
      Python.cast(session, id)

      {:noreply, session}
   end

   def handle_info({:python, message}, session) do
      msg = message |> format_response
      {:ok, time} = Timex.now |> Timex.format("{h12}:{m}{am} {D}/{M}/{YYYY}")
      AppWeb.Endpoint.broadcast("log", "update", %{time: time, body: msg, process: message})

      {:stop, :normal,  session}
   end

   def terminate(_reason, session) do
      Python.stop(session)
      App.Application.broadcast_change

      :ok
   end

   defp format_response(message) do
      if String.contains? message, "[result] Sent" do
         message |> String.split("\n", trim: true) |> Enum.at(-2)
      else
         message |> String.split("\n", trim: true) |> Enum.take(-12) |> Enum.join("\n")
      end
   end
end

You can see at the end if STDOUT doesn't return a certain string from bridge.py (or any other Python module) it'll return a stacktrace. Speaking of, bridge.py looks like this:

import os
import sys
import subprocess

from erlport.erlang import set_message_handler, cast
from erlport.erlterms import Atom

message_handler = None # reference to the elixir process to send

cmd = "xvfb-run -a python"
py = os.path.join("/home/ubuntu/app/priv/python/export.py")

def cast_message(pid, message):
  cast(pid, message)

def register_handler(pid):
  global message_handler
  message_handler = pid

def handle_message(id):
    try:
      result = draw(id)
      print result
      if message_handler:
        cast_message(message_handler, (Atom('python'), result))
    except Exception, error:
      print error
      if message_handler:
        cast_message(message_handler, (Atom('python'), error))
      pass

def draw(id):
  proc = subprocess.check_output(
    "{0} {1} {2}".format(cmd, py, id), stderr = subprocess.STDOUT, shell = True
  )
  return proc

set_message_handler(handle_message)