Python Sub-Process in Emacs Not Getting Input or Sending Back Results

Question

BACKGROUND

I am writing a small package to farm out Emacs Lisp computations to a Python sub-process so I don't have to reproduce complicated numerical routines. For my particular problem, I need Python as a sub-process because I need to maintain certain time-consuming computations done by Python. For small things, I can do things perfectly fine by sending computations to Python via shell commands. However, I DO need the sub-process for more intensive stuff.

SETUP

1. GNU Emacs 25.2.1 under Windows 10

2. Anaconda's Python 3.5 distribution for Windows 10

SAMPLE RUN

From any buffer, hit Mx ielm. This will bring up the IELM Emacs Lisp shell. Here is a transcript of my session a few minutes ago:

*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> (setq PyProc (start-process "python" "*python*" "python"))
#<process python>
ELISP> (process-list)
(#<process python> #<process ielm>)

ELISP> (process-status PyProc)
run
ELISP> (process-type PyProc)
real
ELISP> (process-send-string PyProc "1+1")
nil
ELISP> (process-send-eof PyProc)
#<process python>

As you run the above Emacs Lisp in IELM (or whatever way you want), you can watch the python buffer. Nothing displays. However, as you clearly see, I can kill the process and query its running status.

PROBLEM

Why is the Python process unable to display ANYTHING in the python buffer? I have done this with MySQL to a remote server (using the local install of MySQL) with no problems.

Answer 1

I have done some investigations. I have a partial workaround. However, I still have not clue why the original attempt using start-process does not work.

I know that one can use

M-x run-python

to launch an inferior Python process. One can then use use

(process-send-string PyProc "1+1\n")

where PyProc is the symbol holding your process object. This sends the command to the inferior process and presses return. The strange thing is that the "1+1\\n" does not display in the Python buffer. The returned value does display. Storing the point before and after the output, it is then easy to retrieve Python's output. Here is a partial snippet:

;; Run inferior Python process (assumes you have Python properly installed)
(setq PyProc (run-python))
;; Send a carriage return. There is some issue when the process first starts
(process-send-string PyProc "\n")
;; Store the point prior to sending a command
(setq PyPM (with-current-buffer "*Python*"
     (point)
     )
  )
;; Send a command to Python
(process-send-string PyProc "1+1;\n")
;; Get the result of the computation
(setq Result (with-current-buffer "*Python*"
               (buffer-substring-no-properties PyPM
                                       (- (point) 5)
               ) 
       )
  )

Evaluation of Result gives you

Result -> "2"

You can terminate the process using

(process-send-eof PyProc)

Still don't know why I cannot simply use start-process.

Answer 2

In my first answer above, I resorted to

(setq PyProc (run-python))

to run Python as an inferior process since I had not been able to launch Python using the function start-process. Per the GNU Emacs documentation, there can be lots of issues with DOS programs because of the way they handle the stdin and stdout. However, python provides a "-i" command line flag that solves the problem on windows.

All you have to do is:

(setq PyProc (start-process "python" "*Python*" "python" "-i"))
(process-send-string PyProc "1+1;\n")

If you switch to buffer Python , you will see "2" as the result of the computation. If you want to see this work nicely, simply spit your window vertically by using Cx 3. Switch to Python on the right and run your code from the left buffer.

When you are ready to kill the Python process, just

(process-send-eof PyProc)

Kill you buffer to clean up. Hope this helps those trying to control processes on the Windows platform.