通过 Java 运行 Python 脚本，只需导入一次

Question

Update April 2020 : I have accepted the answer below because it proposes a very good and simple solution to my problem but I never got my code to work! If anyone has already built something similar please contact me!

---

I have a simple script my-script.py that does some calculations very fast using a specific library written in Python :

import sys
import special-library

def fun(x):
  do something

fun(sys.argv[1])

In my Java code, I use/call this script a lot in different part of the code, using the ProcessBuilder approach, which means that I simply run the command python my-script.py argument . This means that every time I call this script, I have to execute the import commands which is the most time-consuming thing in this (the calculation part is actually faster).

Is there a solution so that I call the import just once ? I looked a little bit about Jython - would be possible to write a class or something that would be initiated and do the import once and then calling a function of it every time I want to do the calculations part ( fun )? Has anyone done something similar or have any other solution to this?

First Attempt at Implementation

I've tried to write a 'pipe' Java Class that will execute the python script once. Now the script reads continuously from the stdin and when it gets an input argument , it does the calculations and returns the result:

import sys
import special-library

def fun(x):
  do something
  print('END')

for arg in sys.stdin:
  fun(arg)

And this works of course when I test it from the command line and supplying it with input arguments.

The Java Class is as follows:

package my.package;
import java.io.*;

public class PythonScriptExecuter {

    private static BufferedReader pythonToJavaReader;
    private static PrintWriter javaToPythonWriter;

    public PythonScriptExecuter() throws Exception {
        String MPBNScriptFile = "fullPathToScriptFile";
        ProcessBuilder pb = new ProcessBuilder("python", MPBNScriptFile);
        pb.redirectErrorStream(true);

        // Start script
        Process p = pb.start();

        pythonToJavaReader = getOutputReader(p); // from python to java
        javaToPythonWriter = getInputWriter(p); // from java to python
    }

     // Python => Java
    private static BufferedReader getOutputReader(Process p) {
        return new BufferedReader(new InputStreamReader(p.getInputStream()));
    }

    // Java => Python
    private static PrintWriter getInputWriter(Process p) {
        return new PrintWriter(new OutputStreamWriter(p.getOutputStream()));
    }

    public static Arraylist<String> getResults(String argument) throws IOException {
        // send argument to python script
        javaToPythonWriter.println(argument);
        //javaToPythonWriter.flush();

        // get results back one by one
        ArrayList<String> results = new ArrayList<>();

        int count = 0;
        String line;
        while (!(line = pythonToJavaReader.readLine()).equals("END")) {
            // process `line` string 
            results.add(line);
        }

        return results;
    }
}

So, what I do is that somewhere in the initialization part of my code I have this simple line to start the process:

new MPBNPythonScriptExecuter();

and later when I want to get some results back, I use:

String arg = "something"
Arraylist<String> res = PythonScriptExecuter.getResults(arg);

and the whole thing hangs on the reading-the-output-from-the-script line:

while (!(line = pythonToJavaReader.readLine()).equals("END"))

Any ideas what is going wrong here?

Answer 1

You could communicate between java and Python with pipes.

You run your Python as you do now (without command line args)

Python script

you write an infinite loop in python that will

2. read data from the standard input (it will be your arg for the function)

3. You call your function

4. you write the answer to the standard output

Java

1. Write a method for sending args to python

2. write to the pipe the arg of your function

3. read the answer from the pipe

You're done.

Here is some snippet

Here how you create your tube

        Process p = Runtime.getRuntime().exec(commande);
        BufferedReader output = getOutput(p); //from python to java
        BufferedReader error = getError(p); //from python to java
        PrintWriter input  = getInput(p); //from java to python

private static BufferedReader getOutput(Process p) {
    return new BufferedReader(new InputStreamReader(p.getInputStream()));
}
private static BufferedReader getError(Process p) {
    return new BufferedReader(new InputStreamReader(p.getErrorStream()));
}    
private static PrintWriter getInput(Process p){
    return new PrintWriter (new OutputStreamWriter(p.getOutputStream()));
}

Answer 2

Try the following in your Python script:

import sys

def fun(x):
  print(x)
  print('END')
  sys.stdout.flush()

while 1:
    try:
        line = sys.stdin.readline()
    except KeyboardInterrupt:
        break

    if not line:
        break

    fun(line)

Your Java code should be more or less correct.

Answer 3

I did a similar project. The options we had was Jython, Jep, graalVM and socket technology. Indeed, I have never tested Jython because I need to call numpy which is not possible. Jep looks promising but you have to build it for specific version, and I could not make it work. GraalVM is very nice, however It did not exist at that time and (it is not available for windows, but don't worry you can make the binary) and it is still experimental feature. So the only option was sockets technology. you can easily make a socket in java, and likewise in python. Then write a set of function to convert string to primitive arrays and wise versa. The drawback is needing compilation against your machines python and the time for communications. I put the project on git, I hope you will find it useful.
the Project's repo: https://github.com/amirshamaei/HLSVDPro4J