Compile Python Sources in Maven

Question

I have some python source files in my project (under src/main/python ) along with java classes (under src/main/java ). I would like to include these python files to my maven target jar. Is there a plugin available to do this?

I can observe that the build artifact created by IntelliJ does include the python files but the one created by maven doesn't.

Answer 1

As you mentioned, your current project structure has java files in src/java . This does not conform to maven standard. You should ideally be following the standard structure, makes compatibility simpler with IDEs as well as maven.

Java files should be kept in src/main/java that you want to package in jar.

Similarily, Python are just script files, not compiled. So you can put these files in src/main/resources .

src
|---main
|   |---java (Keep your java files here)
|   |---resources
|   \    |---python (You can keep you python files here)
|        \
|---test
|   |---java (Keep your java unit test files here)
|   |---resources (If you have any resources test specific)
|   \

If its not possible, you need to update pom.xml to ensure, src/python is treated as a source folder. Refer this plugin: build helper plugin

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>add-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>src/python</source>
                ...
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Answer 2

Python does not compile into intermediate files, but just executes .py files. (Just like Node.js executes .js files.)

So we are talking about including and running.

src/main/python folder is not standard, in meaning that there is no plugins that would process python files. You can define it as

<build>
<resources>
    <resource>
        <directory>src/main/python</directory>
    </resource>
    <resource>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Resources are just copied into jar.

To access the file from Java

    private static String SOURCE_FILE_NAME = "main.py";
    private static InputStream SOURCE_FILE_INPUT = YourClassName.class.getClassLoader().getResourceAsStream(SOURCE_FILE_NAME);

For running Python from Java there are too many option, each with own limitations:

• as OS process
• as C/C++ library via Java JNI
• Jython (only Python, so practically outdated)
• GraalVM (as of 2021 experimental)

with GraalVM

        Path path = Paths.get("venv", "bin", "graalpython");
        if (path==null){
            log("venv/ is not yet copied under target/classes/, run `mvn process-resources` or any next maven phase,");
        }
        //String VENV_EXECUTABLE = RunGraalPython3.class.getClassLoader().getResource(path.toString()).getPath();
        String VENV_EXECUTABLE = path.toString();
        log(VENV_EXECUTABLE);

        //try (Context context = Context.create()) {
        //try (Context context = Context.newBuilder(PYTHON).allowAllAccess(true).build()) {
        try (Context context = Context.newBuilder("python").
                allowAllAccess(true).
                option("python.ForceImportSite", "true").
                option("python.Executable", VENV_EXECUTABLE).
                build();) {
            context.eval(PYTHON, "print('Hello Python!')");

            context.eval(PYTHON, "import sys; print(sys.version)");


            InputStreamReader reader = new InputStreamReader(SOURCE_FILE_INPUT);
            Source source;
            try {
                source = Source.newBuilder(PYTHON, reader, SOURCE_FILE_NAME).build();
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
            context.eval(source);

See https://github.com/paulvi/java-python-graalvm-template