使用ProcessBuilder从Java程序运行yarn工作会给出文件不存在错误

Question

I am trying to run a yarn job from a java wrapper program. The mapreduce jar takes two inputs:

1. A header file: I dont know the name of the file but the location and file extension and there's only one file at that location
2. A Input files directory

Apart from these I have an Output directory. the processbuilder code looks like:

HEADER_PATH = INPUT_DIRECTORY+"/HEADER/*.tsv";
INPUT_FILES = INPUT_DIRECTORY+"/DATA/";
OUTPUT_DIRECTORY = OUTPUT_DIRECTORY+"/";

ProcessBuilder mapRProcessBuilder = new ProcessBuilder("yarn","jar",JAR_LOCATION,"-Dmapred.job.queue.name=name","-Dmapred.reduce.tasks=500",HEADER_PATH,INPUT_DIRECTORY,OUTPUT_DIRECTORY);
System.out.println(mapRProcessBuilder.command().toString());
Process mapRProcess = mapRProcessBuilder.start();

On run, I get the following error:

Exception in thread "main" java.io.FileNotFoundException: Requested file /input/path/dir1/HEADER/*.tsv does not exist.

But when I run the same command as :

yarn jar jarfile.jar -Dmapred.job.queue.name=name -Dmapred.reduce.tasks=500 /input/path/dir1/HEADER/*.tsv /input/Dir /output/Dir/

It works all fine.

what can be the issue when running the command from java is causing this issue?

Answer 1

The * is being treated as part of the literal string in this case rather than a wildcard. Therefore globbing isn't expanding to your desired path name.

If there is only one file in the directory, why don't you find what its path is and pass that as the argument instead

eg.

File dir = new File(INPUT_DIRECTORY+"/HEADER);
if (dir.list().length > 0)
    String HEADER_PATH = dir.list()[0].getAbsolutePath();