从Hadoop中的作业conf获取输入路径

Question

I am setting a path as input location to conf

FileInputFormat.setInputPaths(conf, new Path("path/to/folder"));

How can I retrieve this location back from conf as I am trying to implement my own RecordReader

Thanks in advance...

Answer 1

The property set by this call is map.input.dir , so this should work for you:

conf.get("map.input.dir");

On a side note, your record reader should act upon the input split it is given in the initialize(InputSplit, TaskAttemptContext) method, as the folder you pass in setInputPath will actually resolve to a number of input splits, typically one for each file in the folder (and possible multiple input splits for larger, splittable files).

FileInputFormat based input formats are passed a FileSplit to the initialize method, and you should be able to pull out the actual file to be processed from the FileSplit.getPath() method.