简体   繁体   English

将hapi第三方库(jar)添加到map-reduce

[英]Add the hapi 3rd-party lib (jars) to map-reduce

I am trying to add the hapi jars to the map-reduce task. 我正在尝试将hapi jar添加到map-reduce任务中。 I have all the required jars under /home/training/lib 我在/ home / training / lib下有所有必需的罐子
I make the fat jar which contain all the hapi jars that i need and run 我制作了包含我需要和运行的所有hapi罐子的胖罐子
hadoop jar mp.jar stubs.MsgParser infile outfile -libjars /home/training/lib i got this Usage: MsgParser [generic options] <input dir> <output dir> and in my driver i implemented the ToolRunner hadoop jar mp.jar stubs.MsgParser infile outfile -libjars /home/training/lib我得到了这个Usage: MsgParser [generic options] <input dir> <output dir>在我的驱动程序中我实现了ToolRunner
More Info: 更多信息:

  • I am running on CloudEra distribution. 我在运行CloudEra发行版。
  • i need the jars only in the mapper 我只需要在mapper中使用jar
  • mp.jar is exported jar from Eclipse mp.jar是从Eclipse导出的jar

Is my hadoop jar command need modification or is there any other solution to add these jars, i can't copy the jars to /usr/lib/hadoop/client-0.20 cause i am not the owner 我的hadoop jar命令是否需要修改或是否有任何其他解决方案来添加这些罐子,我无法将罐子复制到/usr/lib/hadoop/client-0.20因为我不是所有者
Thank you in advance! 先感谢您!

Solution: upload your jars to hdfs and simply add 解决方案:将您的罐子上传到hdfs并简单地添加

DistributedCache.addFileToClassPath(new Path(HDFSPATH), conf);  

in the main method of the driver 在驱动程序的主要方法中

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM