[英]Hadoop jar - System cannot find the path specified
I'm trying to run a jar file with hadoop using command 我正在尝试使用命令使用hadoop运行jar文件
hadoop jar test.jar org.ipiran.hadoop.sample.TestMapReduce /user/data.txt /output/1.txt 4
But it returns error 但它返回错误
Exit code: 1 Exception message: System cannot find the path specified
退出代码:1异常消息:系统找不到指定的路径
I tried without specifying the class but got the same result. 我尝试不指定类,但得到了相同的结果。 I have file data.txt.
我有data.txt文件。 My main java code
我的主要Java代码
public int run(String[] args) throws Exception {
Configuration conf = this.getConf();
conf.set("fs.defaultFS", "hdfs://localhost:9000/");
conf.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
conf.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());
conf.set("source", args[2]);
long numUpdated = 1;
int code = 0;
int numIterations = 1;
FileSystem hdfs = FileSystem.get(conf);
while (numUpdated > 0) {
logger.info("Iteration: " + numIterations);
String input, output;
Job job = Job.getInstance(conf, "word count");
if (numIterations == 1) {
input = args[0];
} else {
input = args[1] + "-" + (numIterations - 1);
}
output = args[1];// + "-" + numIterations;
job.setJarByClass(TestMapReduce.class);
job.setMapperClass(testmap.class);
job.setReducerClass(TestReduce.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path(input));
FileOutputFormat.setOutputPath(job, new Path(output));
code = job.waitForCompletion(true) ? 0 : 1;
Counters jobCounters = job.getCounters();
numUpdated = jobCounters.
findCounter(MoreIterations.numUpdated).getValue();
if (numIterations > 1) {
hdfs.delete(new Path(input), true);
logger.info("Updated: " + numUpdated);
}
numIterations += 1;
}
return code;
}
I'm using windows 10. Does anybody know what is wrong here? 我正在使用Windows10。有人知道这里出什么问题吗?
You are using Windows 10, however you are treating it is if you are using Linux. 您正在使用Windows 10,但是如果您正在使用Linux,则将其视为是。
In Linux "/someDir/someFile means that there is a file under a directory that is under the root directory. 在Linux中,“ / someDir / someFile”表示在根目录下的目录下有一个文件。
You have to change the paths of your data files to some location that is compitable with windows. 您必须将数据文件的路径更改为Windows可以适应的某个位置。
By the way, do you have hadoop installed on your machine ? 顺便说一句,您的计算机上是否安装了hadoop?
I would recommend using a linux VM, you can use Cloudera VM as it has everything installed 我建议使用linux VM,您可以使用Cloudera VM,因为它已安装了所有内容
https://www.cloudera.com/downloads/quickstart_vms/5-12.html https://www.cloudera.com/downloads/quickstart_vms/5-12.html
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.