在多節點集群中運行Hadoop無法正常工作

Question

我安裝了HDFS並在3台計算機上工作。然后，我嘗試向現有群集中再添加5台PC，但是這樣做之后。當我嘗試在主節點上啟動hadoop時，出現了以下錯誤。

[hduser@dellnode1 ~]$ start-all.sh
starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-namenode-dellnode1.pictlibrary.out
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /usr/local/hadoop/bin/../logs/hadoop-hduser-namenode-dellnode1.pictlibrary.log (Permission denied)
    at java.io.FileOutputStream.openAppend(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
    at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
    at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
    at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
dellnode3.pictlibrary: datanode running as process 4856. Stop it first.
dellnode1.pictlibrary: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-datanode-dellnode1.pictlibrary.out
dellnode2.pictlibrary: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-datanode-dellnode2.pictlibrary.out
dellnode1.pictlibrary: starting secondarynamenode, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-secondarynamenode-dellnode1.pictlibrary.out
starting jobtracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-jobtracker-dellnode1.pictlibrary.out
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /usr/local/hadoop/bin/../logs/hadoop-hduser-jobtracker-dellnode1.pictlibrary.log (Permission denied)
    at java.io.FileOutputStream.openAppend(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:207)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
    at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
    at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
    at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
dellnode3.pictlibrary: starting tasktracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-tasktracker-dellnode3.pictlibrary.out
dellnode1.pictlibrary: starting tasktracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-tasktracker-dellnode1.pictlibrary.out
dellnode2.pictlibrary: starting tasktracker, logging to /usr/local/hadoop/bin/../logs/hadoop-hduser-tasktracker-dellnode2.pictlibrary.out

所有PC都在運行Fedora 17

Answer 1

我會用類似的東西手動創建日志文件

sudo touch  /usr/local/hadoop/bin/logs/hadoop-hduser-namenode-dellnode1.pictlibrary.log

如果你的路是

/usr/local/hadoop/bin/../logs/hadoop-hduser-namenode-dellnode1.pictlibrary.log

如我在第一行中所述，更好地解決了問題。

然后更改文件的權限：

sudo chmod 750 /usr/local/hadoop/bin/logs/hadoop-hduser-namenode-dellnode1.pictlibrary.log

然后再試一次。 這次應該可以工作了;-)