How to run Test class for HDFS?

Question

I want to run a unit test by using hadoop-hdfs-2.9.2-tests.jar in hadoop-2.9.2/share/hadoop/hdfs.

1. It didn't work to run the following command:

(yarn jar hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2-tests.jar
TestWriteRead)

The error message said that there is no main class name.

2. I also tried to manually compile the wordcount class and it failed.

- javac -classpath ${HADOOP_CLASSPATH} -d
'hdfs-test/TestWriteRead-cp/'
'/home/hduser/hdfs-test/TestWriteRead.java'
- jar -cvf TestRW.jar -c TestWriteRead-cp .

but it didn't work.

How can I run the hdfs-tests.jar?

Thanks

Answer 1

You can check out here the dependencies on Hadoop 2.9.2 if you want to run same examples you can use this:

# su - hdfs
$ cd /opt/yarn/hadoop-2.9.2/bin
$ export YARN_EXAMPLES=/opt/yarn/hadoop-2.9.2/share/hadoop/mapreduce
$ ./yarn jar $YARN_EXAMPLES/hadoop-mapreduce-examples-2.9.2.jar pi 16 1000

There are a big list of examples to run included wordcount, a map/reduce program that counts the words in the input files. You can list it:

yarn jar $YARN_EXAMPLES/hadoop-mapreduce-examples.jar

If the path is not the same just run:

 $ find / -name "hadoop-mapreduce-examples*.jar" -print