如何杀死使用hadoop jar命令开始的hadoop任务？

Question

I'm pretty new to using Hadoop.

I used hadoop jar command like the one below -

hadoop jar $jarpath/BDAnalytics.jar \
   bigdat.twitter.crawler.CrawlTwitter \
   $crwlInputFile  > $logsFldr/crawler_$1.log 2>&1 &

But I need to kill this process, and not able to understand how. There are a lot of links to kill hadoop jobs but since this is not a job but a task/java process.

I will high appreciate if you could let me know the command to kill such a process. Thanks in advance! -RG

Answer 1

You can use the shell command kill . For example, use ps -ef | grep bigdat.twitter.crawler.CrawlTwitter ps -ef | grep bigdat.twitter.crawler.CrawlTwitter to find the pid, and use kill -9 pid_of_the_process to kill it. You can write a script containing the following command to do the kill action:

#!/bin/bash    
kill -9 $(ps -ef | grep bigdat.twitter.crawler.CrawlTwitter | sed "s/\s\+/\t/g" | cut -f2)