如何使用Ansible清除Hadoop YARN緩存？

Question

我想刪除Hadoop集群中的所有緩存文件夾。 我已經寫下了一些步驟，但是不確定如何使用Ansible腳本。 總的來說，我理解Ansible，但是我對於應該使用哪些模塊感到有些困惑。

腳步：

1. 停止YARN服務。

    $ sudo service hadoop-mapreduce-historyserver stop $ sudo service hadoop-yarn-resourcemanager stop $ sudo service hadoop-yarn-nodemanager stop 

2. 登錄所有節點並刪除用戶緩存目錄的內容。 例如：

    for i in cat list_of_nodes_in_cluster; do ssh $i rm -rf /prod/HDOP/data/13/daas.na/yarn/nm/usercache/*; done 

3. 驗證所有節點中的所有用戶緩存目錄為空。

4. 啟動YARN服務。

    $ sudo service hadoop-mapreduce-historyserver start $ sudo service hadoop-yarn-resourcemanager start $ sudo service hadoop-yarn-nodemanager start 

5. 將消息“ YARN緩存已刷新。 ”發送到我們的組ID。

Answer 1

您可以使用此Ansible劇本來執行任務。

1. 停止YARN服務。

    - name: service hadoop-mapreduce-historyserver stop service: name: hadoop-mapreduce-historyserver state: stopped - name: service hadoop-yarn-resourcemanager stop service: name: hadoop-yarn-resourcemanager state: stopped - name: service hadoop-yarn-nodemanager stop service: name: hadoop-yarn-nodemanager state: stopped 

2. 登錄所有節點並刪除用戶緩存目錄的內容。

    - name: delete the content of the usercache directories file: path="{{ item }}" state=absent with_fileglob: - /prod/HDOP/data/13/daas.na/yarn/nm/usercache/* 

3. 驗證所有節點中的所有用戶緩存目錄為空。

     - name: Verify all usercache directories are empty. shell: /usr/bin/ls ~/soniya/ | wc -l register: total_size - debug: msg: "Directory total size is {{ total_size.stdout_lines }}" when: total_size.rc == 0 

4. 啟動YARN服務。

    - name: service hadoop-mapreduce-historyserver start service: name: hadoop-mapreduce-historyserver state: started - name: service hadoop-yarn-resourcemanager start service: name: hadoop-yarn-resourcemanager state: started - name: service hadoop-yarn-nodemanager start service: name: hadoop-yarn-nodemanager state: started 

   5. 將“ YARN緩存已被刷新”消息發送到我們的組ID

      您可以為此使用郵件模塊。