[英]Threadpool executor not updating the concurrent hashmap
I am spawning 5 threads using the thread pool executor to execute 5 different commands in parallel. 我使用线程池执行程序生成5个线程,以并行执行5个不同的命令。 After completion of each thread i am updating the concurrent hashmap with the entries of threadid as a key and terminated as value.
在完成每个线程之后,我将使用threadid项作为键并以值终止的方式更新并发哈希图。 But my threadpool is not updating the hashmap of the successful completion of the commands execution.
但是我的线程池没有更新成功执行命令的哈希表。
Main Class: 主类:
package com.cisco.executor;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
public class MainExecutor {
static String element;
static ConcurrentHashMap<Integer, String> map = new ConcurrentHashMap<Integer, String>();
static Integer array[] = { 1, 2, 3, 4, 5 };
// static Integer array[] = { 1 };
static List<Integer> threadid = Arrays.asList(array);
static String SQOOP_XXCCS_DS_SAHDR_CORE = ReadProperties.getInstance().getProperty("SQOOP_XXCCS_DS_SAHDR_CORE");
static String SQOOP_XXCCS_DS_CVDPRDLINE_DETAIL = ReadProperties.getInstance()
.getProperty("SQOOP_XXCCS_DS_CVDPRDLINE_DETAIL");
static String SQOOP_XXCCS_DS_INSTANCE_DETAIL = ReadProperties.getInstance()
.getProperty("SQOOP_XXCCS_DS_INSTANCE_DETAIL");
static String SQOOP_XXCCS_SCDC_PRODUCT_PROFILE = ReadProperties.getInstance()
.getProperty("SQOOP_XXCCS_SCDC_PRODUCT_PROFILE");
static String SQOOP_MTL_SYSTEM_ITEMS_B = ReadProperties.getInstance().getProperty("SQOOP_MTL_SYSTEM_ITEMS_B");
public static void main(String[] args) {
ThreadPoolExecutor executors = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);
// ThreadPoolExecutor executors = (ThreadPoolExecutor) Executors.newFixedThreadPool(1);
System.out.println("at executors step");
List<String> getlist = getList();
Iterator<Integer> itr2 = threadid.iterator();
for (Iterator<String> itr = getlist.iterator(); itr.hasNext() && itr2.hasNext();) {
String element = (String) itr.next();
int thread_id = itr2.next();
String[] command = { "ssh", "hddev-c01-edge-02", "\"" + element + "\"" };
System.out.println("the command is as below ");
System.out.println(Arrays.toString(command));
System.out.println("inside the iterator");
ParallelExecutor pe = new ParallelExecutor(command, thread_id, map);
executors.execute(pe);
}
// executors.shutdown();
for(Map.Entry<Integer, String> entry: map.entrySet())
{
Integer key = entry.getKey();
String value = entry.getValue();
System.out.println("The key is " + key + " The value is " + value);
System.out.println("Thread " + key + " is terminated");
}
}
public static List<String> getList() {
List<String> commandlist = new ArrayList<String>();
System.out.println("inside getList");
commandlist.add(SQOOP_XXCCS_DS_SAHDR_CORE);
commandlist.add(SQOOP_XXCCS_DS_CVDPRDLINE_DETAIL);
commandlist.add(SQOOP_XXCCS_DS_INSTANCE_DETAIL);
commandlist.add(SQOOP_XXCCS_SCDC_PRODUCT_PROFILE);
commandlist.add(SQOOP_MTL_SYSTEM_ITEMS_B);
return commandlist;
}
}
Runnable Class: 可运行类:
package com.cisco.executor;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.concurrent.ConcurrentHashMap;
import org.apache.log4j.Logger;
public class ParallelExecutor implements Runnable {
private static Logger LOGGER = Logger.getLogger(ParallelExecutor.class);
String[] command;
int threadid;
ConcurrentHashMap<Integer, String> map;
public ParallelExecutor(String[] command, int threadid, ConcurrentHashMap<Integer, String> map) {
this.command = command;
this.threadid = threadid;
this.map = map;
}
@Override
public void run() {
ProcessBuilder processbuilder = new ProcessBuilder(command);
LOGGER.info(command);
try {
Process process = processbuilder.inheritIO().start();
System.out.println("inside process builder ");
process.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String readline;
while ((readline = reader.readLine()) != null) {
LOGGER.info(readline);
}
// getting the thread state and adding it to a collection
Thread.State state = Thread.currentThread().getState();
if (state == Thread.State.TERMINATED) {
map.put(threadid, "TERMINATED");
}
} catch (Exception e) {
LOGGER.error(e.getMessage());
}
}
}
Is my implementation wrong. 我的实现是错误的。 Can someone help me with the implementation.
有人可以帮我实施。
Instead of trying to capture the outcome of a thread in the thread (which is error prone esp if an exception/error is thrown) I suggest you retain the Future objects and inspect those. 我建议您保留Future对象并检查它们,而不是尝试捕获线程中线程的结果(如果抛出异常/错误,则容易出错)。
ExecutorService exec = Executors.newFixedThreadPool(5);
System.out.println("at executors step");
Map<String, Future<?>> results = new HashMap<>();
for (String element : getList()) {
String[] command = { "ssh", "hddev-c01-edge-02", "\"" + element + "\"" };
results.put(element, exec.submit(new ParallelExecutor(command, thread_id, map)));
}
for(Map.Entry<String, Future<?>> entry: map.entrySet()) {
try {
entry.getValue().get();
System.out.println(entry.getKey()+ " is complete");
} catch (ExecutionException e) {
System.out.println(entry.getKey()+ " failed with");
e.getCause().printStackTrace(System.out);
}
}
ThreadPoolExecutor does not terminate untill it is asked to do so. ThreadPoolExecutor直到被要求这样做时才终止。 So, first of all you have to call
所以,首先你必须打电话
// executors.shutdown();
, which you kept as commented. ,您将其保留为已评论。 2nd, you need to wait for the threads to terminate properly.
2,您需要等待线程正确终止。 for that add a loop, before for(Map.Entry entry: map.entrySet())
为此,在for(Map.Entry entry:map.entrySet())之前添加一个循环
while (!es.isTerminated()) {
}
But, since one thread will probably run many runnables and if I get you correctly you want to update the CHM once one Runnable is done with it's execution. 但是,由于一个线程可能会运行许多可运行对象,并且如果我正确理解了您的信息,那么一旦执行了一个可运行对象,您就想更新CHM。
To do that, you have to use a CustomThread class. 为此,您必须使用CustomThread类。 extends Thread and override just 1 method,
afterExecute()
where from you need to put code to update CHM with Runnable 's id and terminated status. 扩展线程并仅覆盖
afterExecute()
方法中的1个方法,您需要在其中放置代码以使用Runnable的ID和终止状态更新CHM。 But remember this means completion of passed Runnables run() method, not the underlying Thread's termination. 但是请记住,这意味着完成传递的Runnables run()方法,而不是基础线程的终止。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.