[英]Executorservice and Runnable
I have a list containing data to compute for each pixel (eg list size = 1024x768). 我有一个列表,其中包含要为每个像素计算的数据(例如,列表大小= 1024x768)。 Now I want to iterate multithreaded through the list and save the computation for each pixel in a HashMap. 现在我想在列表中迭代多线程并保存HashMap中每个像素的计算。 But whatever I do, I can't manage to make it right. 但无论我做什么,我都无法做到这一点。 I tried several ways, my last was this one: 我尝试了几种方法,我的最后一种是这样的:
ConcurrentMap<T, Color> map = new ConcurrentHashMap<T, Color>();
ExecutorService pool = Executors.newFixedThreadPool(4);
Iterator<T> it = camera.iterator();
while (it.hasNext()) {
Runnable run = () -> {
int i = 0;
while (it.hasNext() && i < 1000) {
i++;
T cameraRay = it.next();
if (object.collide(cameraRay.getRay()) == null)
map.put(cameraRay, BG_COLOR);
else
map.put(cameraRay, this.shader.shade(cameraRay.getRay(), object.collide(cameraRay.getRay())).getColor());
}
};
pool.execute(run);
}
pool.shutdown();
try {
if (pool.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS)) {
System.out.println("Mapsize: " + map.size());
// Draw Image:
map.forEach((ray, color) -> {image.setColor(ray, color);});
}
} catch (InterruptedException e) {
e.printStackTrace();
}
Note thet the iterators hasNext()
Method is synchronized. 注意迭代器hasNext()
方法是同步的。 The Problem is sometimes a heap problem or simply that the size of the HashMap is less than the list size. 问题有时是堆问题,或者只是HashMap的大小小于列表大小。
I guess that I didn't understand something correct concering Runnables or the ExecutorService. 我想我不明白一些正确的构思Runnables或ExecutorService。
I appreciate any help here. 我感谢任何帮助。
EDIT: I added a System.out.println(i)
just before the i++
statement. 编辑:我在i++
语句之前添加了一个System.out.println(i)
。 Despite of checking for i < 1000
at some point suddenly there appears the following: 尽管在某个时刻突然检查到i < 1000
,但仍然出现以下情况:
507
169
86624
625
626
Exception in thread "pool-2-thread-2" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-2-thread-3" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(Unknown Source)
at com.sun.javafx.application.LauncherImpl.launchApplication(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.launcher.LauncherHelper$FXHelper.main(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.ThreadPoolExecutor.addWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
at raytracer.impl.ParallelRenderer.render(ParallelRenderer.java:78)
at raytracer.ImageViewer.main(ImageViewer.java:118)
... 11 more
Exception in thread "pool-2-thread-4" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
at raytracer.impl.TriangleImpl.collide(TriangleImpl.java:87)
at raytracer.impl.SimpleScene.collide(SimpleScene.java:27)
at raytracer.impl.ParallelRenderer.lambda$0(ParallelRenderer.java:71)
at raytracer.impl.ParallelRenderer$$Lambda$48/24559708.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
EDIT 2: According to the answer of Warkst, I tried the following 编辑2:根据Warkst的回答,我尝试了以下内容
Iterator<T> it = camera.iterator();
List<T> buffer = new ArrayList<T>(1000);
while (it.hasNext()) {
buffer.add(it.next());
if (buffer.size() >= 1000 || !it.hasNext()) {
Runnable run = () -> {
for (T cameraRay : buffer) {
if (object.collide(cameraRay.getRay()) == null) // No collision
map.put(cameraRay, BG_COLOR);
else
map.put(cameraRay, this.shader.shade(cameraRay.getRay(), object.collide(cameraRay.getRay())).getColor());
}
};
pool.execute(run);
buffer.clear();
}
}
But the very strange is, that the Runnable
block is never entered now, why? 但非常奇怪的是, Runnable
块现在永远不会进入,为什么呢?
What confuses me is that your runnables all use the same iterator. 令我困惑的是你的runnables都使用相同的迭代器。 What surprises me even more is that you spawn runnables while iterating over the iterator, but that those runnables ALSO manipulate the iterator. 更令我惊讶的是,你在迭代迭代器时产生了runnables,但那些runnables还操纵迭代器。 This code can (and will, as proven by your question) lead to a bunch of race conditions and the consequent headaches. 这段代码可以(而且,正如您的问题所证明的那样)会导致一系列竞争条件和随之而来的麻烦。
I would suggest the following: 我建议如下:
Assuming your processing of the data is (significantly) slower than iterating once over the camera, this should do the trick. 假设您对数据的处理(显着)慢于在相机上迭代一次,这应该可以解决问题。 If that's not the case, there's really no reason to be multithreading. 如果不是这样,那么就没有理由进行多线程处理。
update 2 更新2
I've updated my code sample to something that works: 我已将我的代码示例更新为有效的内容:
public static void main(String[] args) throws InterruptedException {
ConcurrentHashMap<Integer, String> map = new ConcurrentHashMap<>();
ExecutorService pool = Executors.newFixedThreadPool(4);
Iterator<Integer> it = getIt();
Task t = new Task(map);
while (it.hasNext()) {
t.add(it.next());
if (t.size()>=1000 || !it.hasNext()) {
pool.submit(t);
t = new Task(map);
}
}
pool.shutdown();
pool.awaitTermination(1, TimeUnit.DAYS);
// Breakpoint here to inspect map
System.out.println("Done!");
}
With 同
private static Iterator<Integer> getIt(){
return new Iterator<Integer>() {
private int nr = 0;
@Override
public boolean hasNext() {
return nr < 20000;
}
@Override
public Integer next() {
return nr++;
}
};
}
And 和
private static class Task extends ArrayList<Integer> implements Runnable{
private final ConcurrentHashMap<Integer, String> map;
public Task(ConcurrentHashMap<Integer, String> map) {
this.map = map;
}
@Override
public void run() {
try{
for (Integer i : this) {
// Simulate some crunching: write to the stdout in 10 iterations for each number: 10 000 prints for each Runnable
for (int j = 0; j < 10; j++) {
System.out.println("Iteration "+j+" for "+i);
}
// Store something in the map, namely that this Integer, or T in your case, has been processed
map.put(i, "Done");
}
} catch(Exception e){
e.printStackTrace();
}
}
}
The breakpoint is hit after about 20-30 seconds and the map contains all Integers paired with the String "Done". 断点在大约20-30秒后被击中,并且地图包含与字符串“Done”配对的所有整数。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.