[英]ConcurrentQueue Operation is giving OutofMemory Error.
I understand this question may be too general. 我了解这个问题可能太笼统了。 But I tried many things and I am not able to figure out how to resolve this. 但是我尝试了很多事情,但我不知道该如何解决。
I am using ConcurrentQueue for multithreading operation. 我正在使用ConcurrentQueue进行多线程操作。 One thread is downloading Images from server and saving it to queue. 一个线程正在从服务器下载图像并将其保存到队列。 Here is the code for that: 这是该代码:
public static void DownloadImage()
{
string baseUrl = "http://someurl";
//int numIterations = 5;
HttpWebRequest request = null;
foreach (var fileName in fileNames)
{
string url = string.Format(baseUrl, fileName);
request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
var response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
img = Image.FromStream(stream);
ImageFileName FileNameImage = new ImageFileName(fileName, img);
ImageQueue.Enqueue(FileNameImage);
Console.WriteLine("Count after Enqueue: {0}", ImageQueue.Count);
}
And another thread takes images from Queue and saves them on destination folder. 另一个线程从队列中获取图像并将其保存在目标文件夹中。 Here is the code for that: 这是该代码:
public static void SaveImage()
{
while (true)
{
if (!ImageQueue.IsEmpty)
{
foreach (var newobject2 in ImageQueue)
{
Image img2 = newobject2.Image;
img2.Save("C:\\path" + newobject2.ImageName);
ZoomThumbnail = img2;
ZoomSmall = img2;
ZoomLarge = img2;
ZoomThumbnail = GenerateThumbnail(ZoomThumbnail, 86, false);
ZoomSmall = GenerateThumbnail(ZoomSmall, 400, false);
ZoomLarge = GenerateThumbnail(ZoomLarge, 1200, false);
ZoomThumbnail.Save("C:\\path" + newobject2.ImageName + "_Thumb.jpg");
ZoomSmall.Save("C:\\path" + newobject2.ImageName + "_ZoomSmall.jpg");
ZoomLarge.Save("C:\\path" + newobject2.ImageName + "_ZoomLarge.jpg");
ImageFileName imgobject3 = new ImageFileName();
ImageQueue.TryDequeue(out imgobject3);
Console.WriteLine("Count after Deque: {0}", ImageQueue.Count);
}
}
}
}
I am calling these two threads from Button_Click() like this: 我从Button_Click()调用这两个线程,如下所示:
Thread DownloadThread = new Thread(DownloadImage);
DownloadThread.Start();
Thread SaveThread = new Thread(SaveImage);
SaveThread.Start();
I am getting MemoryFull error whenever queue reaches count of 68. I am not sure how I can avoid that. 每当队列数达到68时,我都会收到MemoryFull错误。我不确定如何避免这种情况。 I have tried using Thread.Sleep to avoid this. 我尝试使用Thread.Sleep来避免这种情况。 For Example, I tried: Thread.Sleep(500)
after foreach
loop. 例如,我尝试了: foreach
循环后的Thread.Sleep(500)
。 Whenever I try it inside foreach
it works totally fine as at any given point ImageQueue.Count = 1
. 每当我在foreach
尝试它时,在任何给定点ImageQueue.Count = 1
它都可以正常工作。 Where I am getting wrong? 我哪里出错了?
You never actually remove your images from the queue. 您实际上从未从队列中删除图像。 You iterate through them, but you never dequeue them. 您遍历它们,但从未使它们出队。 You also have the problem that when enumerating the queue the enumerable will stop whenever there are currently no more items in the queue. 您还有一个问题,当枚举队列时,只要队列中当前没有更多项目,可枚举将停止。 You don't want to do this, you may not be done yet. 您不想这样做,可能还没有完成。
What you want to do is use a BlockingCollection
instead of a ConcurrentQueue
, so that if there are currently no more items you wait for more, rather than stopping. 您要做的是使用BlockingCollection
而不是ConcurrentQueue
,这样,如果当前没有更多的项目,您可以等待而不是停止。 Once you do this, you can use foreach(var item in queue.GetConsumingEnumerable())
. 完成此操作后,可以使用foreach(var item in queue.GetConsumingEnumerable())
。 This makes all of the needed changes. 这将进行所有需要的更改。 It removes the items from the queue as you iterate it, and it waits for more items if there are currently none. 它会在您迭代时从队列中删除项目,如果当前没有项目,它会等待更多项目。
In addition to making this change, as is mentioned in Guffa's answer, you need to dispose of your image objects. 正如Guffa的答案所述,除了进行此更改外,您还需要处理图像对象。 You can dispose of them at the end of your loop body, once you're done with the image. 完成图像处理后,可以将它们放在循环主体的末尾。
You are creating a lot of Image
objects that you don't dispose. 您正在创建许多不处理的Image
对象。 You should call Dispose
on each of the thumbnails that you create. 您应该在创建的每个缩略图上调用“ Dispose
”。
Also, the object that you dequeue contains an Image
object that you should dispose when you don't need it any more. 而且,您要出队的对象包含一个Image
对象,当不再需要它时,应该将其处置。
Side note: You shouldn't enumerate the items in the queue and deqeue the item inside the loop. 旁注:您不应该枚举队列中的项目并在循环内将项目取名。 Instead use the TryDequeue
method in the loop itself: 而是在循环本身中使用TryDequeue
方法:
ImageFileName imgobject2;
while (ImageQueue.TryDequeue(out imgobject2)) {
...
}
Another side note: Your code will go into a tight loop when the queue is empty, using a lot of CPU power to repeatedly make the same check very rapidly. 另一个注意事项:当队列为空时,您的代码将进入一个紧密循环,这需要大量的CPU功能来非常快速地重复进行相同的检查。 You should use a mechanism to suspend the thread until something happens with the queue, or at least sleep for a while before checking the queue again. 您应该使用一种机制来挂起线程,直到队列发生故障,或者至少休眠一会儿再重新检查队列。
You should dispose after you are done with the image so after the ZoomLarge statement because after that you dont make any use of it anymore. 在处理完图像之后,应该在ZoomLarge语句之后进行处理,因为在此之后,您将不再使用它。 With the Generate statements you create new images. 使用Generate语句可以创建新图像。
1.It seems that your are keeping a reference to all your images which keeps them in memory, not allowing the garbage collector to collect them. 1,似乎您正在引用所有图像以将其保留在内存中,而不允许垃圾收集器收集它们。
Also, after you've finished using your objects, you can explictly dispose them, like this: 另外,在完成使用对象之后,可以像下面这样显式地处置它们:
Image img2 = newobject2.Image;
img2.Dispose();
2. What your are doing is enumerating over all your images everytime you are in a while loop, i am not sure this is what you want. 2.您正在做的是每次在while循环中都枚举所有图像,我不确定这是您想要的。 What you should do is try to dequeue an item: 您应该做的是尝试使项目出队:
FileImageName myImage;
ImageQueue.TryDequeue(out myImage);
if (myImage != null)
{
// Do work on the image
}
You might want to look at the BlockingCollection
class ( here ), which I've successfully used for this kind of "producer-consumer" pattern. 您可能要看一下BlockingCollection
类( 在此 ),我已经成功地将其用于这种“生产者-消费者”模式。
Something (a producer) puts items into the collection, with one or more "consumer" b/g threads taking the next available item(s) out of the collection and doing something with them. 某人(生产者)将项目放入集合中,一个或多个“消费者” b / g线程将下一个可用项目从集合中移出并对其进行处理。 You can also configure the number of consumer threads to use (eg for a multi-core PC). 您还可以配置要使用的使用者线程数(例如,用于多核PC)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.