I have written a console application to process image files and match them to database records.
The image files are all located in a file store in different folders one folder per day so the structure of the filestore would be
FileStore -> Day1 -> Images -------------------> Indexfile.csv
So my program will open each folder grab the indexfile and match the image to a record in the database.
The folders are quite large some of them having 90000+ images, so I would like to run up to 5 different task each grabbing a different folder and processing them in parallel, I've tried creating 5 tasks and that work fine except that I don't know how to start a new task once one of the 5 tasks has finish.
Any pointers would be appreciated.
Thanks.
The easy way is to create a list with all the files and use Parallel.ForEach
, checkout the docs https://msdn.microsoft.com/en-us/library/dd460720%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396
Example Code:
// A simple source for demonstration purposes. Modify this path as necessary.
String[] files = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
String newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
System.IO.Directory.CreateDirectory(newDir);
// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
// Be sure to add a reference to System.Drawing.dll.
Parallel.ForEach(files, (currentFile) =>
{
// The more computational work you do here, the greater
// the speedup compared to a sequential foreach loop.
String filename = System.IO.Path.GetFileName(currentFile);
var bitmap = new Bitmap(currentFile);
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
bitmap.Save(Path.Combine(newDir, filename));
// Peek behind the scenes to see how work is parallelized.
// But be aware: Thread contention for the Console slows down parallel loops!!!
Console.WriteLine("Processing {0} on thread {1}", filename, Thread.CurrentThread.ManagedThreadId);
//close lambda expression and method invocation
});
// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();
You should use WhenAny
(non-blocking) or WaitAny
(blocking) in list of tasks. then remove any completed task from list and add new ones.
List<Task<...>> tasks = ... // put your 5 tasks inside this list.
while(condintion)
{
await Task.WhenAny(tasks);
tasks.RemoveAll(t => t.IsCompleted);
while(tasks.Count < 5 && condintion)
{
Task<...> newTask = ... // run your task
tasks.Add(newTask);
}
}
put the appropriate condition so that is only false when all folders are taken.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.