How to run threads in parallel on priority basis in Java?

Question

I want to read csv files from folder and persists their data into database. We can achieve this in a single thread model too but I want to implement it using multithreading. So I have devided the tasks into 4 threads:

Thread 1: Look for files from folder and if it found the suitable file then pick it and put in temp folder and remove the old files.

Thread 2: Read the file from temp folder and create list of data from files. Also create output and error data based on some validations.

Thread 3 [Low Priority]: Check for output and error data and write it into output and error log files.

Thread 4: Checks for data in the list and if found data, inserts them into database.

For all threads, I want to put a check if there is not input data then that thread goes into wait and notify others also once completed work it controll should handover to other thread. And all these threads should execute in countinue bases.

As I am not much familier with multithreading concept, please suggest me how to achieve this scenario.

Answer 1

You have a problem and you want to use multi-threading to solve it. Now you have two problems. :)

First of all you have to look at a way to decompose your problem into something that can be done in parallel. The way you have decomposed your problem right now is inherently sequential. There is no advantage to multi-threading when you decompose it this way because the through put will be limited by the slowest step in the process.

A better way to decompose your problem would be to break up the job into tasks that are repeatedly executed. The slowest, but also most easily parallel part is reading and uploading the files. You can do this for each file in parallel. This allows you to leverage Javas High Level Concurrency Objects which will safe you a world of pain.

Thread 1: Watch input folder for files. When a new file appears, create a task. This task is submitted to a Task Executor . The executor will then execute the task on its own thread using one of the available threads.

Task: For a given file a task reads, parses and validate the data in the file. When the data is valid write it to the database and delete/move the original file.

Thread 2: Watch the task executor for completed tasks. When a task is complete, read its information and write a report or log an error if something went wrong.

Thread 3: Since you have tasks going on in parallel you need something to monitor user requests to stop the service gracefully. You don't want to stop the service halfway through writing a report.

Logging: Logging is a separate concern but has nothing (much) to do with threading. You should use a logging framework for this. Most frameworks can handle logging from multiple threads.

Finally note that your problem doesn't sound like a problem that would benefit much form parallel processing. Reading files, and uploading them to a database are both IO operations and IO operations are glacially slow compared to other computations. They're also the bottle neck for all parallel processes. You can't read files faster then your disk can provide them and you can't upload faster then your connection can handle. So you have to ask yourself if the marginally improved speed is worth the added complexity.

Answer 2

you can use Thread3.setPriority(Thread.MIN_PRIORITY); priority is int number 1-10 1 is the minimum and 10 is the maximum