Threads are not running simultaneously to read files

Question

I want to read multiple files through multi threading I wrote the code for the same but my threads are executing one by one which is very time consuming. I wants them to run simultaneously.

Please correct me what I am doing wrong in the below code where I am doing this by implementing the callable interface because I have to read the file and set its data into the variable of Model object and after that I am returning the list of objects.

Thanks In advance.

Class A{

ExecutorService executor = getExecuterService();

private ExecutorService getExecuterService() {
        int threadPoolSize = Runtime.getRuntime().availableProcessors() - 1;
        System.out.println("Number of COre" + threadPoolSize);
        return Executors.newFixedThreadPool(threadPoolSize);
    }

@SuppressWarnings({ "unused", "unchecked" })
                        FutureTask<List<DSection>> viewList = (FutureTask<List<DSection>>) executor
                        .submit(new MultiThreadedFileReadForDashboard(DashboardSectionList, sftpChannel,customQuery));

                        executor.shutdown();
                        while (!executor.isTerminated()) {

                        }

}

Class for task:

public class MultiThreadedFileReadForDashboard implements Callable {

public MultiThreadedFileReadForDashboard(List<DSection> dashboardSectionList, ChannelSftp sftpChannel,
            CustomQueryImpl customQuery) {

        this.dashboardSectionList = dashboardSectionList;
        this.sftpChannel = sftpChannel;
        this.customQuery = customQuery;
    }

    public List<DSection> call() throws Exception {

        for (int i = 0; i < dashboardSectionList.size(); ++i) { 

            DSection DSection = dashboardSectionList.get(i);
            List<LView> linkedViewList = new ArrayList<LView>(DSection.getLinkedViewList());

            LView lView;

            for (int j = 0; j < linkedViewList.size(); ++j) {
                lView = linkedViewList.get(j);
                int UserQueryId = Integer.parseInt(lView.getUserQueryId());

                outputFileName = customQuery.fetchTableInfo(UserQueryId);


                if ((outputFileName != null) && (!outputFileName.equalsIgnoreCase(""))) {

                    String data = readFiles(outputFileName);
                    lView.setData(data);


                } else {
                    lView.setData("No File is present");
                }

            }
            if (size == dashboardSectionList.size()) {
                break;
            }
        }

        return dSectionList;
    }

    private String readFiles(String outputFileName) {
        String response = null;
        try {

            InputStream in = sftpChannel.get(outputFileName);
            BufferedReader br = new BufferedReader(new InputStreamReader(in, "UTF-8"));
            StringBuilder inputData = new StringBuilder("");
            String line = null;
                while ((line = br.readLine()) != null) {
                inputData.append(line).append("\n");
            }

            JSONArray array = null;

            if (outputFileName.toLowerCase().contains("csv")) {
                array = CDL.toJSONArray(inputData.toString());
            } else {    
            }
            response = array.toString();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return response;
        // TODO Auto-generated method stub
    }
  }
}

Answer 1

I do not see read multiple files through multi threading. I see one task invoked by the ExecuterService and it is reading all the files. the multi threading feature is achieved by submitting multiple tasks to the ExecuterService , each is given one file to process (can be by constructor).

Here is what I think you should do:

inside the inner for loop, you construct a task that is given outputFileName in constructor and submit it to the executor, getting back a Future instance. after all tasks were submitted, you will have a List<Future> that you can query to see when they are done and get result. the task will call readFiles() (odd name for a method that reads one file...)