Multiple While loops in java method

Question

I have a java method which writes content into a text file with values de-limited by | symbol. Contents needs to be picked up from 10 tables depending on conditions and write into file. Currently am doing the following method. Could anyone please suggest a better alternate approach for doing this requirement. Does this method have performance bottleneck

    public static void createFile()
    {
        queryFromTable1
        whileLoopForqueryFromTable1
        {
            writer.write(value1+"|"+value2+"|".....)
        }
        queryFromTable2
        whileLoopForqueryFromTable2
        {
            writer.write("||||"+value4+"|".....)
        }
        queryFromTable2
        whileLoopForqueryFromTable2
        {
            writer.write("||"+value5+"|".....)
        }
    }

Answer 1

A possible bottleneck is if any of the queries are slow, in that case all of the remaining queries have to wait for the earlier to complete. Another possible bottleneck is that the algorithm is single threaded.

To solve that a solution would be to do the reads in parallel and process the writing in multiple writers. When all of that has been completed, simply merge the writers in the correct order (single threaded). The Java 8 class CompletableFuture provides some nice features that can be used here. Simply create some futures that completes and merge the output.

Check out the CompletableFuture JavaDocs for more info.

An example of the algorithm could be something like code below. Please note that this is simply an example and not a full-fledged solution. The use of the StringWriter is just for convencience and is just one way of handling the data.

public class AlgorithmTest {
    public static void main(String[] args) 
            throws IOException, ExecutionException, InterruptedException {

        // Setup async processing of task 1
        final CompletableFuture<String> q1 = CompletableFuture.supplyAsync(() -> {
            // Setup result data
            StringWriter result = new StringWriter();

            // execute query 1

            // Process result
            result.write("result from 1");

            // Return the result (can of course be handled in other ways).
            return result.toString();
        });

        // Setup async processing of task 2
        final CompletableFuture<String> q2 = CompletableFuture.supplyAsync(() -> {
            // Setup result data
            StringWriter result = new StringWriter();

            // execute query 2

            // Process result
            result.write("result from 2");

            // Return the result (can of course be handled in other ways).
            return result.toString();
        });

        // Write the whole thing to file (i.e. merge the streams)
        final Path path = new File("result.txt").toPath();
        Files.write(path, Arrays.asList(q1.get(), q2.get()));
    }
}

Answer 2

You can create separate method for extracting data from tables

private List<String> getDataFromTable("select * from table1"){...}

The obvious bottleneck is string concatenation value1+"|"+value2+"|" . You'd better use single write for each element

for(int i=0;i<tableData.size();i++){
   String str = tableData.get(i);
   if(checkPassed(str)){
      writer.write(str);
      // don't print last |
      if(i<tableData.size()-1)writer.write(DELIMITER); // private static final String DELIMITER = "|";
   }
}

More information will allow us to give better answer.

Answer 3

There is no performance bottleneck in this pseudo code.

If you are using a BufferedWriter I think it's ok to call write many times. You just have to not forget to close the writer at the end of treatments.

Now we don't know what is behind your DB query. Maybe they can be optimized. Do you use Prepared Statements ?

Answer 4

Try breaking down to several methods, below is pseudo code:

void createFile() {
    writeTo(out, query1);
    writeTo(out, query2);
    writeTo(out, query3);
    ....
}

void writeTo(out, query) {
    execute query
    loop(){
      out.write(...)
    }
}