Workbook takes long time to generate excel file

Question

I'm trying to generate excel file with 200k records. But it is taking almost 2 hours to generate the file.

Here is my code of generating excel file.

Workbook workbook=null;

csvFileName = userId+"_Records_"+new SimpleDateFormat("yyyyMMddHHmmss")
         .format(new Date())+".xls";
path = ReadPropertyFile.getProperties("download.reports.path");



 misService.insertXLSRecord(ackNo,"-",null, VspCommonConstants.getIpFromRequest(request),
    new Date(), userId,"N",userReportRoleId);

  workbook = getWorkbook(path+csvFileName);


 Sheet sheet = workbook.createSheet(WorkbookUtil.createSafeSheetName(studAppForm.get(0)
    .getScheme_Id()+"_"+studAppForm.get(0).getEFP_Scholarship_Name(),'_'));

 if(schemeQuestionData.containsKey(currSheetSchemeId))
                   createXLSHeaders(sheet,schemeQuestionData.get(currSheetSchemeId));

 Row row = sheet.createRow(++rowCount);


currAppId=studAppForm.get(j).getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(new BigInteger(currAppId)));
writeBook(studAppForm.get(j), row);

Here is my createXLSHeaders method to create header

void createXLSHeaders( Sheet sheet, List<SchemeMasterBean> schemeMasterBeanList){

       LOGGER.info("Creating XLS SheetHeaders for sheet "+sheet.getSheetName());

     //   Sheet sheet = workbook.createSheet();
        Row header = sheet.createRow(0);
        header.createCell(0).setCellValue("APPLICATION ID");
        header.createCell(1).setCellValue("APPLICATION STATUS");
        header.createCell(2).setCellValue("APPLICATION DATE");
        header.createCell(3).setCellValue("SCHEME/SCHOLARSHIP APPLIED");
        header.createCell(4).setCellValue("SCHEME ID");
        header.createCell(5).setCellValue("STUDENT ID");
        header.createCell(6).setCellValue("STUDENT FULL NAME");
        .
        .
        .
        62 heading...

        int i=73;
        if(schemeMasterBeanList!=null)
        for(SchemeMasterBean schemeMasterBean :schemeMasterBeanList){
               if(!schemeMasterBean.getSmSchemeType().equals("5") && 
                   !schemeMasterBean.getSmSchemeType().equals("6")){
               header.createCell(i).setCellValue(schemeMasterBean.getSmScholarshipName());
               i++;
               }
        }
    }

and finally writebook method

private void writeBook(StudentAppFormVsp saf, Row row) throws JSONException {


    Cell cell = row.createCell(0);
    cell.setCellValue(saf.getApp_Id()!=null?saf.getApp_Id().toString():"");
    cell = row.createCell(1);
    cell.setCellValue(saf.getApp_Status()!=null?getApplicationStatusMap().get(saf.getApp_Status()):"");
    cell = row.createCell(2);
    cell.setCellValue(saf.getCrtn_time()!=null?saf.getCrtn_time().toString():"");
    cell = row.createCell(3);
    cell.setCellValue(saf.getEFP_Scholarship_Name()!=null?saf.getEFP_Scholarship_Name().toString():"");
    cell = row.createCell(4);
    cell.setCellValue(saf.getScheme_Id()!=null?saf.getScheme_Id().toString():"");
    cell = row.createCell(5);
    cell.setCellValue(saf.getStud_Id()!=null?saf.getStud_Id().toString():"");

                  .
                  .
                  62 rows

}

How to reduce the excel sheet generation time?

Answer 1

First: play around with memory for the application if possible.

Then: the tip on using a profiler is really worth the effort.

Any DOM, XML, Excel or otherwise often suffer from location references searching from top to the actual position.

Creating a DOM instead of writing sequentially is costly with respect to memory, and can slow things down. Maybe consider this .

You could make two loop: writing to a CSV file, and then creating an XLS(X). Then you know where the complexity resides.

The following (I rewrote a bit) is slightly suspect: toString + new BigInteger points to a conversion; I hope not from BigInteger to String to BigInteger.

StudentAppFormVsp saf = studAppForm.get(j);
currAppId = saf.getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(BigInteger.valueOf(currAppId)));
writeBook(saf, row);