Shall I process excel in database or process it using aspose.cells?

Question

I am trying to upload a very large excel file potentially containing millions of records to run a name cleaning process on one of its cell columns. I match the column names with a particular column in a table in SQL database and then inform the user to download the processed excel file.

I have multiple ways of doing it:

1) bulk copy the excel in database run the name cleaning process on the excel data in database and then extract the results from the table and write them to an excel and let the user download the excel.

2)upload the file and read it using aspose library and do the processing in memory and when once the operation is done intimate the user to download the file.

I am confused right now which option would be better and if there is a better approach of doing this please feel free to share.

Any leads would be really appreciated

Thanks

Answer 1

As you are talking about processing millions of records in memory using Aspose.Cells, it may affect the performance and memory utilization by Aspose.Cells. I think you should try both the methods and if you face some issue using Aspose.Cells, then let us know. I suggest you use LightCells API in Aspose.Cells which is best for reading and writing large data in Excel files.

https://docs.aspose.com/display/cellsnet/Using+LightCells+API

https://docs.aspose.com/display/cellsjava/Using+LightCells+API

Similarly, Excel may also cause issue while processing large files as it takes lot of time to process large files. Its matter of test and trial both the scenario and come up with a comparison.

One option is that if entire column data is to be used against a column name, then better to save single column excel file as blob in database and return the ready to use Excel file as it is.

You may try these scenarios and provide your feedback.

Note: I am working as Support developer/ Evangelist at Aspose.