How to convert files to PDF simultaneously?

Question

I have a node.js web application and I want to be able to convert many documents (in the same time) to PDF . In this moment I use libreoffice with a queue (The purpose of queue is to avoid infinite conversion for a file - if libreoffice cannot convert the file in a specific period of time then I kill the process). The problem is that if I have two users which upload files and both of them upload big files the second user must wait for the first one to finish.

Is there any way to convert the files simultaneously?

I am willing to replace libreoffice with another PDF converter.

Thank you in advance.

Answer 1

One way to do multiple conversions at the same time is to just fire up multiple processes that are doing conversions. Then, whenever something is added to the queue, it would be handed off to the next available external process that is working on conversions. Since the file conversion is likely a mix of CPU and disk I/O, you could experiment with the best number of conversion processes to maintain.

There are a number of ways to manage the queue. If your node.js app was maintaining the queue, then it would be managing the process of handing off an item from the queue to the external process and keeping track of which external processes were free and ready for a new job. If the queue was in a database or in the file system that all the external processes could read, then the external processes themselves could watch the queue.