如何将CopyManager与连接池数据源一起使用？

Question

I'm trying to use postgres CopyManager.copyIn() for batch inserts. My datasource is a c3p0 ComboPooledDataSource .

The sql statements are batch with:

dataSource.getConnection().getCopyAPI().copyIn(sql, items); //pseudocode

Now to speed up database inserts even more (hundred GB to import after pre-processing), I'm trying to send the copyIn command in async threads.

But does this make sense if the database is located on a single disk filesystem? Would this gain performance?

And how can I actually verify that the copyIn is using the connection pool in parallel?

I tried VisualVM MBeans screen, where i can see a single PooledDataSource entry. But how can I know that the pool is used and items are send to DB parallel?

Answer 1

But does this make sense if the database is located on a single disk filesystem? Would this gain performance?

If it's spinning rust it might not, and there's certainly little benefit in much concurrency. For SSDs it'll sometimes produce quite a significant improvement. Depends a lot on the drive.

I tried VisualVM MBeans screen, where i can see a single PooledDataSource entry. But how can I know that the pool is used and items are send to DB parallel?

Look at pg_stat_activity in the database and see if there are multiple concurrent COPY commands from the app.