How to copy data from (PL/)SQL in Oracle to another DBMS?

Question

Task : copy all data in a database (without schema) to another database (possibly of a different type). I can't modify source database, so it is a read-only backup

Context : integrate Oracle with a number of DBs. Now I'm integrating Oracle and Postgres.

Resources : connection string only, with ability to connect database with highest available privileges. (I can't access it via ssh - no way to create ordinal backup and download files via ssh, or to compile and start web/ftp server, etc.)

Question : Is there any proven and FAST way to pull this data? Maybe someone has an open source solution with clean code?

The word "fast" is present here because just selecting N rows in a turn (using rownum or row_number()) and transfering to a target database or intermediate file is too slow.

Answer 1

I suggest you take a look at Liquibase . I have used it successfully to keep both schemas and data aligned across several environments (albeit only SQLServers, but I'm certain it works for disparate RDBMS' as well).

As for the performance I am a bit worried, as you mention "terabytes of data"... Still, it might be worth a try.

Cheers,

Answer 2

A light-weight ETL tool like spring-batch might be the perfect tool for this task.

Micromanagement of data chunks is what it is written around.

Take a look at the jdbc cursor example: you just configure the SELECT and INSERT statement, and the mapping, and Spring-Batch will take care of the pagination.

You can find it in github: https://github.com/SpringSource/spring-batch/blob/master/spring-batch-samples/src/main/resources/jobs/iosample/jdbcCursor.xml

Reference can be found at: http://static.springsource.org/spring-batch/reference/html/readersAndWriters.html#database

Spring batch keeps state of how many records already have been processed, and allows to continue a previously crashed process. It does this by saving counters in a 'jobRepository', which can be in a third database, for example.

Of course, this is a pure java solution, a native solution may be faster, but if you only get JDBC connection strings, you might give this a shot. This also assumes you know the table structure of all tables you want to transfer. If not, simple JDBC tools like eg SquirrelSQL can help you.

Greets, Geert.