Best Way to compare a list of tables in a dynamic fashion from two DBs

Question

We have a use case wherein we need to compare 100s of tables between two databases (Oracle and AWS Redshift) in a summarized way

The tables are identical and we need to know if the tables are matching. Let me know if there is any easy way to compare the data in a performant way

Answer 1

The best way I have developed for doing this is to make md5 signatures of every column in every table across both databases and compare these signatures (or just combine column signatures to make table signatures). I had to do a bid of coding to make sure that NULLs and empty strings mis-compare and handle a few other corner cases but nothing too extreme.

Redshift can perform this signature analysis very quickly but the biggest issue I have run into in the past is the speed of other databases in computing signatures. As you asked "in a performant way". So I have had to write "light weight" hash functions in the past when source databases are just too wimpy to compute large numbers of md5s.