使用 Python 在 SQL Server 中将表从一个数据库复制到另一个数据库

Question

Does anybody know of a good Python code that can copy large number of tables (around 100 tables) from one database to another in SQL Server?

I ask if there is a way to do it in Python, because due to restrictions at my place of employment, I cannot copy tables across databases inside SQL Server alone.

Here is a simple Python code that copies one table from one database to another. I am wondering if there is a better way to write it if I want to copy 100 tables.

print('Initializing...')

import pandas as pd
import sqlalchemy
import pyodbc

db1 = sqlalchemy.create_engine("mssql+pyodbc://user:password@db_one")
db2 = sqlalchemy.create_engine("mssql+pyodbc://user:password@db_two")

print('Writing...')

query = '''SELECT * FROM [dbo].[test_table]'''
df = pd.read_sql(query, db1)
df.to_sql('test_table', db2, schema='dbo', index=False, if_exists='replace')

print('(1) [test_table] copied.')

Answer 1

SQLAlchemy is actually a good tool to use to create identical tables in the second db:

table = Table('test_table', metadata, autoload=True, autoload_with=db1)
table.create(engine=db2)

This method will also produce correct keys, indexes, foreign keys. Once the needed tables are created, you can move the data by either select/insert if the tables are relatively small or use bcp utility to dump table to disk and then load it into the second database (much faster but more work to get it to work correctly)

If using select/insert then it is better to insert in batches of 500 records or so.

Answer 2

You can do something like this:

tabs = pd.read_sql("SELECT table_name FROM INFORMATION_SCHEMA.TABLES", db1)

for tab in tabs['table_name']:
    pd.read_sql("select * from {}".format(tab), db1).to_sql(tab, db2, index=False)

But it might be be awfully slow. Use SQL Server tools to do this job.

Consider using sp_addlinkedserver procedure to link one SQL Server from another. After that you can execute:

SELECT * INTO server_name...table_name FROM table_name

for all tables from the db1 database.

PS this might be done in Python + SQLAlchemy as well...

Answer 3

MUPSAT 我尝试了您的技术来复制 sql server 数据库，但这仅适用于一个表，如果我想创建整个数据库的表，那么表的问题序列会发生，就像正在创建订单表但它引用另一个表是尚未创建它会给我一个错误，因此任何避免这种情况的技术都可以创建数据库的完整副本及其密钥和其他约束