[英]Import MySQL dump to PostgreSQL database
如何将“xxxx.sql”转储从 MySQL 导入 PostgreSQL 数据库?
This question is a little old but a few days ago I was dealing with this situation and found pgloader.io .这个问题有点老了,但几天前我正在处理这种情况并找到了pgloader.io 。
This is by far the easiest way of doing it, you need to install it, and then run a simple lisp script ( script.lisp ) with the following 3 lines:这是迄今为止最简单的方法,您需要安装它,然后使用以下 3 行运行一个简单的 lisp 脚本 ( script.lisp ):
/* content of the script.lisp */
LOAD DATABASE
FROM mysql://dbuser@localhost/dbname
INTO postgresql://dbuser@localhost/dbname;
/*run this in the terminal*/
pgloader script.lisp
And after that your postgresql DB will have all of the information that you had in your MySQL SB.之后,您的 postgresql 数据库将拥有您在 MySQL SB 中拥有的所有信息。
On a side note, make sure you compile pgloader since at the time of this post, the installer has a bug.附带说明一下,请确保您编译 pgloader,因为在撰写本文时,安装程序存在错误。 (version 3.2.0) (版本 3.2.0)
Don't expect that to work without editing.不要指望它在没有编辑的情况下工作。 Maybe a lot of editing.也许很多编辑。
mysqldump has a compatibility argument , --compatible=name
, where "name" can be "oracle" or "postgresql", but that doesn't guarantee compatibility. mysqldump 有一个兼容性参数--compatible=name
,其中“name”可以是“oracle”或“postgresql”,但这并不能保证兼容性。 I think server settings like ANSI_QUOTES have some effect, too.我认为像 ANSI_QUOTES 这样的服务器设置也有一些影响。
You'll get more useful help here if you include the complete command you used to create the dump, along with any error messages you got instead of saying just "Nothing worked for me."如果您包含用于创建转储的完整命令以及您收到的任何错误消息,而不是仅仅说“对我没有任何帮助”,您将在此处获得更多有用的帮助。
brew update && brew install pgloader
pgloader mysql://user@host/db_name postgresql://user@host/db_name
For those Googlers who are in 2015+ .对于那些在 2015+ 年的 Google 员工。
I've wasted all day on this and would like to sum things up.我已经浪费了一整天,想总结一下。
I've tried all the solutions described at this article by Alexandru Cotioras (which is full of despair).我已经尝试了Alexandru Cotioras (充满绝望)在这篇文章中描述的所有解决方案。 Of all the solutions mentioned there only one worked for me.在提到的所有解决方案中,只有一个对我有用。
— lanyrd/mysql-postgresql-converter @ github.com (Python) — lanyrd/mysql-postgresql-converter @ github.com (Python)
But this alone won't do.但光靠这个是不行的。 When you'll be importing your new converted dump file:当您要导入新的转换后的转储文件时:
# \i ~/Downloads/mysql-postgresql-converter-master/dump.psql
PostgreSQL
will tell you about messed types from MySQL
: PostgreSQL
会告诉你来自MySQL
混乱类型:
psql:/Users/jibiel/Downloads/mysql-postgresql-converter-master/dump.psql:381: ERROR: type "mediumint" does not exist
LINE 2: "group_id" mediumint(8) NOT NULL DEFAULT '0',
So you'll have to fix those types manually as per this table.因此,您必须按照此表手动修复这些类型。
In short it is:简而言之就是:
tinyint(2) -> smallint
mediumint(7) -> integer
# etc.
You can use regex
and any cool editor to get it done.您可以使用regex
和任何很酷的编辑器来完成它。
MacVim
+ Substitute
: MacVim
+ Substitute
:
:%s!tinyint(\w\+)!smallint!g
:%s!mediumint(\w\+)!integer!g
The fastest (and most complete) way I found was to use Kettle.我发现的最快(也是最完整)的方法是使用 Kettle。 This will also generate the needed tables, convert the indexes and everything else.这还将生成所需的表、转换索引和其他所有内容。 The mysqldump
compatibility argument does not work.该mysqldump
兼容性参数不起作用。
The steps:步骤:
Download Pentaho ETL from http://kettle.pentaho.org/ (community version)从http://kettle.pentaho.org/ (社区版)下载 Pentaho ETL
Unzip and run Pentaho (spoon.sh/spoon.bat depending on unix/windows)解压并运行 Pentaho (spoon.sh/spoon.bat 取决于 unix/windows)
Create a new job创建新工作
Create a database connection for the MySQL source (Tools -> Wizard -> Create database connection)为 MySQL 源创建数据库连接(工具 -> 向导 -> 创建数据库连接)
Create a database connection for the PostgreSQL source (as above)为 PostgreSQL 源创建数据库连接(如上)
Run the Copy Tables
wizard (Tools -> Wizard -> Copy Tables)运行Copy Tables
向导(工具 -> 向导 -> 复制表)
Run the job运行作业
You can use pgloader.您可以使用 pgloader。
sudo apt-get install pgloader
Using:使用:
pgloader mysql://user:pass@host/database postgresql://user:pass@host/database
您可以潜在地从 MySQL 导出到 CSV ,然后将 CSV 导入到 PostgreSQL 中。
I have this bash script to migrate the data, it doesn't create the tables because they are created in migration scripts, so I need only to convert the data.我有这个 bash 脚本来迁移数据,它不会创建表,因为它们是在迁移脚本中创建的,所以我只需要转换数据。 I use a list of the tables to not import data from the migrations
and sessions
tables.我使用表列表来不从migrations
和sessions
表中导入数据。 Here it is, just tested:这是,刚刚测试:
#!/bin/sh
MUSER="root"
MPASS="mysqlpassword"
MDB="origdb"
MTABLES="car dog cat"
PUSER="postgres"
PDB="destdb"
mysqldump -h 127.0.0.1 -P 6033 -u $MUSER -p$MPASS --default-character-set=utf8 --compatible=postgresql --skip-disable-keys --skip-set-charset --no-create-info --complete-insert --skip-comments --skip-lock-tables $MDB $MTABLES > outputfile.sql
sed -i 's/UNLOCK TABLES;//g' outputfile.sql
sed -i 's/WRITE;/RESTART IDENTITY CASCADE;/g' outputfile.sql
sed -i 's/LOCK TABLES/TRUNCATE/g' outputfile.sql
sed -i "s/'0000\-00\-00 00\:00\:00'/NULL/g" outputfile.sql
sed -i "1i SET standard_conforming_strings = 'off';\n" outputfile.sql
sed -i "1i SET backslash_quote = 'on';\n" outputfile.sql
sed -i "1i update pg_cast set castcontext='a' where casttarget = 'boolean'::regtype;\n" outputfile.sql
echo "\nupdate pg_cast set castcontext='e' where casttarget = 'boolean'::regtype;\n" >> outputfile.sql
psql -h localhost -d $PDB -U $PUSER -f outputfile.sql
You will get a lot of warnings you can safely ignore like this:你会收到很多警告,你可以像这样安全地忽略:
psql:outputfile.sql:82: WARNING: nonstandard use of escape in a string literal
LINE 1: ...,(1714,38,2,0,18,131,0.00,0.00,0.00,0.00,NULL,'{\"prospe...
^
HINT: Use the escape string syntax for escapes, e.g., E'\r\n'.
pgloader
使用pgloader
Get a recent version of pgloader
;获取最新版本的pgloader
; the one provided by Debian Jessie (as of 2019-01-27) is 3.1.0 and won't work since pgloader
will error with Debian Jessie 提供的版本(截至 2019 年 1 月 27 日)是 3.1.0 并且无法工作,因为pgloader
会出错
Can not find file mysql://...
Can not find file postgres://...
First, make sure you can establish a connection to mysqld
on the server running MySQL using首先,确保您可以使用以下命令在运行 MySQL 的服务器上建立到mysqld
的连接
telnet theserverwithmysql 3306
If that fails with如果那失败了
Name or service not known名称或服务未知
log in to theserverwithmysql
and edit the configuration file of mysqld
.用theserverwithmysql
登录theserverwithmysql
,编辑mysqld
的配置文件。 If you don't know where the config file is, use find / -name mysqld.cnf
.如果您不知道配置文件在哪里,请使用find / -name mysqld.cnf
。
In my case I had to change this line of mysqld.cnf
就我而言,我不得不更改mysqld.cnf
这一行
# By default we only accept connections from localhost
bind-address = 127.0.0.1
to到
bind-address = *
Mind that allowing access to your MySQL database from all addresses can pose a security risk, meaning you probably want to change that value back after the database migration.请注意,允许从所有地址访问您的 MySQL 数据库可能会带来安全风险,这意味着您可能希望在数据库迁移后将该值改回。
Make the changes to mysqld.cnf
effective by restarting mysqld
.通过重新启动mysqld
使对mysqld.cnf
的更改生效。
Assuming you are logged in on the system that runs Postgres, create the database with假设您已登录运行 Postgres 的系统,请使用以下命令创建数据库
createdb databasename
The user for the Postgres database has to have sufficient privileges to create the schema, otherwise you'll run into Postgres 数据库的用户必须有足够的权限来创建模式,否则你会遇到
permission denied for database databasename数据库 databasename 的权限被拒绝
when calling pgloader
.调用pgloader
。 I got this error although the user had the right to create databases according to psql > \\du
.尽管用户有权根据psql > \\du
创建数据库,但我收到了此错误。
You can make sure of that in psql
:您可以在psql
确保这一点:
GRANT ALL PRIVILEGES ON DATABASE databasename TO otherusername;
Again, this might be privilege overkill and thus a security risk if you leave all those privileges with user otherusername
.同样,如果您将所有这些权限留给用户otherusername
,这可能是特权过度使用,因此存在安全风险。
Finally, the command最后,命令
pgloader mysql://theusername:thepassword@theserverwithmysql/databasename postgresql://otherusername@localhost/databasename
executed on the machine running Postgres should produce output that ends with a line like this:在运行 Postgres 的机器上执行应该产生以这样的行结尾的输出:
Total import time ✓ 877567 158.1 MB 1m11.230s
Mac/Win Mac/Win
Download Navicat trial for 14 days (I don't understand $1300) - full enterprise package:下载 Navicat 试用 14 天(我不明白 1300 美元)-完整的企业包:
connect both databases mysql and postgres连接两个数据库 mysql 和 postgres
menu - tools - data transfer菜单——工具——数据传输
connect both dbs on this first screen.在第一个屏幕上连接两个数据库。 While still on this screen there is a general / options - under the options check on the right side check - continue on error * note you probably want to un-check index's and keys on the left.. you can reassign them easily in postgres.虽然仍然在这个屏幕上有一个常规/选项 - 在右侧检查的选项检查下 - 继续错误 * 注意你可能想要取消选中左侧的索引和键......你可以在 postgres 中轻松地重新分配它们。
at least get your data from MySQL into Postgres!至少将您的数据从 MySQL 导入 Postgres!
hope this helps!希望这可以帮助!
It is not possible to import an Oracle (binary) dump to PostgreSQL.无法将 Oracle(二进制)转储导入 PostgreSQL。
If the MySQL dump is in plain SQL format, you will need to edit the file to make the syntax correct for PostgreSQL (eg remove the non-standard backtick quoting, remove the engine definition for the CREATE TABLE statements adjust the data types and a lot of other things)如果 MySQL 转储是纯 SQL 格式,您将需要编辑该文件以使 PostgreSQL 的语法正确(例如,删除非标准的反引号引用,删除 CREATE TABLE 语句的引擎定义,调整数据类型等其他东西)
Here is a simple program to create and load all tables in a mysql database (honey) to postgresql.这是一个简单的程序,用于创建 mysql 数据库(honey)中的所有表并将其加载到 postgresql。 Type conversion from mysql is coarse-grained but easily refined.来自 mysql 的类型转换是粗粒度的,但很容易细化。 You will have to recreate the indexes manually:您必须手动重新创建索引:
import MySQLdb
from magic import Connect #Private mysql connect information
import psycopg2
dbx=Connect()
DB=psycopg2.connect("dbname='honey'")
DC=DB.cursor()
mysql='''show tables from honey'''
dbx.execute(mysql); ts=dbx.fetchall(); tables=[]
for table in ts: tables.append(table[0])
for table in tables:
mysql='''describe honey.%s'''%(table)
dbx.execute(mysql); rows=dbx.fetchall()
psql='drop table %s'%(table)
DC.execute(psql); DB.commit()
psql='create table %s ('%(table)
for row in rows:
name=row[0]; type=row[1]
if 'int' in type: type='int8'
if 'blob' in type: type='bytea'
if 'datetime' in type: type='timestamptz'
psql+='%s %s,'%(name,type)
psql=psql.strip(',')+')'
print psql
try: DC.execute(psql); DB.commit()
except: pass
msql='''select * from honey.%s'''%(table)
dbx.execute(msql); rows=dbx.fetchall()
n=len(rows); print n; t=n
if n==0: continue #skip if no data
cols=len(rows[0])
for row in rows:
ps=', '.join(['%s']*cols)
psql='''insert into %s values(%s)'''%(table, ps)
DC.execute(psql,(row))
n=n-1
if n%1000==1: DB.commit(); print n,t,t-n
DB.commit()
As with most database migrations, there isn't really a cut and dried solution.与大多数数据库迁移一样,并没有真正的一劳永逸的解决方案。
These are some ideas to keep in mind when doing a migration:这些是在进行迁移时需要牢记的一些想法:
Keep these in mind.记住这些。 The best way is probably to write a conversion utility.最好的方法可能是编写一个转换实用程序。 Have a happy conversion!有一个快乐的转换!
I had to do this recently to a lot of large .sql files approximately 7 GB in size.我最近不得不对许多大约 7 GB 的大型 .sql 文件执行此操作。 Even VIM had troubling editing those.甚至 VIM 也很难编辑这些。 Your best bet is to import the .sql into MySql and then export it as a csv which can be then imported to Postgres.最好的办法是将 .sql 导入 MySql,然后将其导出为 csv,然后可以将其导入 Postgres。
But, the MySQL export as a csv is horrendously slow as it runs the select * from yourtable query.但是,MySQL 导出为 csv 的速度非常慢,因为它运行select * from yourtable查询。 If you have a large database/table I would suggest using some other method.如果您有一个大型数据库/表,我建议您使用其他方法。 One way is to write a script that reads the sql inserts line by line and uses string manipulation to reformat it to "Postgres-compliant" insert statements and then execute these statements in Postgres一种方法是编写一个脚本,逐行读取 sql inserts 并使用字符串操作将其重新格式化为“符合 Postgres”的插入语句,然后在 Postgres 中执行这些语句
I could copy tables from MySQL to Postgres using DBCopy Plugin for SQuirreL SQL Client .我可以使用DBCopy Plugin for SQuirreL SQL Client将表从 MySQL 复制到 Postgres。 This was not from a dump, but between live databases.这不是来自转储,而是来自实时数据库。
Use your xxx.sql file to set up a MySQL database and make use of FromMysqlToPostrgreSQL .使用 xxx.sql 文件设置 MySQL 数据库并使用FromMysqlToPostrgreSQL 。 Very easy to use, short configuration and works like a charm.非常易于使用,配置简短,工作起来很迷人。 It imports your database with the set primary keys, foreign keys and indices on the tables.它使用表上设置的主键、外键和索引导入您的数据库。 You can even import data alone if you set appropriate flag in the config file.如果您在配置文件中设置了适当的标志,您甚至可以单独导入数据。
FromMySqlToPostgreSql migration tool by Anatoly Khaytovich, provides an accurate migration of table data, indices, PKs, FKs... Makes an extensive use of PostgreSQL COPY protocol. Anatoly Khaytovich 的 FromMySqlToPostgreSql 迁移工具,提供表数据、索引、PK、FK 的准确迁移......广泛使用 PostgreSQL COPY 协议。
See here too: PG Wiki Page参见此处: PG Wiki 页面
如果您使用的是 phpmyadmin,您可以将数据导出为 CSV,然后在 postgres 中导入会更容易。
1.Take a duump file of mysql database. 1.取一个mysql数据库的转储文件。 2.use this tool for converting loacal mysql database to local postgresql database. 2.使用此工具将本地mysql数据库转换为本地postgresql数据库。
"source": { "host": "localhost", "port": 3306, "database": "test_db", "charset": "utf8mb4", "supportBigNumbers": true, "user": "root", "password": "0123456789" } "target": { "host" : "localhost", "port" : 5432, "database" : "test_db", "charset" : "UTF8", "user" : "postgres", "password" : "0123456789" } "source": { "host": "localhost", "port": 3306, "database": "test_db", "charset": "utf8mb4", "supportBigNumbers": true, "user": "root", "密码”:“0123456789”}“目标”:{“主机”:“本地主机”,“端口”:5432,“数据库”:“test_db”,“字符集”:“UTF8”,“用户”:“postgres”, “密码”:“0123456789”}
how to do this with docker you can ask me!如何使用 docker 执行此操作,您可以问我!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.