Importing .db file into Postgresql database

Question

I am currently working on a script to import a .db file into Postgresql database, including the data. Is there any way to do so without using third party tools and by using python?

Answer 1

You can do it with Django for sure.

• python manage.py dumpdata > db.json
• Change the database settings to new database such as of PostgreSQL.
• python manage.py migrate
• python manage.py shell
• Enter the following in the shell

from django.contrib.contenttypes.models import ContentType
ContentType.objects.all().delete()

• python manage.py loaddata db.json

Otherwise if you want to tinker your way. You need to install psycopg2

$ pip install psycopg2

Then you connect to Postgres.

import psycopg2
conn = psycopg2.connect("host=localhost dbname=postgres user=postgres")

This is how you insert values.

cur = conn.cursor()
insert_query = "INSERT INTO users VALUES {}".format("(10, 'hello@dataquest.io', 'Some Name', '123 Fake St.')")
cur.execute(insert_query)
conn.commit()

Now with SQLAlchemy you can easily open an SQLite file.

import sqlite3
conn = sqlite3.connect('database.db')

Fetch the data.

r = conn.execute("""SELECT * FROM books""")
r.fetchall()

Here is how to fetch all tables from SQLite

all the table names within your database:

SELECT name FROM sqlite_master WHERE type = 'table'

sqlite_master can be thought of as a table that contains information about your databases (metadata).

A quick but most likely inefficient way (because it will be running 700 queries with 700 separate resultsets) to get the list of table names, loop through those tables and return data where columnA = "-":

for row in connection.execute('SELECT name FROM sqlite_master WHERE type = "table" ORDER BY name').fetchall()
    for result in connection.execute('SELECT * FROM ' + row[1] + ' WHERE "columnA" = "-"').fetchall()
    # do something with results

Here is an other approach

import sqlite3
try:
    conn = sqlite3.connect('/home/rolf/my.db')
except sqlite3.Error as e:
    print('Db Not found', str(e))
db_list = []
mycursor = conn.cursor()
for db_name in mycursor.execute("SELECT name FROM sqlite_master WHERE type = 'table'"):
    db_list.append(db_name)
for x in db_list:
    print "Searching",x[0]
    try:
        mycursor.execute('SELECT * FROM '+x[0]+' WHERE columnA" = "-"')
        stats = mycursor.fetchall()
        for stat in stats:
            print stat, "found in ", x
    except sqlite3.Error as e:
       continue
conn.close()