null value for primary key when trying to insert list of dictionaries

Question

I'm trying to do a large number of inserts with one call, and the way someone here recommended was by giving .insert a list of dictionaries. This is using SQLAlchemy Core.

As an example:

    try:
        engine = db.create_engine(f"postgres://user:pass@myip/addressbook", connect_args={'connect_timeout': 5})
        connection = engine.connect()
        metadata = db.MetaData()
    except exc.OperationalError:
        print_error(f":: Could not connect to myip!")
        sys.exit()

    table_addressbook = db.Table('addressbook', metadata, autoload=True, autoload_with=engine)

    list = []
    list.append({'firstname': "John", 'lastname': "Doe"})
    list.append({'firstname': "Jane", 'lastname': "Doe"})

    query = db.insert(table_addressbook).values(list)
    connection.execute(query)

But I'm getting an error saying the column id violates a non-null constraint. This is because insert normally auto-generates the primary-key id . How do I use this method but specify that id should be auto-generated? Or is there a different method I should use?

edit

Table name is addressbook .

Column id is type integer with default sequence 'untitled_table_id_seq', constraints are PRIMARY_KEY . This was autogenerated by Postico for Mac, but I've always been able to insert without including id and it auto increments from the last inserted ID.

Columns firstname and lastname are type text , no default, no constraints.

Answer 1

Without any information on your model and/or connection it is a bit difficult to answer your question. Please find below a piece of code which uses insert without throwing non-null constraint errors. Hopefully it helps you.

from sqlalchemy import create_engine, Column, Integer, String, Table
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql import insert

engine = create_engine('sqlite:///:memory:', echo=True)

Base = declarative_base()

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    firstname = Column(String)
    lastname = Column(String)

Base.metadata.create_all(engine)

Session = sessionmaker(bind=engine)
Session.configure(bind=engine)  # once engine is available
session = Session()

new_users = []
new_users.append({'firstname': "John", 'lastname': "Doe"})
new_users.append({'firstname': "Jane", 'lastname': "Doe"})

i = insert(User).values(new_users)
session.execute(i)

PS: most of this is coming from the tutorial on: https://docs.sqlalchemy.org/en/13/orm/tutorial.html

Answer 2

from sqlalchemy import Column
from sqlalchemy import create_engine
from sqlalchemy import Integer
from sqlalchemy import String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import Session

engine = create_engine('sqlite:///:memory:', echo=True)

Base = declarative_base()

# Example Model definition for the illustration
class Customer(Base):
    __tablename__ = "customer"
    id = Column(Integer, primary_key=True)
    name = Column(String(255))
    description = Column(String(255))

Base.metadata.create_all(engine)

######################################################
# Bulk insert using dictionaries.
######################################################
# Insert test records into `customer`table.

def bulk_insert_customers(n):
    session = Session(bind=engine)
    session.bulk_insert_mappings(
        Customer,
        [
            dict(
                name="customer name %d" % i,
                description="customer description %d" % i,
            )
            for i in range(n)
        ],
    )
    session.commit()

Refer these for more examples of how to do bulk inserts in different ways: https://docs.sqlalchemy.org/en/13/_modules/examples/performance/bulk_inserts.html