Pandas dataframe to sql database

Question

I got a problem with a pandas dataframe which I create to store my experimental results and some postprocessings. The creation of the whole dataframe takes about 6 hours , thus I need to store the database, so I can reload it to jupyter. And here occurs my problem.

I thought the best way to to this is to create a sql database from my pandas dataframe. but I get the error:

InterfaceError: Error binding parameter 1 - probably unsupported type.

Below you will find a minimal reproducable example to generate this error. I suggest it occurs since I have object from the uncertainties toolbox as well as numpy . How can I solve this problem, or have someone maybe a better idea to reach my goal?

I would be very thankful.

Cheers, Arch

from uncertainties import ufloat
import pandas as pd
from sqlalchemy import create_engine
import numpy as np

a = ufloat(1,0.1)
b = ufloat(2,0.2)
b = ufloat(3,0.3)

c = np.array([1,2,3])
d = np.array([3,4,5])
e = np.array([6,7,8])

data = {
  "A": [a, b, c],
  "B": [a, a, c],
}

df = pd.DataFrame(data)

df['C'] = pd.Series(dtype=object)

df['C'] = [c,d,e]

display(df)

engine = create_engine('sqlite://', echo=False)
df.to_sql('sql_db', con=engine)

Answer 1

There is no datatype mapping for uncertainties.ufloat in SQLAlchemy.

from sqlalchemy import create_engine, select, text
from uncertainties import ufloat

a = ufloat(1, 0.1)

engine = create_engine("sqlite://", echo=True)

with engine.connect() as con:
    con.execute("SELECT ?", a)

raises

ProgrammingError: (sqlite3.ProgrammingError) Error binding parameter 1: type 'Variable' is not supported
[SQL: SELECT ?]
[parameters: (1.0+/-0.1,)]
(Background on this error at: https://sqlalche.me/e/14/f405)