Insert pandas data frame into SQL temp table

Question

I am attempting to great a temporary table in an SQL database and populate the table from a pandas dataframe. I am receiving an error when using the df.to_sql to populate the temp table. Thank you for the assistance.

import pandas as pd
from sqlalchemy import create_engine
import pandas.io.sql as psql
import urllib

params = urllib.quote_plus("DRIVER={SQL Server};SERVER=ServerAddressHere;DATABASE=DatabaseNameHere;Trusted_Connection=yes")

engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
resoverall = connection.execute('''SELECT DISTINCT
a.CountryRegionID AS ISO_Short,
b.Name
FROM
CustTable AS a
LEFT JOIN AddressCountryRegion AS b
ON b.CountryRegionID = a.CountryRegionID''')


Countries= pd.DataFrame(resoverall.fetchall())
Countries.columns = resoverall.keys()

Countries= pd.Countries['ISO_Short'].str.upper()

Countries= pd.DataFrame(data=Countries)

temp = connection.execute('''
create table #tempTable
(
ISO_Short varchar(5)
)
''')

Countries.to_sql('Countries',engine)

The error I'm receiving is:

ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]CREATE TABLE permission denied in database 'databasename'. (262) (SQLExecDirectW)") [SQL: u'\\nCREATE TABLE [Countries] (\\n\\t[index] BIGINT NULL, \\n\\t[ISO_Short] VARCHAR(max) NULL\\n)\\n\\n'

UPDATE:

The other option I thought of is to use Pyodbc and convert Countries to a dictionary and then pass the dictionary values into the temporary table. Using this method, everything works until I try and pass the dictionary to the temp table. I have the following code using this approach:

import pandas as pd
import pyodbc
import pandas.io.sql as psql

cnxn = pyodbc.connect('''DRIVER={SQL Server};SERVER=telsmith;
DATABASE=DatabaseNameHere;Trusted_Connection=yes;''')
cursor = cnxn.cursor()

Countries= '''
SELECT DISTINCT
a.CountryRegionID AS ISO_Short,
b.Name
FROM
CustTable AS a
LEFT JOIN AddressCountryRegion AS b
ON b.CountryRegionID = a.CountryRegionID
'''

Countries= psql.read_sql(Countries, cnxn)


Countries= Countries['ISO_Short'].str.upper()

Countries= pd.DataFrame(data=Countries)

Countriesdict = Countries.to_dict()


Temp = '''
create table #tempTable
(
    ISO_Short varchar(5)
)

'''

cnxn.commit()

# This is where I run into difficulty
placeholders = ', '.join(['%s'] * len(Countriesdict ))
columns = ', '.join(Countriesdict .keys())
sql = "INSERT INTO #tempTable VALUES ( %s )" % (placeholders)
cursor.execute(sql, Countriesdict.values())

Answer 1

This might sound little dumb but look at the error:

ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]CREATE TABLE permission denied in database 'databasename'. (262) (SQLExecDirectW)") [SQL: u'\\nCREATE TABLE [Countries] (\\n\\t[index] BIGINT NULL, \\n\\t[ISO_Short] VARCHAR(max) NULL\\n)\\n\\n'

Do you have any database called databasename ? Since It can't find database, it can't create table. I ran the same code and it worked just fine. I believe that's the reason

Answer 2

Not strictly a SQLAlchemy concern. You need to obtain "CREATE TABLE" permissions on the server from your DBA for some username and password, and use those credentials to access your DB. Try including "UID=uname;PWD=pword;" in your params for some set of permissioned credentials.

Answer 3

I might have a solution that worked for me:

from sqlalchemy import create_engine
import urllib
params = urllib.parse.quote_plus("DRIVER={SQL Server};SERVER=10.233.6.52;DATABASE=databaseName;UID=xxx;PWD=Welcome1!")

engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
df.to_sql('tempTable',engine)