[英]Python and Snowflake appending new data into an existing table using SQL Alchemy engine is returning current session does not have a current database
I need to append some new data into an existing table on snowflake.我需要将一些新数据附加到雪花上的现有表中。 I am using
sqlalchemy
as the engine along with pandas data frame to_sql()
.我使用
sqlalchemy
作为引擎以及to_sql()
数据框to_sql()
。 Here is the imports and the script:这是导入和脚本:
import pandas as pd
import os
import snowflake.connector as snowCtx
import getpass
import json
import numpy as np
from datetime import date, datetime
import time
from sqlalchemy import create_engine
from sqlalchemy.dialects import registry
import snowflake.sqlalchemy
from snowflake.connector.pandas_tools import pd_writer
from sqlalchemy.ext.declarative import declarative_base
registry.register('snowflake', 'snowflake.sqlalchemy', 'dialect')
columns_df = pd.DataFrame(data.columns.to_list(), columns={'survey_column_name'})
columns_df['survey_id'] = nextval
columns_df = columns_df[['survey_id', 'survey_column_name']]
columns_df.to_sql('SURVEY_METADATA_COLUMN_NAMES',
index = False,
index_label = None,
con = engine,
schema = 'PUBLIC',
if_exists = 'append',
chunksize = 300,
method = pd_writer)
The error I am getting is as follows:我得到的错误如下:
ProgrammingError: (snowflake.connector.errors.ProgrammingError) 090105 (22000): Cannot perform CREATE
ProgrammingError: (snowflake.connector.errors.ProgrammingError) 090105 (22000): 无法执行 CREATE
TABLE.
桌子。 This session does not have a current database.
此会话没有当前数据库。 Call 'USE DATABASE', or use a qualified name.
调用“USE DATABASE”,或使用限定名称。 [SQL:
[SQL:
CREATE TABLE "PUBLIC"."SURVEY_METADATA_COLUMN_NAMES" (
创建表“PUBLIC”。“SURVEY_METADATA_COLUMN_NAMES”(
survey_id INTEGER,
调查 ID 整数,
survey_column_name TEXT )
调查列名文本 )
]
]
The connections are as follows:连接如下:
user = input('Your Snowflake username: ')
password = getpass.getpass('Your Snowflake Password: ')
account = 'MY_ACCOUNT'
conn = snowCtx.connect(
user=user,
password=password,
account=account,
database='MY_DB',
schema='PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN'
)
engine = create_engine(
'snowflake://{user}:{password}@{account}/'.format(
user=user,
password=password,
account=account,
database='MY_DB',
schema = 'PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN',
cache_column_metadata=True
)
)
I switched into using write_pandas()
instead:我改用
write_pandas()
代替:
success, nchunks, nrows, _ = write_pandas(conn,
columns_df,
'SURVEY_METADATA_COLUMN_NAMES',
chunk_size = 300,
schema = 'PUBLIC')
print(success, nchunks, nrows)
if(success):
print(filename+' columns uploaded')
else:
print(filename+' columns were not uploaded')
Which needs pyarrow
library, so I installed it using:需要
pyarrow
库,所以我使用以下方法安装它:
pip install pyarrow
I removed all imports related to sqlalchemy
and kept the following:我删除了所有与
sqlalchemy
相关的导入并保留了以下内容:
import pandas as pd
import os
import snowflake.connector as snowCtx
import getpass
import json
import numpy as np
from datetime import date, datetime
import time
from snowflake.connector.pandas_tools import write_pandas
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.