[英]Python: SQL Server insert multiple values with datetime (Conversion failed when converting date and/or time from character string)
[英]From Pandas Dataframe to pyodbc (Azure SQL DB) using sqlalchemy: Conversion failed when converting date and/or time from character string
I'm trying to load Salesforce data to Azure SQL Database incrementally by launching a Python script on Azure Databricks.
由於我無法在 Azure Databricks 中安裝 Devart ODBC,因此我正在嘗試使用 simple_salesforce 從 salesforce 獲取數據:
import pandas as pd
import pyodbc
from simple_salesforce import Salesforce, SalesforceLogin, SFType
from sqlalchemy.types import Integer, Text, String, DateTime
from sqlalchemy import create_engine
import urllib
sf = Salesforce(password = password, username=username, security_token=jeton)
rep_qr = "SELECT SOMETHING FROM Account WHERE CONDITION"
soql = prep_qr.format(','.join(field_names))
results = sf.query_all(soql)['records']
我得到以下結果(一個例子):
[OrderedDict([('attributes', OrderedDict([('type', 'Account'), ('url', '/services/data/v42.0/sobjects/Account/0014K000009aoU3QAI')])), ('Id', XY1), (Name, Y), (Date, 2020-11-24T09:16:17.000+0000)])]
然后我將 output 轉換為 pandas Dataframe:
results = pd.DataFrame(sf.query_all(soql)['records'])
results.drop(columns=['attributes'], inplace=True) #to keep only the columns
我得到了這樣的東西(只是一個例子):
ID | 姓名 | 日期 |
---|---|---|
XY1 | 是的 | 2020-11-24T09:16:17.000+0000 |
In order to ingest this data into Azure SQL Database I have used "sqlalchemy" to convert the Dataframe into sql, then pyodbc will take in charge the insertion part into the destination (Azure SQL Database), as shown bellow:
df = pd.DataFrame(results)
df.reset_index(drop=True, inplace=True) #just to remove the index from dataframe
#Creating the engine from and pyodbc which is connected to Azure SQL Database:
params = urllib.parse.quote_plus \
(r'DRIVER={ODBC Driver 17 for SQL Server};SERVER=' + server + ';DATABASE=' + database + ';UID=' + username + ';PWD=' + password)
conn_str = 'mssql+pyodbc:///?odbc_connect={}'.format(params)
engine_azure = create_engine(conn_str, echo=True)
df.to_sql('account',engine_azure,if_exists='append', index=False)
但我收到以下錯誤:
sqlalchemy.exc.DataError: (pyodbc.DataError) ('22007', '[22007] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')
我認為問題是庫 simple_salesforce 以這種格式設置日期/時間:
2020-11-24T09:16:17.000+0000
但是在 Azure SQL 數據庫中應該是這樣的:
2020-11-24T09:16:17.000
這里的問題是我正在動態加載表(我什至不知道我正在加載的表和列)我無法轉換這些數據類型的原因,我需要一種將數據類型傳遞給 pyodbc 的方法自動地。
請問有什么可以推薦的?
謝謝,
如果日期/時間值始終作為2020-11-24T11:22:33.000+0000
形式的字符串返回,那么您可以使用 pandas 的.apply()
方法將字符串轉換為2020-11-24 11:22:33.000
SQL 服務器將接受的格式:
df = pd.DataFrame(
[
(1, "2020-11-24T11:22:33.000+0000"),
(2, None),
(3, "2020-11-24T12:13:14.000+0000"),
],
columns=["id", "dtm"],
)
print(df)
"""console output:
id dtm
0 1 2020-11-24T11:22:33.000+0000
1 2 None
2 3 2020-11-24T12:13:14.000+0000
"""
df["dtm"] = df["dtm"].apply(lambda x: x[:23].replace("T", " ") if x else None)
print(df)
"""console output:
id dtm
0 1 2020-11-24 11:22:33.000
1 2 None
2 3 2020-11-24 12:13:14.000
"""
df.to_sql(
table_name,
engine,
index=False,
if_exists="append",
)
with engine.begin() as conn:
pprint(conn.execute(sa.text(f"SELECT * FROM {table_name}")).fetchall())
"""console output:
[(1, datetime.datetime(2020, 11, 24, 11, 22, 33)),
(2, None),
(3, datetime.datetime(2020, 11, 24, 12, 13, 14))]
"""
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.