[英]Writing to Oracle: TypeError: expecting string or bytes object
I have been breaking my head on this.我一直在努力解决这个问题。 I am trying to push 65000+ rows with 51 columns to oracle DB but i end up receiving a type error.我正在尝试将 51 列的 65000+ 行推送到 oracle DB,但我最终收到类型错误。 is there a way i can find out on which column this error is coming from so that i can debug.有没有办法可以找出这个错误来自哪一列,以便我可以调试。
Another question - Can a Datatype "Object" in python dataframe be read a 'Number' Dtype in Oracle?另一个问题 - python dataframe 中的数据类型“对象”能否被读取为 Oracle 中的“数字”Dtype?
Traceback (most recent call last):
File "c:\users\so-go- activating strategic people capability - deliverable files\ finance\codes-to_use\s1_3_supply_forecasting_input_revamped.py", line 160, in <module>
hcar.to_sql('HISTORICAL_HCAR', engine, if_exists='append', index=False,schema='HIM_PA_EXTERN_PROD_FIN',dtype=dtyp)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\pandas\core\generic.py", line 2605, in to_sql
sql.to_sql(
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\pandas\io\sql.py", line 589, in to_sql
pandas_sql.to_sql(
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\pandas\io\sql.py", line 1398, in to_sql
table.insert(chunksize, method=method)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\pandas\io\sql.py", line 830, in insert
exec_insert(conn, keys, chunk_iter)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\pandas\io\sql.py", line 747, in _execute_insert
conn.execute(self.table.insert(), data)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\engine\base.py", line 1011, in execute
return meth(self, multiparams, params)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\sql\elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\engine\base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\engine\base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\engine\base.py", line 1514, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\util\compat.py", line 182, in raise_
raise exception
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\engine\base.py", line 1256, in _execute_context
self.dialect.do_executemany(
File "C:\ProgramData\Anaconda3-2020.11\lib\site-packages\sqlalchemy\dialects\oracle\cx_oracle.py", line 1182, in do_executemany
cursor.executemany(statement, parameters)
TypeError: expecting string or bytes object
Unfortunately, Oracle won't tell you which column is causing the error.不幸的是,Oracle 不会告诉您是哪一列导致了错误。 So this is a Python / cx_Oracle question, not really an Oracle one.所以这是一个 Python / cx_Oracle 问题,而不是真正的 Oracle 问题。 And I assume when you say "dataframe" that you mean a Pandas dataframe, and not PySpark/Dask/Veux/etc.我假设当您说“数据帧”时,您的意思是 Pandas dataframe,而不是 PySpark/Dask/Veux/等。
There's several similar questions about this error with Pandas dataframes. Pandas 数据帧有几个类似的问题。 The issue is usually that Pandas dataframe columns have a dtype
, but the rows don't all have to match that type - object
columns will allow different types in each row.问题通常是 Pandas dataframe 列具有dtype
,但行不必都匹配该类型- object
列将允许每行中的不同类型。
# example - an int, a float, and a str in the same column
pd.DataFrame([12, np.NaN, 'hi'], columns=['ABC'])
When you (or sqlalchemy) use executemany()
, all the rows have to have the same matching set of column types.当您(或 sqlalchemy)使用executemany()
时,所有行都必须具有相同的匹配列类型集。
You can check the types in a single column by using: 您可以使用以下方法检查单个列中的类型:
df['ABC'].map(type)
And so you can check all the columns in a dataframe at once with something like:因此,您可以一次检查 dataframe 中的所有列,例如:
df.applymap(type).nunique()
Which shows the number of types that each column contains.其中显示了每列包含的类型数。 Any column > 1 will probably cause this error.任何 > 1 的列都可能会导致此错误。 Fix it using df['ABC'].astype(str)
or df['ABC'].fillna('')
before sending to Oracle.在发送到 Oracle 之前使用df['ABC'].astype(str)
或df['ABC'].fillna('')
修复它。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.