[英]python pandas to SQL server Error overload columns
i am trying this code 我正在尝试此代码
df = pandas.read_sql_table(details)
df.to_sql(details)
meaning i tried to take a table and just put it back in SQL in a new name and i have tried do some manipulation on the table but still i got the same error: 意思是我试图取一个表,然后以新名称将其放回SQL中,并且尝试对表进行一些操作,但是仍然遇到相同的错误:
programmingError
\sqlalchemy\engine\base.py in _execute_context(self , dialect , constructor , statement , parameters , *args)
line 1192 parameters ,
line -> 1193 context)
line 1194 except BaseException as e
..\enging\defaulte.py
line 506 def do_execute(self , cursor , statment , parmetrs , context)
line 507 cursor.execute(statment , parametrs)
and programmingError : ('the SQL containe 12366 parameters marks , but 77902 parameters were supplied' , 'HY000') 和programmingError:('SQL包含12366个参数标记,但是提供了77902个参数','HY000')
and 和
DBAPIError :(pyodoc.Error) ('[07002] [Microsoft][SQL Server Native Client 11.0]COUNT field incorrect or syntax error (0) (SQLExecDirectw) [SQL : 'INSERT INTO [testName] ([index] , [col1] , [col2] ... [colN]) VALUES (? , ? , ? ,...?) , (? , ? , ? ,...?) , (? , ? , ? ,...?) , (? , ? , ? ,...?) ... (? , ? , ? ,...?)]')
the number of ? , ? is N times as the col number and the (? , ? , ? ,...?) is as the number of the rows.
when i tried the same code on smaller table or when i dived this table two table with less the 7 cols this load and update perfectly. 当我在较小的表上尝试相同的代码时,或者当我以较少的7列潜入该表的两个表时,此负载并完美地更新。
I'm guessing you've hit an internal limit on the number of bind parameters in a SQL statement within SQL Server or the driver used to connect to it. 我猜您在SQL Server或用于连接到它的驱动程序中的SQL语句中的绑定参数数量达到了内部限制。 Interestingly,
77902 - 12366 == 65536 == 2**16
, which makes me think that a 16-bit integer somewhere has overflowed. 有趣的是,
77902 - 12366 == 65536 == 2**16
,这让我认为某个地方的16位整数已溢出。
Fortunately there seems to be a parameter in the to_sql
method that you can use to break the SQL statements into chunks rather than sending the lot in one go. 幸运的是,在
to_sql
方法中似乎有一个参数,您可以使用该参数将SQL语句拆分为多个块,而不必一次性发送。 From the Pandas documentation for to_sql
: 从
to_sql
的Pandas文档中 :
chunksize : int, optional
chunksize : int,可选
Rows will be written in batches of this size at a time.
行将一次以这种大小批量写入。 By default, all rows will be written at once.
默认情况下,所有行将被一次写入。
The total number of parameters (77902) factorises as 2 × 11 × 3541, with 3541 being prime. 参数总数(77902)分解为2×11×3541,其中3541为质数。 It looks like your table has either 11 columns and 7082 rows or 22 columns and 3541 rows.
看起来您的表有11列7082行或22列3541行。 To avoid this error I would recommend keeping the number of bind parameters in a chunk below 32767 just in case the 16-bit integer I suspect to be causing this problem is signed, so
chunksize=1000
would probably work for you. 为避免此错误,我建议将绑定参数的数量保持在低于32767的块中,以防万一我怀疑是导致此问题的16位整数被签名的原因,因此
chunksize=1000
可能对您chunksize=1000
。 At the same time, however, this is unlikely to give much faster performance than chunksize=100
so it's not worth spending that much time trying to find the largest chunk size that will work. 但是,与此同时,这不太可能提供比
chunksize=100
快得多的性能,因此不值得花费太多时间试图找到可以使用的最大块大小。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.