使用to_sql將pandas df寫入mysql時拋出錯誤

Question

我正在使用 praw 抓取 reddit 並將記錄存儲在 pandas df 中。 使用 sqlalchemy 和 pymysql 的組合連接到我的 AWS RDS 數據庫和 to_sql 到 append 記錄到現有表。 在我點擊 to_sql 方法之前，一切似乎都運行良好。 它拋出以下錯誤，我不太確定從這里到 go 的位置。 任何幫助或建議都會很棒！

engine = sqlalchemy.create_engine('mysql+pymysql://username:password@database...rds.amazonaws.com:3306/socialdata')
df_comment = pd.DataFrame(comment_table)

df_comment.to_sql(name='reddit_comments', con=engine, index=False, if_exists='append')

Traceback (most recent call last):
  File "/Users/ty/Desktop/Python/reddit_scraper.py", line 121, in <module>
    df_comment.to_sql(name='reddit_comments', con=engine, index=False, if_exists='append')
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/core/generic.py", line 2605, in to_sql
    sql.to_sql(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/io/sql.py", line 589, in to_sql
    pandas_sql.to_sql(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/io/sql.py", line 1398, in to_sql
    table.insert(chunksize, method=method)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/io/sql.py", line 830, in insert
    exec_insert(conn, keys, chunk_iter)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pandas/io/sql.py", line 747, in _execute_insert
    conn.execute(self.table.insert(), data)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1256, in _execute_context
    self.dialect.do_executemany(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 148, in do_executemany
    rowcount = cursor.executemany(statement, parameters)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/cursors.py", line 188, in executemany
    return self._do_execute_many(q_prefix, q_values, q_postfix, args,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/cursors.py", line 206, in _do_execute_many
    v = values % escape(next(args), conn)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/cursors.py", line 120, in _escape_args
    return {key: conn.literal(val) for (key, val) in args.items()}
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/cursors.py", line 120, in <dictcomp>
    return {key: conn.literal(val) for (key, val) in args.items()}
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/connections.py", line 469, in literal
    return self.escape(obj, self.encoders)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/connections.py", line 462, in escape
    return converters.escape_item(obj, self.charset, mapping=mapping)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/converters.py", line 27, in escape_item
    val = encoder(val, mapping)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/converters.py", line 123, in escape_unicode
    return u"'%s'" % _escape_unicode(value)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pymysql/converters.py", line 78, in _escape_unicode
    return value.translate(_escape_table)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/praw/models/reddit/base.py", line 35, in __getattr__
    return getattr(self, attribute)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/praw/models/reddit/base.py", line 36, in __getattr__
    raise AttributeError(
AttributeError: 'Redditor' object has no attribute 'translate'

Answer 1

DataFrame 中的一列包含自定義“Redditor”object，它不是 map 對應的 SQL 數據類型。 pymysql 調用對象的翻譯 function 當它不是像 int float 或 string 這樣明顯的東西時

如果 Redditor 只是存儲用戶名和其他元數據的包裝器 object，那么您可以執行一些操作，例如將該列重新映射到 Redditor object 的字符串/數字表示形式。如果它是您定義的 object，您可以添加一個translate() function 到 Redditor 類的定義以返回適當的值。 例如，如果 Redditor.id 包含您要存儲在列中的值：-

class Redditor():
  def translate(self):
    # Change self.id with the value you care about
    return self.id 

或在 pandas 保存之前

df[REDDITOR_COLUMN] = df[REDDITOR_COLUMN].apply(lambda x: x.id)