python：错误处理带有Unicode数据的有序字典

Question

My script migrates data from MySQL to mongodb. It runs perfectly well when there are no unicode columns included. But throws me below error when OrgLanguages column is added.

    mongoImp = dbo.insert_many(odbcArray)
  File "/home/lrsa/.local/lib/python2.7/site-packages/pymongo/collection.py", line 711, in insert_many
    blk.execute(self.write_concern.document)
  File "/home/lrsa/.local/lib/python2.7/site-packages/pymongo/bulk.py", line 493, in execute
    return self.execute_command(sock_info, generator, write_concern)
  File "/home/lrsa/.local/lib/python2.7/site-packages/pymongo/bulk.py", line 319, in execute_command
    run.ops, True, self.collection.codec_options, bwc)
bson.errors.InvalidStringData: strings in documents must be valid UTF-8: 'Portugu\xeas do Brasil, ?????, English, Deutsch, Espa\xf1ol latinoamericano, Polish'

My code:

import MySQLdb, MySQLdb.cursors, sys, pymongo, collections

odbcArray=[]
mongoConStr = '192.168.10.107:36006'
sqlConnect = MySQLdb.connect(host = "54.175.170.187", user = "testuser", passwd = "testuser", db = "testdb", cursorclass=MySQLdb.cursors.DictCursor)
mongoConnect = pymongo.MongoClient(mongoConStr)

sqlCur = sqlConnect.cursor()
sqlCur.execute("SELECT ID,OrgID,OrgLanguages,APILoginID,TransactionKey,SMTPSpeed,TimeZoneName,IsVideoWatched FROM organizations")

dbo = mongoConnect.eaedw.mysqlData
tuples = sqlCur.fetchall()

for tuple in tuples:
    odbcArray.append(collections.OrderedDict(tuple))

mongoImp = dbo.insert_many(odbcArray)

sqlCur.close()
mongoConnect.close()
sqlConnect.close()
sys.exit()

Above script migraates data perfectly when tried without OrgLanguages column in the SELECT query. To overcome this, I have tried to use the OrderedDict() in another way but gives me a different type of error
Changed Code:

for tuple in tuples:
    doc = collections.OrderedDict()
    doc['oid'] = tuple.OrgID
    doc['APILoginID'] = tuple.APILoginID
    doc['lang'] = unicode(tuple.OrgLanguages)
    odbcArray.append(doc)
mongoImp = dbo.insert_many(odbcArray)

Error Received:

Traceback (most recent call last):
  File "pymsql.py", line 19, in <module>
    doc['oid'] = tuple.OrgID
AttributeError: 'dict' object has no attribute 'OrgID'

Answer 1

您的MySQL连接返回的字符采用与UTF-8不同的编码，UTF-8是所有BSON字符串必须使用的编码。尝试使用原始代码，但将charset='utf8'传递给MySQLdb.connect 。