使用latin1字符集在表中编码UTF8数据

Question

I have a [legacy] mysql table with character set of "latin-1" but storing json information in "utf-8" . A user interface is connected to this table which shows the characters correctly. I need to update this table using a python script but I can't get rid of encoding hell.

On mysql shell I issue "select words from pip where id_pip=42" and receive:

"ventilationsplåtslageri":{"day":"1000","hour":"200","min":"30"}

But when I tried to fetch it from database, I couldn't get the same encoding even though I try several different encodings.

#!/usr/bin/env python                                                           
# -*- coding: utf-8 -*-                                                         
import MySQLdb                                                                  
import json                                                                     
dbconn = MySQLdb.connect(host="host",port=3306,user="user",       
                passwd="pass",db="db", use_unicode=True, charset="utf8")
cursor1 = dbconn.cursor()                                                       
cursor1.execute("select words from pip where id_pip=42")  
track = cursor1.fetchall()                                               
print json.dumps(track, encoding="utf8" )

I tried many different configuarations on this code, eg I changed "use_unicode=False, charset="latin1" with print json.dumps(filter_track, encoding="utf8" ) but I still get either "ventilationspl\Ã\¥tslageri\\" or "ventilationspl\åtslageri\\" and not what I want which is: "ventilationsplåtslageri" I couldn't change the database and I need to update this field of database with sql update command, so I am afraid if I mess up the lagacy database.

Answer 1

I'm not sure if I understand your question, but...

If the content is being returned in Latin-1 and you want it in UTF-8 , I would assume that you'd first need to decode the content from Latin-1 and then encode it to UTF-8 .

latin1_content.decode('latin1').encode('utf8')