How do I change unicode u'\xb0C' from SQL to string °C in python?

Question

I SELECT data from SQL into temp_data dictionary but there is a unit value like "u't_unit': u'\\xb0C'" which I can't convert to string °C in python.

Python Code:

temp_data = [{u'thresh_id': 508, u'app_guid': u'7D83E6F8D879BE17C95D14879159973A', 
u'override': 1, u'thresh_guid': u'BB8F3E8184783E0416EEA7D19D9C9FBC', 
u'app_id': 1102, u'edit_user': 1, u'h_range': Decimal('100.000'), 
u't_type': 1, u't_value': Decimal('0.000'), u'l_range': Decimal('0.000'), 
u't_unit': u'\xb0C', u'edit_date': datetime.datetime(2019, 4, 30, 4, 24, 44), 
u'name': u'Sensor Temperature High Major'}]  

I used the following code to get my required values:

Python Code:

b = []  
for i in temp_data:  
    a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])  
    b.append(a)  
    b = [tuple(str(item) for item in t) for t in b]  
final_data = (",".join(map(str,b)))  

The error message received:
UnicodeEncodeError: 'ascii' codec can't encode character u'\\xb0' in position 0: ordinal not in range(128)

Expected result:

('Dynamic', 'Sensor Temperature High Major', '508', '°C', '0.000', '0.000', '100.000') 

I tried to use 'u'\\xb0C'.encode('utf8')' to convert it, but it is not working.

How could I get the result as expected?
Thank you very much.

Answer 1

I suppose you use python 2.* .That's because you're printing the tuple. For example if you access the specific value you'll see that it is printed normally. Try this:

b = []
for i in temp_data:
    a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])
    print(i['t_unit'])  # prints °C
    print(a[3])  # prints °C

Answer 2

Now, I add following codes on top and it is working without any error.

# encoding=utf8
import os, sys
reload(sys)
sys.setdefaultencoding('utf8')

The output will like following: ('Dynamic', 'Sensor Temperature High Major', '508', '\\xc2\\xb0C', '0.000', '0.000', '100.000')

But, there is another issue that it will show 'xc2xb0C' if I INSERT INTO mysql database table.

Can anyone provide a method to convert it to °C in mysql table? Thank you.

Answer 3

When processing unicode data in Python 2, it's important to use the unicode and str types consistently. Ideally, use only unicode inside your application, and convert to and from str only when receiving input, or generating output, only when necessary.

The values in temp_data are either unicode or non-string types, so we want to process them as unicode objects rather than str .

>>> b = []
>>> for i in temp_data:
...     a = ('Dynamic', i['name'], i['thresh_id'], i['t_unit'], i['t_value'], i['l_range'], i['h_range'])
...     b.append(a)
...     # Cast items to unicode, not str
...     b = [tuple(unicode(item) for item in t) for t in b]
... 
>>> # Now all our items are unicode strings
>>> b
[(u'Dynamic', u'Sensor Temperature High Major', u'508', u'\xb0C', u'0.000', u'0.000', u'100.000')]
>>> # Join each tuple with unicode strings as the join value.
>>> final_data = (u','.join(u','.join(x) for x in b))  
>>> # Now the data prints correctly.
>>> print final_data
Dynamic,Sensor Temperature High Major,508,°C,0.000,0.000,100.000