Python encoding unicode utf-8

Question

I'm using selenium to insert text input with german umlauts in a web formular. The declared coding for the python script is utf-8. The page uses utf-8 encoding. When i definine a string like that everything works fine:

q = u"Hällö" #type(q) returns unicode
...
textbox.send_keys(q)

But when i try to read from a config file using ConfigParser (or another kind of file) i get malformed output in the webformular ( Hällö ). This is the code i use for that:

the_encoding = chardet.detect(q)['encoding'] #prints utf-8
q = parser.get('info', 'query') # type(q) returns str
q = q.decode('unicode-escape') # type(q) returns unicode
textbox.send_keys(q)

Whats the difference between the both q's given to the send_keys function?

Answer 1

This is probably bad encoding. Try printing q before the last statement, and see if it's equal. This line q = parser.get('info', 'query') # type(q) returns str should return the string 'H\\xc3\\xa4ll\\xc3\\xb6' . If it's different, then you are using the wrong coding.

>>> q = u"Hällö"  # unicode obj
>>> q
u'H\xe4ll\xf6'
>>> print q
Hällö
>>> q.encode('utf-8')
'H\xc3\xa4ll\xc3\xb6'
>>> a = q.encode('utf-8')  # str obj
>>> a
'H\xc3\xa4ll\xc3\xb6'  # <-- this should be the value of the str
>>> a.decode('utf-8')  # <-- unicode obj
u'H\xe4ll\xf6'
>>> print a.decode('utf-8')
Hällö
>>> 

Answer 2

from ConfigParser import SafeConfigParser
import codecs

parser = SafeConfigParser()

with codecs.open('cfg.ini', 'r', encoding='utf-8-sig') as f:
    parser.readfp(f)
greet = parser.get('main', 'greet')

print 'greet:', greet.encode('utf-8-sig')

greet: Hällö

cfg.ini file

[main]
greet=Hällö