编码和解码 - 希伯来语字符串看起来像乱码 - Teradata 的 Python 3

Question

I connected to my Teradata DB and extract simple query result to a dataframe. The dataframe (df_clients) has 1 column (FirstName) and 1 row with Hebrew string ('שלום לך') as I print the dataframe I get gibberish instead of Hebrew language: ùìåí ìê

I find a solution to encode and decode string:

strr = "ùìåí ìê"

print( strr) #not good

print( strr.encode('cp1252').decode('cp1255',errors='replace')) #good

it worked.. but when I tried the same solution with Pandas Dataframe it doesn't work (no error, but don't work):

df_clients.FirstName.apply(lambda x : x.encode('cp1252').decode('cp1255',errors='replace') )

Answer 1

The equivalent of your string encoding in Pandas is .str.<de|encode> . I have generated example of the same number as demo:

import pandas as pd

dataf = pd.DataFrame({
    'name':["ùìåí ìê", "ùìåí ìê"]
})

dataf["name"].str.encode('cp1252').str.decode('cp1255',errors='replace')

# result
# 0    שלום לך
# 1    שלום לך

Update

[assign and] a solution that can handle all rows[|columns].

import pandas as pd

dataf = pd.DataFrame({
    'first_name':["ùìåí ìê", "ùìåí ìê"], 'last_name': ["ùìåí ìê", "ùìåí ìê"]
})

# all columns [assuming all are text]

dataf = dataf.transform(lambda x: x.str.encode('cp1252', ,errors='replace').str.decode('cp1255', errors='replace'))

# or selecting subset of columns

dataf[["first_name","last_name"]] = dataf[["first_name","last_name"]].transform(lambda x: x.str.encode('cp1252', errors='replace' ).str.decode('cp1255',errors='replace'))

dataf

#    first_name last_name
# 0 שלום לך שלום לך
# 1 שלום לך שלום לך

If there are errors, try set errors='ignore' and see what worked and which characters are still gibberish.