How to properly encode data in JSON from Dataframe in Pandas

Question

I have a Pandas Dataframe with the Nordic letters æ, ø and å. I want to format this dataframe to JSON. Everything is working fine except that in the JSON file å is written "\\\å" for instance. I have tried the following:

import json

df_qnapairs.questions = df_qnapairs.questions.str.encode('utf-8')

json_dump = json.dumps(df_qnapairs.to_json(orient = 'records'), ensure_ascii = False)

json_dump

However, the output looks just the same and it does not handle æ, ø, å.

Any suggestions? This is in Databricks with Python

Answer 1

After a while, I found a solution and it was the to_json function that was the problem. I needed to add the force_ascii = False parameter

df_qnapairs.to_json(orient = 'records', force_ascii = False)