将结构的字符串表示形式格式化为 python 字典

Question

I need a little help processing a String to a Dict, considering that the String is not in a common format, but an output from a UDF function

The return from the PySpark UDF looks like the string below:

"{list=[{a=1}, {a=2}, {a=3}]}"

And I need to convert it to a python dictionary with the structure below:

{
  "list": [
    {"a": 1}
    {"a": 2}
    {"a": 3}
  ]
}

So I can access it's values, like

dict["list"][1]["a"]

I already tried using:

• JSON.loads
• ast_eval()

Could someone please help me?

As an example of how this unparsed string is generated:

@udf()
def execute_method():
  return {"list": [{"a":1},{"b":1}{"c":1}]}

df_result = df_source.withColumn("result", execute_method())

Answer 1

By the very least you will need to replace = with : and surround keys with double quotes:

import json
import re

string = "{list=[{a=1}, {a=2}, {a=3}]}"
fixed_string = re.sub(r'(\w+)=', r'"\1":', string)
print(type(fixed_string), fixed_string)
parsed = json.loads(fixed_string)
print(type(parsed), parsed)

outputs

<class 'str'> {"list":[{"a":1}, {"a":2}, {"a":3}]}
<class 'dict'> {'list': [{'a': 1}, {'a': 2}, {'a': 3}]}

Answer 2

try this:

import re
import json  
data="{list=[{a=1}, {a=2}, {a=3}]}"
data=data.replace('=',':')
pattern=[e.group() for e in re.finditer('[a-z]+', data, flags=re.IGNORECASE)]
for e in set(pattern):
    data=data.replace(e,"\""+e+"\"")
print(json.loads(data))