堆栈内存溢出
  简体   繁体   English

如何转换从字符串中删除 "\\n" & "\\" 并将该字符串转换为 JSON 或 dict

[英]How to convert remove "\n" & "\" from the string & convert that string to JSON or dict

 原文  2021-11-02 18:08:31  9   4    python/ json

问题描述

I want to convert this data into json or python dictionary.我想将此数据转换为 json 或 python 字典。

What would be the best possible way of doing it as I have at least 3GB of data like this.什么是最好的方法，因为我至少有 3GB 这样的数据。

"{\"domain\":\"defb2b00f609c6bb8fcfd43af70c146bc8a26036f800e8f7563bc366fb88aa1f\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,66],\"confidence\":0.90371,\"tier\":2}\n{\"domain\":\"e7378a78724fcd254b59764f451be766d0e1c6683eac9aa3d5f29798600d91af\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,59],\"confidence\":0.90371,\"tier\":2}\n{\"domain\":\"5f616b8a7b283395961018da6ac75a563efdfcf743ce7e1cd1bcbec0a23a5349\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[18],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"4e219482bd58e2c7d91c55e52aa5db37785f29314e4db9c319ab9edb9ee5de1e\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"e3ad60e6f8786da0253d8ce00fcb90ee5bf497a75c5b42753acba203800ad6fe\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"49ae5ad8cc8de0136f3f99a1330710a14912fb743578d4ce39318281979162b4\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[26],\"confidence\":0.9594,\"tier\":2}\n{\"domain\":\"f93af67b58299d7317841de70464624ffc0190b93a4af860dcc58038162a30cf\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,62],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"c9356044593f00f2b779cbd59246695e654bf1f105a265867d4f06c1bb6a2ea2\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[66,68],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"7920da2f3dc7de646d3434d467ffccdd8ca31115c54529bc2a4d758896ae1a19\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[4,74],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"08a4027677824509beee405482a5f1a5f4feddbf0aafbf4b649c2909732f6909\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[75],\"confidence\":0.8198000000000001,\"tier\":2}\n"

4 个解决方案

解决方案1
 1  2021-11-02 18:17:21

I suggest you fix what is generating the data.我建议您修复生成数据的内容。 Otherwise, split the newlines否则，拆分换行符

>>> s = "... your string ..."
>>> data = [json.loads(x) for x in s.splitlines()]
>>> for x in data:
...   print(x['tier'])
2
2
2
2
2
2
2
2
2
2

https://jsonlines.org/ https://jsonlines.org/

解决方案2
 1  2021-11-02 18:37:46

You can turn it to a bytes type string, then decode it using unicode_escape to remove the escapes.您可以将其转换为bytes类型字符串，然后使用unicode_escape进行解码以删除转义unicode_escape 。 It's the opposite of re.escape :这是re.escape ：

unescaped = bytes(s, 'utf-8').decode('unicode_escape')

Then just read it as any other JSONL ( jsonlines ) object然后将其作为任何其他 JSONL ( jsonlines )对象读取

list(map(json.loads, unescaped.split('\n')))

Output:输出：

[
    {'domain': 'defb2b00f609c6bb8fcfd43af70c146bc8a26036f800e8f7563bc366fb88aa1f', 'path': '/', 'scope': 'domain', 'categories': [9, 66], 'confidence': 0.90371, 'tier': 2},
    {'domain': 'e7378a78724fcd254b59764f451be766d0e1c6683eac9aa3d5f29798600d91af', 'path': '/', 'scope': 'domain', 'categories': [9, 59], 'confidence': 0.90371, 'tier': 2},
    {'domain': '5f616b8a7b283395961018da6ac75a563efdfcf743ce7e1cd1bcbec0a23a5349', 'path': '/', 'scope': 'domain', 'categories': [18], 'confidence': 0.70767, 'tier': 2},
    {'domain': '4e219482bd58e2c7d91c55e52aa5db37785f29314e4db9c319ab9edb9ee5de1e', 'path': '/', 'scope': 'domain', 'categories': [9], 'confidence': 0.8198000000000001, 'tier': 2},
    {'domain': 'e3ad60e6f8786da0253d8ce00fcb90ee5bf497a75c5b42753acba203800ad6fe', 'path': '/', 'scope': 'domain', 'categories': [9], 'confidence': 0.8198000000000001, 'tier': 2},
    {'domain': '49ae5ad8cc8de0136f3f99a1330710a14912fb743578d4ce39318281979162b4', 'path': '/', 'scope': 'domain', 'categories': [26], 'confidence': 0.9594, 'tier': 2},
    {'domain': 'f93af67b58299d7317841de70464624ffc0190b93a4af860dcc58038162a30cf', 'path': '/', 'scope': 'domain', 'categories': [9, 62], 'confidence': 0.70767, 'tier': 2},
    {'domain': 'c9356044593f00f2b779cbd59246695e654bf1f105a265867d4f06c1bb6a2ea2', 'path': '/', 'scope': 'domain', 'categories': [66, 68], 'confidence': 0.70767, 'tier': 2},
    {'domain': '7920da2f3dc7de646d3434d467ffccdd8ca31115c54529bc2a4d758896ae1a19', 'path': '/', 'scope': 'domain', 'categories': [4, 74], 'confidence': 0.8198000000000001, 'tier': 2},
    {'domain': '08a4027677824509beee405482a5f1a5f4feddbf0aafbf4b649c2909732f6909', 'path': '/', 'scope': 'domain', 'categories': [75], 'confidence': 0.8198000000000001, 'tier': 2}
]

解决方案3
 0  2021-11-02 18:17:08

Use regex for this:为此使用正则表达式：

import re
import json

regex = r"\\(\")|\\n"

test_str = ("\"{\\\"domain\\\":\\\"defb2b00f609c6bb8fcfd43af70c146bc8a26036f800e8f7563bc366fb88aa1f\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[9,66],\\\"confidence\\\":0.90371,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"e7378a78724fcd254b59764f451be766d0e1c6683eac9aa3d5f29798600d91af\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[9,59],\\\"confidence\\\":0.90371,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"5f616b8a7b283395961018da6ac75a563efdfcf743ce7e1cd1bcbec0a23a5349\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[18],\\\"confidence\\\":0.70767,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"4e219482bd58e2c7d91c55e52aa5db37785f29314e4db9c319ab9edb9ee5de1e\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[9],\\\"confidence\\\":0.8198000000000001,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"e3ad60e6f8786da0253d8ce00fcb90ee5bf497a75c5b42753acba203800ad6fe\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[9],\\\"confidence\\\":0.8198000000000001,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"49ae5ad8cc8de0136f3f99a1330710a14912fb743578d4ce39318281979162b4\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[26],\\\"confidence\\\":0.9594,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"f93af67b58299d7317841de70464624ffc0190b93a4af860dcc58038162a30cf\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[9,62],\\\"confidence\\\":0.70767,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"c9356044593f00f2b779cbd59246695e654bf1f105a265867d4f06c1bb6a2ea2\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[66,68],\\\"confidence\\\":0.70767,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"7920da2f3dc7de646d3434d467ffccdd8ca31115c54529bc2a4d758896ae1a19\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[4,74],\\\"confidence\\\":0.8198000000000001,\\\"tier\\\":2}\\n{\\\"domain\\\":\\\"08a4027677824509beee405482a5f1a5f4feddbf0aafbf4b649c2909732f6909\\\",\\\"path\\\":\\\"/\\\",\\\"scope\\\":\\\"domain\\\",\\\"categories\\\":[75],\\\"confidence\\\":0.8198000000000001,\\\"tier\\\":2}\\n\"\n")

subst = "\\g<1>"

result = re.sub(regex, subst, test_str, 0, re.MULTILINE)
if result.startswith('"') and result.endswith('"'):
    result = result[1:-1]
if result:
    print (result)
    object_result = json.loads(result)
    print(object_result)

解决方案4
 0  2021-11-02 18:57:56

We can make use of ast.literal_eval() to safely evaluate list of objects within string literal.我们可以使用ast.literal_eval()来安全地评估字符串文字中的对象列表。

>>> raw_string = "{\"domain\":\"defb2b00f609c6bb8fcfd43af70c146bc8a26036f800e8f7563bc366fb88aa1f\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,66],\"confidence\":0.90371,\"tier\":2}\n{\"domain\":\"e7378a78724fcd254b59764f451be766d0e1c6683eac9aa3d5f29798600d91af\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,59],\"confidence\":0.90371,\"tier\":2}\n{\"domain\":\"5f616b8a7b283395961018da6ac75a563efdfcf743ce7e1cd1bcbec0a23a5349\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[18],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"4e219482bd58e2c7d91c55e52aa5db37785f29314e4db9c319ab9edb9ee5de1e\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"e3ad60e6f8786da0253d8ce00fcb90ee5bf497a75c5b42753acba203800ad6fe\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"49ae5ad8cc8de0136f3f99a1330710a14912fb743578d4ce39318281979162b4\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[26],\"confidence\":0.9594,\"tier\":2}\n{\"domain\":\"f93af67b58299d7317841de70464624ffc0190b93a4af860dcc58038162a30cf\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[9,62],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"c9356044593f00f2b779cbd59246695e654bf1f105a265867d4f06c1bb6a2ea2\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[66,68],\"confidence\":0.70767,\"tier\":2}\n{\"domain\":\"7920da2f3dc7de646d3434d467ffccdd8ca31115c54529bc2a4d758896ae1a19\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[4,74],\"confidence\":0.8198000000000001,\"tier\":2}\n{\"domain\":\"08a4027677824509beee405482a5f1a5f4feddbf0aafbf4b649c2909732f6909\",\"path\":\"/\",\"scope\":\"domain\",\"categories\":[75],\"confidence\":0.8198000000000001,\"tier\":2}\n"

>>> modified_string = raw_string.replace('\\','').replace('\n',',')

>>> import ast
>>> dicts = ast.literal_eval(modified_string)      # a tuple of dictionaries

>>> for d in dicts:
...     print(d)
... 
{'domain': 'defb2b00f609c6bb8fcfd43af70c146bc8a26036f800e8f7563bc366fb88aa1f', 'path': '/', 'scope': 'domain', 'categories': [9, 66], 'confidence': 0.90371, 'tier': 2}
{'domain': 'e7378a78724fcd254b59764f451be766d0e1c6683eac9aa3d5f29798600d91af', 'path': '/', 'scope': 'domain', 'categories': [9, 59], 'confidence': 0.90371, 'tier': 2}
{'domain': '5f616b8a7b283395961018da6ac75a563efdfcf743ce7e1cd1bcbec0a23a5349', 'path': '/', 'scope': 'domain', 'categories': [18], 'confidence': 0.70767, 'tier': 2}
{'domain': '4e219482bd58e2c7d91c55e52aa5db37785f29314e4db9c319ab9edb9ee5de1e', 'path': '/', 'scope': 'domain', 'categories': [9], 'confidence': 0.8198000000000001, 'tier': 2}
{'domain': 'e3ad60e6f8786da0253d8ce00fcb90ee5bf497a75c5b42753acba203800ad6fe', 'path': '/', 'scope': 'domain', 'categories': [9], 'confidence': 0.8198000000000001, 'tier': 2}
{'domain': '49ae5ad8cc8de0136f3f99a1330710a14912fb743578d4ce39318281979162b4', 'path': '/', 'scope': 'domain', 'categories': [26], 'confidence': 0.9594, 'tier': 2}
{'domain': 'f93af67b58299d7317841de70464624ffc0190b93a4af860dcc58038162a30cf', 'path': '/', 'scope': 'domain', 'categories': [9, 62], 'confidence': 0.70767, 'tier': 2}
{'domain': 'c9356044593f00f2b779cbd59246695e654bf1f105a265867d4f06c1bb6a2ea2', 'path': '/', 'scope': 'domain', 'categories': [66, 68], 'confidence': 0.70767, 'tier': 2}
{'domain': '7920da2f3dc7de646d3434d467ffccdd8ca31115c54529bc2a4d758896ae1a19', 'path': '/', 'scope': 'domain', 'categories': [4, 74], 'confidence': 0.8198000000000001, 'tier': 2}
{'domain': '08a4027677824509beee405482a5f1a5f4feddbf0aafbf4b649c2909732f6909', 'path': '/', 'scope': 'domain', 'categories': [75], 'confidence': 0.8198000000000001, 'tier': 2}

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何将此json字符串转换为dict？ - How to convert this json string to dict? 将字符串转换为Dict / JSON - Convert string into Dict / JSON 如何将 dict 转换为转义的 JSON 字符串？ - How to convert a dict to an escaped JSON string? 如何将字典转换为字符串？ - How to convert a dict to string? 如何将Dict转换为字符串 - How to Convert Dict to string 如何将字符串转换为dict - How to convert string to dict 如何将字符串从列表转换为字典？ - How to convert a string to a dict from a list? 从Tweepy解析大量JSON字符串或转换为dict / JSON格式 - Parse massive JSON string from Tweepy or convert to dict/JSON format 如何将unicoded dict转换为字符串 - How to convert unicoded dict into a string 如何在连接中将dict转换为字符串？ - How to convert dict to string in concatenation?
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM