有没有一种特定的方法可以使用 python3 从 json 文件中只读取 url？

Question

I'm trying to collect the links from the google history and past their content to .txt files. All the code works (when I created a json which contains only urls), but when the links are there as in the Source example, then I get the error mentioned below. I suspect it's because of the “ in the source data, but how can I get it to just read the URL part?

Source Data:

{
    "Browser History": [
        {
            "favicon_url": "https://www.google.com/favicon.ico",
            "page_transition": "LINK",
            "title": "Google Datenexport",
            "url": "https://takeout.google.com/",
            "client_id": "cWD5MfDDekj1z9aA5VeCQQ\u003d\u003d",
            "time_usec": 1607693084782187
},
        {
            "favicon_url": "https://support.google.com/favicon.ico",
            "page_transition": "LINK",
            "title": "So laden Sie Ihre Google-Daten herunter - Google-Konto-Hilfe",
            "url": "https://support.google.com/accounts/answer/3024190?visit_id\u003d637432898341218017-3159218066\u0026hl\u003dde\u0026rd\u003d1",
            "client_id": "cWD5MfDDekj1z9aA5VeCQQ\u003d\u003d",
            "time_usec": 1607693036534748
},
        {
            "favicon_url": "https://www.google.com/favicon.ico",
            "page_transition": "LINK",
            "title": "Google \u2013 Meine Aktivitäten",
            "url": "https://myactivity.google.com/activitycontrols/webandapp?view\u003ditem",
            "client_id": "cWD5MfDDekj1z9aA5VeCQQ\u003d\u003d",
            "time_usec": 1607693013403569
},
        {
            "favicon_url": "https://www.com-magazin.de/favicon.ico",
            "page_transition": "LINK",
            "title": "Google-Suchverlauf herunterladen und deaktivieren - com! professional",
            "url": "https://www.com-magazin.de/news/google/google-suchverlauf-herunterladen-deaktivieren-928063.html#:~:text\u003dUm%20die%20eigenen%20Suchanfragen%20herunterzuladen,Nutzer%20den%20Eintrag%20%22Herunterladen%22.",
            "client_id": "cWD5MfDDekj1z9aA5VeCQQ\u003d\u003d",
            "time_usec": 1607692994577620
}```
 
  

The code I use at the moment:

import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
from dhooks import Webhook, Embed
import json


def getHtml(url):
    global response
    ua = UserAgent()
    {'user-agent': ua.random}
    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()
    except Exception as e:
        print(e)
    return response.content


with open('urls.json', 'r') as history:
    json_data = history.read()
    data = json.loads(json_data)

    for block in data:
        print("scraping " + block["url"] + "...")
        html = getHtml(json_data)
        soup = BeautifulSoup(markup, "html5lib")
        text = soup.find_all(text=True)

        output = ''

        blacklist = [
            "style",
            "url",
            "404",
            "ngnix",
            "url"

        ]

        for t in text:
            if t.parent.name not in blacklist:
                output += '{} '.format(t)

        with open("{}.txt".format(i), "w") as out_fd:
            out_fd.write(output)

Answer 1

If your Source Data looks something like that,

[{
            "page_transition": "LINK",
            "title": "Niedersachsen nimmt geplante Corona-Lockerungen für Silvester zurück",
            "url": "https://www.rnd.de/politik/niedersachsen-nimmt-geplante-corona-lockerungen-fur-silvester-zuruck-IEW2P4XT4M24ZFFZSX7ILE6JGM.html?outputType\u003damp\u0026utm_source\u003dupday\u0026utm_medium\u003dreferral",
            "client_id": "59VD9fg/2RVO1jSDxOwfxw\u003d\u003d",
            "time_usec": 1607593733981438
},{
            "page_transition": "LINK",
            "title": "Niedersachsen nimmt geplante Corona-Lockerungen für Silvester zurück",
            "url": "https://www.rnd.de/politik/niedersachsen-nimmt-geplante-corona-lockerungen-fur-silvester-zuruck-IEW2P4XT4M24ZFFZSX7ILE6JGM.html?outputType\u003damp\u0026utm_source\u003dupday\u0026utm_medium\u003dreferral",
            "client_id": "59VD9fg/2RVO1jSDxOwfxw\u003d\u003d",
            "time_usec": 1607593733981438
}, {
            "page_transition": "LINK",
            "title": "Niedersachsen nimmt geplante Corona-Lockerungen für Silvester zurück",
            "url": "https://www.rnd.de/politik/niedersachsen-nimmt-geplante-corona-lockerungen-fur-silvester-zuruck-IEW2P4XT4M24ZFFZSX7ILE6JGM.html?outputType\u003damp\u0026utm_source\u003dupday\u0026utm_medium\u003dreferral",
            "client_id": "59VD9fg/2RVO1jSDxOwfxw\u003d\u003d",
            "time_usec": 1607593733981438
}]

and if I'm reading your question correctly, it's better off parsing your Source Data as JSON directly, and grab the URL with the 'url' key.

with open ('history.json','r') as history:
    json_data = history.read()
    data = json.loads(json_data)

    for k, v in data.items():  #because now your Source data is a dictionary
        for block in v:        #because v is the list of textblocks
            print("scraping " + block["url"] + "...")