在Python中将html转换为json

Question

I am trying to convert some html files to json. From the beginning: I downloaded a kind of old dataset called SarcasmAmazonReviewsCorpus . It has several txt files, all with comments, reactions, name of product and so on, as it follows in the image:

I was able to pick up each txt file and using os module I created a list with every file content. The code was:

files_content = []

for filename in filter(lambda p: p.endswith("txt"), os.listdir(path)):
    filepath = os.path.join(path, filename)
    with open(filepath, mode='r') as f:
        files_content += [f.read()]

Then, I am trying to use Beatifulsoup:

soup = BeautifulSoup(files_content[2], 'html5lib')
soup

The output is like:

Is there a way that I can convert all the itens in the files_content list into a json file ? Tkanks for the help!

Answer 1

You might have to change soup.find in dictionary to get the data you want.

import json

dictionary = {
    "title": soup.find("title"),
    "date": soup.find("date")
}

json_object = json.dumps(dictionary, indent=4)

with open("saveFile.json", "w") as outfile:
    outfile.write(json_object)

Answer 2

As it looks like you are parsing some very simple HTML data, I think you could simply use xmltodict package for this:

data = []
for txt in txt_files:
   with open(txt, "r") as file:
     data += [xmltodict.parse(file.read())]

json_str = json.dumps(data)