解析 CSV 文件中的 URL - Python

Question

I have a CSV file of URL's and I'm attempting to write a code to loop through the URLs and append specific variables them I have in a dictionary. Unfortunately, whenever I attempt this using beautiful soup the program does not separate the URLs or only does the first URL. I understand this is likely a simple issue but I have been unable to resolve this using solutions from similar questions. Below I've attached an excerpt of the code. Thank you for any guidance.

csv_data:
'https://www.sec.gov/Archives/edgar/data/78003/000007800313000017,https://www.sec.gov/Archives/edgar/data/78003/000115752312004450,https://www.sec.gov/Archives/edgar/data/78003/000115752312002789,https://www.sec.gov/Archives/edgar/data/78003/000007800313000013,https://www.sec.gov/Archives/edgar/data/78003/000007800313000029,https://www.sec.gov/Archives/edgar/data/78003/000007800312000008,https://www.sec.gov/Archives/edgar/data/78003/000007800314000046'


content = requests.get(csv_data[1]).content
soup = BeautifulSoup(content, 'lxml')

reports = soup.find('myreports')

master_reports = []

for report in reports.find_all('report')[:-1]:

report_dict = {}
report_dict['name_short'] = report.shortname.text
report_dict['category'] = report.menucategory.text
report_dict['url'] = base_url + report.htmlfilename.text

master_reports.append(report_dict)

print(base_url + report.htmlfilename.text)
print(report.shortname.text)
print(report.menucategory.text)

Answer 1

Is this what you are looking for? splitting the url list and looping? If so, you'll have to collect the output for each loop, which isn't coded here.

csv_data = 'https://www.sec.gov/Archives/edgar/data/78003/000007800313000017,https://www.sec.gov/Archives/edgar/data/78003/000115752312004450,https://www.sec.gov/Archives/edgar/data/78003/000115752312002789,https://www.sec.gov/Archives/edgar/data/78003/000007800313000013,https://www.sec.gov/Archives/edgar/data/78003/000007800313000029,https://www.sec.gov/Archives/edgar/data/78003/000007800312000008,https://www.sec.gov/Archives/edgar/data/78003/000007800314000046'
csv_url_list = csv_data.split(',')
for url in csv_url_list:
    content = requests.get(url).content
    soup = BeautifulSoup(content, 'lxml')
    reports = soup.find('myreports')

    master_reports = []

    for report in reports.find_all('report')[:-1]:

    report_dict = {}
    report_dict['name_short'] = report.shortname.text
    report_dict['category'] = report.menucategory.text
    report_dict['url'] = base_url + report.htmlfilename.text

    master_reports.append(report_dict)

    print(base_url + report.htmlfilename.text)
    print(report.shortname.text)
    print(report.menucategory.text)