Exporting BeautifulSoup/Python loop to CSV or txt file

Question

Not exactly sure how this should look, but I want this code to be written into a csv file. I've tried making taking the entire output and doing output.to_csv('output.csv') but I can't seem to get that to work.

import csv ;import requests
from bs4 import BeautifulSoup
import csv
import re

url_list = ['https://basketball.realgm.com/player/player/Summary/2',
            'https://basketball.realgm.com/player/player/Summary/1']

for url in url_list:
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')

    player = soup.find_all('div', class_='wrapper clearfix container')[0]

    playerprofile = re.sub(
        r'\n\s*\n', r'\n', player.get_text().strip(), flags=re.M)

    output = playerprofile + "\n"

Answer 1

If I get your question right, you need to open a file descriptor and write in append mode to it. file = open('output.csv', 'a' ) for URL in url_list: ... file.write(output)

Answer 2

You are using output.to_csv() which is the function of pandas but in your code you have imported csv module. To write in csv using csv module use this code below your output variable

with open('output.csv', 'a') as output_file:
  output_file = csv.writer(output_file, delimiter=',')
  output_file.writerow([output])

Feel free to ask me in case of any question.