Python: Writing multiple variables to a file

Question

I'm fairly new to Python and I've written a scraper that prints the data I scrap the exact way I need it, but I'm having trouble writing the data to a file. I need it to look the exact same way and be in the same order as it does when it prints in IDLE

import requests
import re
from bs4 import BeautifulSoup

year_entry = raw_input("Enter year: ")

week_entry = raw_input("Enter week number: ")

week_link = requests.get("http://sports.yahoo.com/nfl/scoreboard/?week=" + week_entry + "&phase=2&season=" + year_entry)

page_content = BeautifulSoup(week_link.content)

a_links = page_content.find_all('tr', {'class': 'game link'})

for link in a_links:
        r = 'http://www.sports.yahoo.com' + str(link.attrs['data-url'])
        r_get = requests.get(r)
        soup = BeautifulSoup(r_get.content)
        stats = soup.find_all("td", {'class':'stat-value'})
        teams = soup.find_all("th", {'class':'stat-value'})
        scores = soup.find_all('dd', {"class": 'score'})

        try:
                game_score = scores[-1]
                game_score = game_score.text
                x = game_score.split(" ")
                away_score = x[1]
                home_score = x[4]
                home_team = teams[1]
                away_team = teams[0]
                away_team_stats = stats[0::2]
                home_team_stats = stats[1::2]
                print away_team.text + ',' + away_score + ',',
                for stats in away_team_stats:
                        print stats.text + ',',
                print '\n'
                print home_team.text + ',' + home_score +',',
                for stats in home_team_stats:
                        print stats.text + ',',
                print '\n'

        except:
                pass

I am totally confused on how to get this to print to a txt file the same way it prints in IDLE. The code is built to only run on completed weeks of the NFL season. So if you test the code, I recommend year = 2014 and week = 12 (or before)

Thanks,

JT

Answer 1

To write to a file you need to build up the line as a string, then write that line to a file.

You'd use something like:

# Open/create a file for your output   
with open('my_output_file.csv', 'wb') as csv_out:
    ...
    # Your BeautifulSoup code and parsing goes here
    ...
    # Then build up your output strings
    for link in a_links: 
        away_line = ",".join([away_team.text, away_score])
        for stats in away_team_stats:
            away_line += [stats.text]
        home_line = ",".join(home_team.text, home_score])
        for stats in home_team_stats:
                home_line += [stats.text]

        # Write your output strings to the file   
        csv_out.write(away_line + '\n')
        csv_out.write(home_line + '\n')

This is a quick and dirty fix. To do it properly you probably want to look into the csv module ( docs )

Answer 2

From the structure of your output I agree with Jamie that using CSV is a logical choice.

But since you're using Python 2, it's possible to use an alternate form of the print statement to print to a file.

From https://docs.python.org/2/reference/simple_stmts.html#the-print-statement

print also has an extended form, defined by the second portion of the syntax described above. This form is sometimes referred to as “print chevron.” In this form, the first expression after the >> must evaluate to a “file-like” object, specifically an object that has a write() method as described above. With this extended form, the subsequent expressions are printed to this file object. If the first expression evaluates to None, then sys.stdout is used as the file for output.

Eg,

outfile = open("myfile.txt", "w")
print >>outfile, "Hello, world"
outfile.close()

However, this syntax is not supported in Python 3, so I guess it's probably not a good idea to use it. :) FWIW, I generally use the file write() method in my code when writing to files, except that I tend to use print >>sys.stderr for error messages.