网站抓取后仅写一行

Question

I am trying to extract a list of all the golf courses in the USA through this link . I need to extract the name of the golf course, address, and the phone number. My script is suppose to extract all the data from the website but it looks like it only prints one row in my csv file. I noticed that when I print the "name" field it only prints once despite the find_all function. All I need is the data and not just one field from multiple links on the website.

How do I go about fixing my script so that it prints all the needed data into a CSV file.

Here is my script:

import csv
import requests 
from bs4 import BeautifulSoup

courses_list = []

for i in range(1):
 url="http://www.thegolfcourses.net/page/1?ls&location=California&orderby=title&radius=6750#038;location=California&orderby=title&radius=6750" #.format(i)
 r = requests.get(url)
 soup = BeautifulSoup(r.content)


g_data2=soup.find_all("div",{"class":"list"})

for item in g_data2:
  try:
    name= item.contents[7].find_all("a",{"class":"entry-title"})[0].text
    print name
  except:
        name=''
  try:
    phone= item.contents[7].find_all("p",{"class":"listing-phone"})[0].text
  except:
      phone=''
  try:
    address= item.contents[7].find_all("p",{"class":"listing-address"})[0].text
  except:
      address=''

  course=[name,phone,address]
  courses_list.append(course)


with open ('PGN_Final.csv','a') as file:
  writer=csv.writer(file)
  for row in courses_list:
          writer.writerow([s.encode("utf-8") for s in row])

Answer 1

Here is a neat implementation for your code. You can use the library urllib2 instead of requests . And bs4 works the same though.

import csv
import urllib2
from BeautifulSoup import *

url="http://www.thegolfcourses.net/page/1?ls&location=California&orderby=title&radius=6750#038;location=California&orderby=title&radius=6750" #.format(i)
r = urllib2.urlopen(url).read()
soup = BeautifulSoup(r)

courses_list = []
courses_list.append(("Course name","Phone Number","Address"))

names = soup.findAll('h2', attrs={'class':'entry-title'})
phones = soup.findAll('p', attrs={'class':'listing-phone'})
address = soup.findAll('p', attrs={'class':'listing-address'})
for na, ph, add in zip(names,phones, address):
    courses_list.append((na.text,ph.text,add.text))


with open ('PGN_Final.csv','a') as file:
    writer=csv.writer(file)
    for row in courses_list:
        writer.writerow([s.encode("utf-8") for s in row])