使用python从网页中提取姓名和电话号码

Question

What I want to do is, on this site:

http://www.yellowpages.com/memphis-tn/gift-shops

I want to extract the name of the shop and its associated phone number into a CSV. For example, the first entry should be:

Babcock Gifts, (901) 763-0700

etc..

I am using Python. After performing a urllib2.urlopen( ), I have the entire blurb. how do I process this text to achieve my goal?

Answer 1

I would suggest using regular expressions and hit on unique content in the lines.

IE:

<a href="http://www.yellowpages.com/memphis-tn/mip/babcock-gifts-14131113?lid=187490699" class="url " data-analytics="{&quot;click_id&quot;:1600,&quot;rank&quot;:1,&quot;act&quot;:1,&quot;FL&quot;:&quot;list&quot;,&quot;position&quot;:0}" title="Babcock Gifts">Babcock Gifts</a>

You would use something like:

re_name=re.compile('<a href=.*class=\"url\".*')
re_front=re.compile('^.*title="')
re_back=re.compile('".*')
for line in page:
 if re_name.search(line):
  out = re.front.sub('',line)
  out = re.back.sub('',line)
print out

Answer 2

I tried BeautifulSoup

 import urllib
 import re
 from BeautifulSoup import *
 url = 'http://www.yellowpages.com/memphis-tn/gift-shops' 

 u = urllib.urlopen(url) 
 soup = BeautifulSoup(u)

test = soup.findAll('div', {'class':"info"})

for each in test:
    aref = each.findAll('a',{'class':"url "})
    phone = each.findAll('span',{'class':"business-phone phone"})
        x = re.sub(r'[^0-9]',"",str(phone))
    print aref[0]['title'] + " - " + x

I derived this script by looking at source code of html page. I found the 'div' section which contained listings. And then each company is listed in tags, which I got in 'aref'.

Strangely, I picked up 'phone', but the text contained the whole string include tag. I am not sure why. So, I used a regex to substitute everything except numbers, which made up phone num.

here is the documentation for beautifulsoup. http://www.crummy.com/software/BeautifulSoup/bs3/documentation.html