如何在python中删除样式和元素后解析代码

Question

This is a very basic question regarding html parsing:

I am new to python(coding,computer science, etc), teaching myself to parse html and I have imported both pattern and beautiful soup modules to parse with. I found this code on the internet to cut out all formatting.

import requests
import json
import urllib
from lxml import etree
from pattern import web
from bs4 import BeautifulSoup


url = "http://webrates.truefx.com/rates/connect.html?f=html"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)

# kill all script and style elements
for script in soup(["script", "style"]):
    script.extract()    # rip it out

# get text
text = soup.get_text()

# break into lines and remove leading and trailing space on each
lines = (line.strip() for line in text.splitlines())
# break multi-headlines into a line each
chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
# drop blank lines
text = '\n'.join(chunk for chunk in chunks if chunk)


print(text)

This produces the following Output:

EUR/USD14265522866931.056661.056751.056081.057911.05686USD/JPY1426552286419121.405121.409121.313121.448121.382GBP/USD14265522866821.482291.482361.481941.483471.48281EUR/GBP14265522865290.712790.712900.712300.713460.71273USD/CHF14265522866361.008041.008291.006551.008791.00682EUR/JPY1426552286635128.284128.296128.203128.401128.280EUR/CHF14265522866551.065121.065441.063491.066281.06418USD/CAD14265522864891.278211.278321.276831.278531.27746AUD/USD14265522864960.762610.762690.761150.764690.76412GBP/JPY1426552286682179.957179.976179.854180.077179.988

now from this point how can I parse further to say If I only want the string 'USD/CHF' or a particular point of data?

Is there a easier method to webscrape and parse with? Any suggestions would be great!

System Specs: windows 7 64bit IDE: idle python: 2.7.5

Thank you all in advance, Rusty

Answer 1

Keep it simple . Find the cell by text ( USD/CHF , for example) and get the following siblings :

text = 'USD/CHF'
cell = soup.find('td', text=text)
for td in cell.next_siblings:
    print td.text

Prints:

1426561775912
1.00
768
1.00
782
1.00655
1.00879
1.00682

Answer 2

In my experience, beautiful soup is pretty much as easy as it gets. I would write a regex to strip the number out after a string of characters. I hope this gets you on the right track.

Answer 3

You can try something quick and dirty like this. Obviously code like this will change based on the string itself. More advanced ways would use python's regex library. But sometimes it's nice to keep it simple.

string = []
starting_position = text[text.find("USD/CHF")+7:] #+7 to start after the tag USD/CHF
for ch in starting_position:
    if ch.isdigit() == True or ch == ".":
        string.append(str(ch))
    else:
        break
print "".join(string)