在大字符串中搜索文件路径。 返回文件路径+文件名

Question

I've got a little project where I'm trying to download a series of wallpapers from a web page. I'm new to python.

I'm using the urllib library, which is returning a long string of web page data which includes

<a href="http://website.com/wallpaper/filename.jpg">

I know that every filename I need to download has

'http://website.com/wallpaper/'  

How can i search the page source for this portion of text, and return the rest of the image link, ending with "*.jpg" extension?

r'http://website.com/wallpaper/ xxxxxx .jpg'

I'm thinking if I could format a regular expression with the xxxx portion not being evaluated? Just check for the path, and the .jpg extension. Then return the whole string once a match is found

Am I on the right track?

Answer 1

I think a very basic regex will do.
Like:

(http:\/\/website\.com\/wallpaper\/[\w\d_-]*?\.jpg)

and if you use $1 this will return the whole String .

And if you use

(http:\/\/website\.com\/wallpaper\/([\w\d_-]*?)\.jpg)

then $1 will give the whole string and $2 will give the file name only.

Note: escaping ( \\/ ) is language dependent so use what is supported by python.

Answer 2

BeautifulSoup is pretty convenient for this sort of thing.

import re
import urllib3
from bs4 import BeautifulSoup

jpg_regex = re.compile('\.jpg$')
site_regex = re.compile('website\.com\/wallpaper\/')

pool = urllib3.PoolManager()
request = pool.request('GET', 'http://your_website.com/')
soup = BeautifulSoup(request)

jpg_list = list(soup.find_all(name='a', attrs={'href':jpg_regex}))
site_list = list(soup.find_all(name='a', attrs={'href':site_regex}))

result_list = map(lambda a: a.get('href'), jpg_list and site_list)

Answer 3

Don't use a regular expression against HTML.

Instead, use a HTML parsing library.

BeautifulSoup is a library for parsing HTML and urllib2 is a built-in module for fetching URLs

import urllib2
from bs4 import BeautifulSoup as bs

content = urllib2.urlopen('http://website.com/wallpaper/index.html').read()
html = bs(content)
links = [] # an empty list

for link in html.find_all('a'):
   href = link.get('href')
   if '/wallpaper/' in href:
      links.append(href)

Answer 4

Search for the " http://website.com/wallpaper/ " substring in url and then check for ".jpg" in url, as shown below:

domain = "http://website.com/wallpaper/"
url = str("your URL")
format = ".jpg"
for domain in url and format in url:
    //do something