如何从<a>标签中</a>提取 url GET 参数<a>，从完整的 html 文本</a>

Question

So I have an html page. It's full of various tags, most of them have sessionid GET parameter in their href attribute. Example:

...
<a href="struct_view_distrib.asp?sessionid=11692390">
...
<a href="SHOW_PARENT.asp?sessionid=11692390">
...
<a href="nakl_view.asp?sessionid=11692390">
...
<a href="move_sum_to_7300001.asp?sessionid=11692390&mode_id=0">
...

So, as you see, sessionid is the same, i just need to get it's value into variable, no matter from which one: x=11692390 I'm newbie in regex, but google wasn't helpful. Thx a lot!

Answer 1

This does not use regexes, but anyway, this is what you would do in Python 2.6:

from BeautifulSoup import BeautifulSoup
import urlparse

soup = BeautifulSoup(html)
links = soup.findAll('a', href=True)

for link in links:
  href = link['href']
  url = urlparse.urlparse(href)
  params = urlparse.parse_qs(url.query)
  if 'sessionid' in params:
    print params['sessionid'][0]

Answer 2

Parse your HTML with a DOM parsing library and use getElementsByTagName('a') to grab anchors, iterate through them and use getAttribute('href') and then extract the string. Then you can use regex or split on ? to match/retrieve the session id.

Answer 3

I would do this - before I was told it was a python issue ;)

<script>
function parseQString(loc) {
  var qs = new Array();
  loc = (loc == null) ? location.search.substring(1):loc.split('?')[1];
  if (loc) {
    var parms = loc.split('&');
    for (var i=0;i<parms.length;i++) {
      nameValue = parms[i].split('=');
      qs[nameValue[0]]=(nameValue.length == 2)? unescape(nameValue[1].replace(/\+/g,' ')):null; // use null or ""
    }
  }
  return qs;
}
var ids = []; // will hold the IDs
window.onload=function() {
  var links = document.links;
  var id;
  for (var i=0, n=links.length;i<n;i++) {
    ids[i] = parseQString(links[i].href)["sessionid"];
  }
  alert(ids); // remove this when happy
  // here you can do 
  alert(ids[3]); 
  //to get the 4th link's sessionid
}


</script>

<a href="struct_view_distrib.asp?sessionid=11692390">
...</a>
<a href="SHOW_PARENT.asp?sessionid=11692390">
...</a>
<a href="nakl_view.asp?sessionid=11692390">
...</a>
<a href="move_sum_to_7300001.asp?sessionid=11692390&mode_id=0">
...</a>

Answer 4

下面是一个正则表达式，您可以使用它来匹配 href 并提取其值：

\b(?<=(href="))[^"]*?(?=")

Answer 5

Complete example for Python3, inspired by AbdealiJK:

response = """...
<a href="struct_view_distrib.asp?sessionid=11692390">
...
<a href="SHOW_PARENT.asp?sessionid=11692390">
...
<a href="nakl_view.asp?sessionid=11692390">
...
<a href="move_sum_to_7300001.asp?sessionid=11692390&mode_id=0">
..."""

from bs4 import BeautifulSoup
import urllib.parse
soup = BeautifulSoup(response, "lxml")
for i in soup.find_all('a', href=True):
    try:
        print(urllib.parse.parse_qs(urllib.parse.urlparse(i['href']).query)["sessionid"])
    except:
        pass

Answer 6

bs4 4.7.1.+ has all the functionality you need for this. Use css AND syntax combined with :not to specify url with param sessionid only and select_one to limit to first match, then split on that param and grab the ubound array value

soup.select_one("[href*='asp?sessionid']:not([href*='&'])")['href'].split('sessionid=')[-1]