Scraping data from the tag names in python

Question

Hi I am trying to scrape user data from a website. I need User ID which are available in the tag names itself.I am trying to scrape the UID using python selenium and beautiful soup in the div tag.

Example:

<"div id="UID_**60CE07D6DF5C02A987ED7B076F4154F3**-SRC_328619641" class="memberOverlayLink" onmouseover="ta.trackEventOnPage('Reviews','show_reviewer_info_window','user_name_photo'); ta.call('ta.overlays.Factory.memberOverlayWOffset', event, this, 's3 dg rgba_gry update2012', 0, (new Element(this)).getElement('.avatar')&amp;&amp;(new Element(this)).getElement('.avatar').getStyle('border-radius')=='100%'?-10:0);">

I am trying to scrape the UID using python selenium and beautiful soup in the div tag . I have looked through all the documentation and several web pages but I can't find a solution for this. If anyone can please tell me if such a thing is possible I would be very grateful.

Answer 1

Assuming the id attribute value is always in the format UID_ followed by one or more alphanumeric characters followed by -SRC_ followed by one or more digits:

import re
from bs4 import BeautifulSoup

soup = BeautifulSoup(html)

pattern = re.compile(r"UID_(\w+)\-SRC_\d+")
id = soup.find("div", id=pattern)["id"]

uid = pattern.match(id).group(1)
print(uid)

Here we are using BeautifulSoup and searching for an id attribute value to match a specific regular expression . It contains a saving group (\\w+) that helps us to extract the UID value.

Answer 2

you can use .get method and scrape the tag names easily,

in your question;

soup.get('id')

of course, if there are many id tags exist, you need to use more specific tags with find or find_all method before using the .get