How do I use scrapy re() selector?

Question

This is my regex :

".*\/(.*)\?ref"

This is my test string:

/product/sam/go-with-me?ref=popular

I can get : go-with-me
I did try on https://regex101.com/#python

But I don't know how to write with scrapy, it get nothing

Here is my code:

for site in sites:
    title = sel.css("a::text").re(r".*\/(.*)\?ref")
    print title
    break

Answer 1

It's difficult to say without seeing your real HTML input data, but you might just need to look into href attribute value instead of a text:

for site in sites:
    title = site.xpath(".//a/@href").re(r".*\/(.*)\?ref")
    print title
    break