用于查找<a>链接</a>的“href”值的正则表达式

Question

I need a regex pattern for finding web page links in HTML.

I first use @"(<a.*?>.*?</a>)" to extract links ( <a> ), but I can't fetch href from that.

My strings are:

1. <a href="www.example.com/page.php?id=xxxx&name=yyyy"....></a>
2. <a href="http://www.example.com/page.php?id=xxxx&name=yyyy"....></a>
3. <a href="https://www.example.com/page.php?id=xxxx&name=yyyy"....></a>
4. <a href="www.example.com/page.php/404"....></a>

1, 2 and 3 are valid and I need them, but number 4 is not valid for me ( ? and = is essential)

---

Thanks everyone, but I don't need parsing <a> . I have a list of links in href="abcdef" format.

I need to fetch href of the links and filter it, my favorite urls must be contain ? and = like page.php?id=5

Thanks!

Answer 1

I'd recommend using an HTML parser over a regex, but still here's a regex that will create a capturing group over the value of the href attribute of each links. It will match whether double or single quotes are used.

<a\s+(?:[^>]*?\s+)?href=(["'])(.*?)\1

You can view a full explanation of this regex at here .

Snippet playground:

 const linkRx = /<a\\s+(?:[^>]*?\\s+)?href=(["'])(.*?)\\1/; const textToMatchInput = document.querySelector('[name=textToMatch]'); document.querySelector('button').addEventListener('click', () => { console.log(textToMatchInput.value.match(linkRx)); });

 <label> Text to match: <input type="text" name="textToMatch" value='<a href="google.com"'> <button>Match</button> </label>

Answer 2

Using regex to parse html is not recommended

regex is used for regularly occurring patterns. html is not regular with it's format(except xhtml ).For example html files are valid even if you don't have a closing tag !This could break your code.

Use an html parser like htmlagilitypack

You can use this code to retrieve all href's in anchor tag using HtmlAgilityPack

HtmlDocument doc = new HtmlDocument();
doc.Load(yourStream);

var hrefList = doc.DocumentNode.SelectNodes("//a")
                  .Select(p => p.GetAttributeValue("href", "not found"))
                  .ToList();

hrefList contains all href`s

Answer 3

Thanks everyone (specially @plalx)

I find it quite overkill enforce the validity of the href attribute with such a complex and cryptic pattern while a simple expression such as

would suffice to capture all URLs. If you want to make sure they contain at least a query string, you could just use

---

My final regex string:

 st = @"((www\\.|https?|ftp|gopher|telnet|file|notes|ms-help):((//)|(\\\\\\\\))+ \\w\\d:#@%/;$()~_?\\+-=\\\\\\.&]*)"; st = @"<a href[^>]*>(.*?)</a>"; st = @"((([A-Za-z]{3,9}:(?:\\/\\/)?)(?:[-;:&=\\+\\$,\\w]+@)?[A-Za-z0-9.-]+|(?:www.|[-;:&=\\+\\$,\\w]+@)[A-Za-z0-9.-]+)((?:\\/[\\+~%\\/.\\w-_]*)?\\??(?:[-\\+=&;%@.\\w_]*)#?(?:[\\w]*))?)"; st = @"((?:(?:https?|ftp|gopher|telnet|file|notes|ms-help):(?://|\\\\\\\\)(?:www\\.)?|www\\.)[\\w\\d:#@%/;$()~_?\\+,\\-=\\\\.&]+)"; st = @"(?:(?:https?|ftp|gopher|telnet|file|notes|ms-help):(?://|\\\\\\\\)(?:www\\.)?|www\\.)"; st = @"(((https?|ftp|gopher|telnet|file|notes|ms-help):((//)|(\\\\\\\\))+)|(www\\.)[\\w\\d:#@%/;$()~_?\\+-=\\\\\\.&]*)"; st = @"href=[""'](?<url>(http|https)://[^/]*?\\.(com|org|net|gov))(/.*)?[""']"; st = @"(<a.*?>.*?</a>)"; st = @"(?:hrefs*=)(?:[s""']*)(?!#|mailto|location.|javascript|.*css|.*this.)(?.*?)(?:[s>""'])"; st = @"http://([\\\\w+?\\\\.\\\\w+])+([a-zA-Z0-9\\\\~\\\\!\\\\@\\\\#\\\\$\\\\%\\\\^\\\\&amp;\\\\*\\\\(\\\\)_\\\\-\\\\=\\\\+\\\\\\\\\\\\/\\\\?\\\\.\\\\:\\\\;\\\\'\\\\,]*)?"; st = @"http(s)?://([\\w-]+\\.)+[\\w-]+(/[\\w- ./?%&=]*)?"; st = @"(http|https)://([a-zA-Z0-9\\\\~\\\\!\\\\@\\\\#\\\\$\\\\%\\\\^\\\\&amp;\\\\*\\\\(\\\\)_\\\\-\\\\=\\\\+\\\\\\\\\\\\/\\\\?\\\\.\\\\:\\\\;\\\\'\\\\,]*)?"; st = @"((http|ftp|https):\\/\\/[\\w\\-_]+(\\.[\\w\\-_]+)+([\\w\\-\\.,@?^=%&amp;:/~\\+#]*[\\w\\-\\@?^=%&amp;/~\\+#])?)"; st = @"http://([\\\\w+?\\\\.\\\\w+])+([a-zA-Z0-9\\\\~\\\\!\\\\@\\\\#\\\\$\\\\%\\\\^\\\\&amp;\\\\*\\\\(\\\\)_\\\\-\\\\=\\\\+\\\\\\\\\\\\/\\\\?\\\\.\\\\:\\\\;\\\\'\\\\,]*)?"; st = @"http(s?)\\:\\/\\/[0-9a-zA-Z]([-.\\w]*[0-9a-zA-Z])*(:(0-9)*)*(\\/?)([a-zA-Z0-9\\-\\.\\?\\,\\'\\/\\\\\\+&amp;%\\$#_]*)?$"; st = @"(?<Protocol>\\w+):\\/\\/(?<Domain>[\\w.]+\\/?)\\S*";

my choice is

@"(?<Protocol>\\w+):\\/\\/(?<Domain>[\\w.]+\\/?)\\S*"

Second Use this:

 st = "(.*)?(.*)=(.*)";

Problem Solved. Thanks every one :)

Answer 4

Try this :

 public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            var res = Find(html);
        }

        public static List<LinkItem> Find(string file)
        {
            List<LinkItem> list = new List<LinkItem>();

            // 1.
            // Find all matches in file.
            MatchCollection m1 = Regex.Matches(file, @"(<a.*?>.*?</a>)",
                RegexOptions.Singleline);

            // 2.
            // Loop over each match.
            foreach (Match m in m1)
            {
                string value = m.Groups[1].Value;
                LinkItem i = new LinkItem();

                // 3.
                // Get href attribute.
                Match m2 = Regex.Match(value, @"href=\""(.*?)\""",
                RegexOptions.Singleline);
                if (m2.Success)
                {
                    i.Href = m2.Groups[1].Value;
                }

                // 4.
                // Remove inner tags from text.
                string t = Regex.Replace(value, @"\s*<.*?>\s*", "",
                RegexOptions.Singleline);
                i.Text = t;

                list.Add(i);
            }
            return list;
        }

        public struct LinkItem
        {
            public string Href;
            public string Text;

            public override string ToString()
            {
                return Href + "\n\t" + Text;
            }
        }

    }  

Input:

  string html = "<a href=\"www.aaa.xx/xx.zz?id=xxxx&name=xxxx\" ....></a> 2.<a href=\"http://www.aaa.xx/xx.zz?id=xxxx&name=xxxx\" ....></a> "; 

Result:

[0] = {www.aaa.xx/xx.zz?id=xxxx&name=xxxx}
[1] = {http://www.aaa.xx/xx.zz?id=xxxx&name=xxxx}

C# Scraping HTML Links

Scraping HTML extracts important page elements. It has many legal uses for webmasters and ASP.NET developers. With the Regex type and WebClient, we implement screen scraping for HTML.

Edited

Another easy way:you can use a web browser control for getting href from tag a ,like this:(see my example)

 public Form1()
        {
            InitializeComponent();
            webBrowser1.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(webBrowser1_DocumentCompleted);
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            webBrowser1.DocumentText = "<a href=\"www.aaa.xx/xx.zz?id=xxxx&name=xxxx\" ....></a><a href=\"http://www.aaa.xx/xx.zz?id=xxxx&name=xxxx\" ....></a><a href=\"https://www.aaa.xx/xx.zz?id=xxxx&name=xxxx\" ....></a><a href=\"www.aaa.xx/xx.zz/xxx\" ....></a>";
        }

        void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            List<string> href = new List<string>();
            foreach (HtmlElement el in webBrowser1.Document.GetElementsByTagName("a"))
            {
                href.Add(el.GetAttribute("href"));
            }
        }

Answer 5

Try this regex:

"href\\s*=\\s*(?:\"(?<1>[^\"]*)\"|(?<1>\\S+))"

You will get more help from discussions over:

Regular expression to extract URL from an HTML link

and

Regex to get the link in href. [asp.net]

Hope its helpful.

Answer 6

 HTMLDocument DOC = this.MySuperBrowser.Document as HTMLDocument;
 public IHTMLAnchorElement imageElementHref;
 imageElementHref = DOC.getElementById("idfirsticonhref") as IHTMLAnchorElement;

Simply try this code

Answer 7

I came up with this one, that supports anchor and image tags, and supports single and double quotes.

<[a|img]+\\s+(?:[^>]*?\\s+)?[src|href]+=[\"']([^\"']*)['\"]

So

<a href="/something.ext">click here</a>

Will match:

 Match 1: /something.ext

And

<a href='/something.ext'>click here</a>

Will match:

 Match 1: /something.ext

Same goes for img src attributes

Answer 8

I took a much simpler approach. This one simply looks for href attributes, and captures the value (between apostrophes) trailing it into a group named url:

href=['"](?<url>.*?)['"]

Answer 9

I think in this case it is one of the simplest pregmatches

/<a\s*(.*?id[^"]*")/g

gets links with the variable id in the address

starts from href including it, gets all characters/signs (. - excluding new line signs) until first id occur, including it, and next all signs to nearest next " sign ([^"]*)