Java正则表达式从文本中检索链接

Question

I have a input String as:

String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it";

I want to convert this text to:

Some content which contains link as http://www.google.com/relative-path/fruit.cgi?param1=abc&param2=xyz&myParam=pqr (URL Label) and some text after it

So here:

1) I want to replace the link tag with plain link. If the tag contains label then it should go in braces after the URL.

2) If the URL is relative, I want to prefix the base URL ( http://www.google.com ).

3) I want to append a parameter to the URL. (&myParam=pqr)

I am having issues retrieving the tag with URL and label, and replacing it.

I wrote something like:

public static void main(String[] args) {
    String text = "String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it";";
    text = text.replaceAll("&lt;", "<");
    text = text.replaceAll("&gt;", ">");
    text = text.replaceAll("&amp;", "&");

    // this is not working
    Pattern p = Pattern.compile("href=\"(.*?)\"");
    Matcher m = p.matcher(text);
    String url = null;
    if (m.find()) {
        url = m.group(1);

    }
}

// helper method to append new query params once I have the url
public static URI appendQueryParams(String uriToUpdate, String queryParamsToAppend) throws URISyntaxException {
    URI oldUri = new URI(uriToUpdate);
    String newQueryParams = oldUri.getQuery();
    if (newQueryParams == null) {
        newQueryParams = queryParamsToAppend;
    } else {
        newQueryParams += "&" + queryParamsToAppend;  
    }
    URI newUri = new URI(oldUri.getScheme(), oldUri.getAuthority(),
            oldUri.getPath(), newQueryParams, oldUri.getFragment());
    return newUri;
}

Edit1:

Pattern p = Pattern.compile("HREF=\"(.*?)\"");

This works. But then I want it to be capitalization agnostic. Href, HRef, href, hrEF, etc. all should work.

Also, how do I handle if my text has several URLs.

Edit2:

Some progress.

Pattern p = Pattern.compile("href=\"(.*?)\"");
Matcher m = p.matcher(text);
String url = null;
while (m.find()) {
  url = m.group(1);
  System.out.println(url);
}

This handles the case of multiple URLs.

Last pending issue is, how do I get hold of the label and replace the href tags in original text with URL and label.

Edit3:

By multiple URL cases, I mean there are multiple url present in given text.

String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it and another link <A HREF=\"/relative-path/vegetables.cgi?param1=abc&param2=xyz\">URL2 Label</A> and some more text";

Pattern p = Pattern.compile("href=\"(.*?)\"", Pattern.CASE_INSENSITIVE);
Matcher m = p.matcher(text);
String url = null;
while (m.find()) {
 url = m.group(1); // this variable should contain the link URL
 url = appendBaseURI(url);
 url = appendQueryParams(url, "license=ABCXYZ");
 System.out.println(url);
}

Answer 1

You can use apache commons text StringEscapeUtils to decode the html entities and then replaceAll , ie:

import org.apache.commons.text.StringEscapeUtils;

String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it";
String output = StringEscapeUtils.unescapeHtml4(text).replaceAll("([^<]+).+\"(.*?)\">(.*?)<[^>]+>(.*)", "$1https://google.com$2&your_param ($3)$4");
System.out.print(output);
// Some content which contains link as https://google.com/relative-path/fruit.cgi?param1=abc&param2=xyz&your_param (URL Label) and some text after it

---

Demos:

1. jdoodle
2. Regex Explanation

Answer 2

public static void main(String args[]) {
    String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it and another link <A HREF=\"/relative-path/vegetables.cgi?param1=abc&param2=xyz\">URL2 Label</A> and some more text";
    text = StringEscapeUtils.unescapeHtml4(text);
    Pattern p = Pattern.compile("<a href=\"(.*?)\">(.*?)</a>", Pattern.CASE_INSENSITIVE);
    Matcher m = p.matcher(text);
    while (m.find()) {
        text = text.replace(m.group(0), cleanUrlPart(m.group(1), m.group(2)));
    }
    System.out.println(text);
}

private static String cleanUrlPart(String url, String label) {
    if (!url.startsWith("http") && !url.startsWith("www")) {
        if (url.startsWith("/")) {
            url = "http://www.google.com" + url;
        } else {
            url = "http://www.google.com/" + url;
        }
    }
    url = appendQueryParams(url, "myParam=pqr").toString();
    if (label != null && !label.isEmpty()) url += " (" + label + ")";
    return url;
}

Output

Some content which contains link as http://www.google.com/relative-path/fruit.cgi?param1=abc&param2=xyz&myParam=pqr (URL Label) and some text after it and another link http://www.google.com/relative-path/vegetables.cgi?param1=abc&param2=xyz&myParam=pqr (URL2 Label) and some more text

Answer 3

// this is not working

Because your regex is case-sensitive.

Try:-

Pattern p = Pattern.compile("href=\"(.*?)\"", Pattern.CASE_INSENSITIVE);

Edit1 :
To get the label, use Pattern.compile("(?<=>).*?(?=</a>)", Pattern.CASE_INSENSITIVE) and m.group(0) .

Edit2 :
To replace the tag (including label) with your final string, use:-

text.replaceAll("(?i)<a href=\"(.*?)</a>", "new substring here")

Answer 4

Almost there:

public static void main(String[] args) throws URISyntaxException {
        String text = "Some content which contains link as <A HREF=\"/relative-path/fruit.cgi?param1=abc&param2=xyz\">URL Label</A> and some text after it and another link <A HREF=\"/relative-path/vegetables.cgi?param1=abc&param2=xyz\">URL2 Label</A> and some more text";
        text = StringEscapeUtils.unescapeHtml4(text);
        System.out.println(text);
        System.out.println("**************************************");
        Pattern patternTag = Pattern.compile("<a([^>]+)>(.+?)</a>", Pattern.CASE_INSENSITIVE);
        Pattern patternLink = Pattern.compile("href=\"(.*?)\"", Pattern.CASE_INSENSITIVE);
        Matcher matcherTag = patternTag.matcher(text);

        while (matcherTag.find()) {
            String href = matcherTag.group(1); // href
            String linkText = matcherTag.group(2); // link text
            System.out.println("Href: " + href);
            System.out.println("Label: " + linkText);
            Matcher matcherLink = patternLink.matcher(href);
            String finalText = null;
            while (matcherLink.find()) {
                String link = matcherLink.group(1);
                System.out.println("Link: " + link);
                finalText = getFinalText(link, linkText);
                break;
            }
            System.out.println("***************************************");
            // replacing logic goes here
        }
        System.out.println(text);
    }

    public static String getFinalText(String link, String label) throws URISyntaxException {
        link = appendBaseURI(link);
        link = appendQueryParams(link, "myParam=ABCXYZ");
        return link + " (" + label + ")";
    }

    public static String appendQueryParams(String uriToUpdate, String queryParamsToAppend) throws URISyntaxException {
        URI oldUri = new URI(uriToUpdate);
        String newQueryParams = oldUri.getQuery();
        if (newQueryParams == null) {
            newQueryParams = queryParamsToAppend;
        } else {
            newQueryParams += "&" + queryParamsToAppend;  
        }
        URI newUri = new URI(oldUri.getScheme(), oldUri.getAuthority(),
                oldUri.getPath(), newQueryParams, oldUri.getFragment());
        return newUri.toString();
    }

    public static String appendBaseURI(String url) {
        String baseURI = "http://www.google.com/";
        if (url.startsWith("/")) {
            url = url.substring(1, url.length());
        }
        if (url.startsWith(baseURI)) {
            return url;
        } else {
            return baseURI + url;
        }
    }