如何在Java中使用正则表达式模式从HTML页面提取数据

Question

I'm trying to extract data from an html page as to store them in a String array

In the HTML page values are shown like this

 <tbody>
                      <tr>
                        <td style="width: 14%;">88055</td>
                        <td style="width: 19%;" class="gris">Ville</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/88055/" >Amos</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/880/" >Abitibi</a></td>
                      </tr>
                      <tr>
                        <td style="width: 14%;">85080</td>
                        <td style="width: 19%;" class="gris">Village</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/85080/" >Angliers</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/850/" >Témiscamingue</a></td>
                      </tr>
                      <tr>
                        <td style="width: 14%;">87050</td>
                        <td style="width: 19%;" class="gris">Municipalité</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/87050/" >Authier</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/870/" >Abitibi-Ouest</a></td>
                      </tr>

I need to extract only the string where the href = Municipality

witch means Amos ,Angliers , etc... and store them into an array of string

So far I have tried this and I'm lost

  public static final String EXPRESSION = "";//How to write the regex expression?
String [] data = new String [20]
    URL url = new URL("http://myur.com");


 BufferedReader in  = new BufferedReader(new InputStreamReader(url.openStream()));

        while ((ligne = in.readLine()) != null) {
          //What to write here? 
            }
            in.close();

PS : I'm aware the best method is to use an HTML parser instead but I'm really forced to apply this way

Much appreciation ,

Bass

Answer 1

You can use something like this to hardcode match the url having municipalite and get the text inside wrt to > and < characters.

This is my data file:

 <tbody>
                      <tr>
                        <td style="width: 14%;">88055</td>
                        <td style="width: 19%;" class="gris">Ville</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/88055/" >Amos</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/880/" >Abitibi</a></td>
                      </tr>
                      <tr>
                        <td style="width: 14%;">85080</td>
                        <td style="width: 19%;" class="gris">Village</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/85080/" >Angliers</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/850/" >Témiscamingue</a></td>
                      </tr>
                      <tr>
                        <td style="width: 14%;">87050</td>
                        <td style="width: 19%;" class="gris">Municipalité</td>
                        <td style="width: 33%;"><a href="repertoire-des-municipalites/fiche/municipalite/87050/" >Authier</a></td>
                        <td style="width: 34%;"><a href="repertoire-des-municipalites/fiche/mrc/870/" >Abitibi-Ouest</a></td>
                      </tr>

Here is the java file:

import java.util.*;
import java.util.regex.*;
import java.lang.*;
import java.io.*;

class test
{
    public static void main (String[] args) throws java.lang.Exception
    {
        BufferedReader in  = new BufferedReader(new FileReader(new File("data")));
        String line="";
        Pattern p=Pattern.compile("href\\s*=\\s*(?:\"|').*municipalite/[^>]*>(?:<.*>)*([^<]*)<.*$");
        while ((line = in.readLine()) != null)
        {
            Matcher m=p.matcher(line);
            while(m.find())
                System.out.println(m.group(1)); 
        }
        in.close();
    }
}

Output:

$ javac test.java 
$ java test 
Amos
Angliers
Authier
$

Regular expression breakdown:

href\\s*=\\s*(?:\"|').*municipalite/[^>]*>(?:<.*>)*([^<]*)<.*$

1. href\\\\s*=\\\\s* matches href following by 0 or more spaces followed by = and then 0 or more spaces

2. (?:\\"|') -> (?:) means a non capturing group ie it matches single or double quotes but doesn't capture/remember it

3. .*municipalite/ matches any char till municipalite/ occurs

4. [^>]*>(?:<.*>)* matches any char that is not a > for the rest of the url and then matches > , then tries to match 0 or more (all optional) opening tags into a non capturing group using this (?:<.*>)

5. ([^<]*) this group actually captures your string into group 1

6. <.*$ matches the rest of the line

Answer 2

I have shown in python. But the regex is the same in Java, I believe. Use Java functions to find the matches.

import re
reg = r"<a href=.*?municipalite.*?>(.+?)</a>"
result = re.findall(html)

Answer 3

Try ".*\\\\bhref=\\"repertoire-des-municipalites/fiche/municipalite/\\\\d+/\\"[^>]*>([^<]*)<.*"

My demo code (below) gives console output:

Console Output

Amos
Angliers
Authier

Demo Code

public class HrefRegex
{
    public static void main(final String[] args)
    {
        final String[] sampleLines = new String[] {
            "  </tr>",
            "    <td style=\"width: 14%;\">88055</td>",
            "    <td style=\"width: 19%;\" class=\"gris\">Ville</td>",
            "    <td style=\"width: 33%;\"><a href=\"repertoire-des-municipalites/fiche/municipalite/88055/\" >Amos</a></td>",
            "    <td style=\"width: 34%;\"><a href=\"repertoire-des-municipalites/fiche/mrc/880/\" >Abitibi</a></td>",
            "  </tr>",
            "  <tr>",
            "    <td style=\"width: 14%;\">85080</td>",
            "    <td style=\"width: 19%;\" class=\"gris\">Village</td>",
            "    <td style=\"width: 33%;\"><a href=\"repertoire-des-municipalites/fiche/municipalite/85080/\" >Angliers</a></td>",
            "    <td style=\"width: 34%;\"><a href=\"repertoire-des-municipalites/fiche/mrc/850/\" >Témiscamingue</a></td>",
            "  </tr>",
            "  <tr>",
            "    <td style=\"width: 14%;\">87050</td>",
            "    <td style=\"width: 19%;\" class=\"gris\">Municipalité</td>",
            "    <td style=\"width: 33%;\"><a href=\"repertoire-des-municipalites/fiche/municipalite/87050/\" >Authier</a></td>",
            "    <td style=\"width: 34%;\"><a href=\"repertoire-des-municipalites/fiche/mrc/870/\" >Abitibi-Ouest</a></td>",
            "  </tr>",
          };


        final Pattern pattern = Pattern.compile(".*\\bhref=\"repertoire-des-municipalites/fiche/municipalite/\\d+/\"[^>]*>([^<]*)<.*");

        for (final String s : sampleLines)
        {
            final Matcher matcher = pattern.matcher(s);

            if (matcher.matches())
            {
                System.out.println(matcher.group(1));
            }
        }
    }
}