JSoup - Parse HTML tag by tag

Question

I'm actually developping a text parser in Java and I was asked to enhance it by parsing HTML with it. The parser's purpose is to divide the file parsed into 3 other files, one with all the words contained in the file, one with all sentences and the other with all questions.

The *.txt part works perfectly, but I got a problem when parsing HTML.

I create a temporary file with *.txt extension and pass it in my text parser, but if I pass an URL with HTML file linked which is formed like this:

<!DOCTYPE html>
    <head>
        ... some HTML here ...
    </head>
    <body>
        <ul class="some_menu">
            <li class="some_menu_item">n1</li>
            <li class="some_menu_item">n2</li>
            <li class="some_menu_item">n2</li>
        </ul>
        <div>
            This is a question ?
            This is a sentence .
            ... some other text ...
        </div>
    </body>
</html>

the question file will be filled with: n1 n2 n3 This is a question

So, I just was wondering, is there a way to parse with JSoup tags by tags so I can add a line feed each time a block is closed?

If you need some new informations, don't bother to ask!

I should have 3 output files, which are, for this example: 我应该有3个输出文件，对于此示例：

One with all the words

 n1 n2 n3 This is a question sentence ... some other words ...

One with all the sentences
```
 This is a sentence 
```
One with all the questions
```
 This is a question 
```

TimmyM

Answer 1

To get all the text in an html body, you can use:

Document doc = Jsoup.connect(url).get();
Elements body = doc.select("body");
String allText = body[0].text();

You can then split the text to get each word separate. To get the text in the div tag, you can use:

Elements div = doc.select("div");
String divText = div[0].text();

You can then split the divText to get each sentence.

Notice that the return type of the select query is actually a list of Element ie, Elements. That's because there can be more than one elements matching you select query. In this case, since there is only one element for each case we access it by accessing the index 0 of the returned array.

Edit: In order to iterate through all elements check this answer . Basically

Elements elements = doc.body().select("*");

for (Element element : elements) {
    System.out.println(element.text());
}

Though there might be elements with no texts so you can put a check on that.

Answer 2

There are quite a lot HTML parser available in market like

HTMLUnit
HTMLCleaner
Jericho
JSoup

https://en.wikipedia.org/wiki/Comparison_of_HTML_parsers

Thanks, Vineet

JSoup - Parse HTML tag by tag

Question

2 answers

solution1
0 ACCPTED 2016-01-28 10:36:48

solution2
-2 2016-01-28 10:54:09

JSoup - Parse HTML tag by tag

Question

2 answers

solution1 0 ACCPTED 2016-01-28 10:36:48

solution2 -2 2016-01-28 10:54:09

solution1
0 ACCPTED 2016-01-28 10:36:48

solution2
-2 2016-01-28 10:54:09