如何将Element从Jsoup保存到数据库

Question

I use Jsoup get all data from website and save element if match some content when i get. I want when we get element. If it match some thing character , I save element from database(MYSQL,Postgress...). I code look like :

Connection conn = Jsoup.connect("https://viblo.asia");
                Document doc = conn.userAgent("Mozilla").get();
                Elements elements = doc.getElementsByClass("post-feed").get(0).children();
                Elements list = new Elements();
                Elements strings = new Elements();
                for (Element element : elements) {
                    if (element.hasClass("post-feed-item")) {
                        list.add(element);
                        Element e = element.children().get(1).children().get(1).children().get(0);
                        if (e.text().matches("^.*?(Docker|docker|DOCKER).*$")) {
                            strings.add(e);
                            //save to element to DB
                        }

                    }
                }

                for (Element page : elements) {
                    if (links.add(URL)) {
                        //Remove the comment from the line below if you want to see it running on your editor
                        System.out.println(URL);
                    }
                    getPageLinks(page.attr("abs:href"));
                }

I want if title from element contain : "Docker" it save my element to Database. But in element, It contain div and some thing link url, img , content. How to i save it to database. What if I want to save each element in a field in a database that is feasible? If not I can convert element to html and save it? Please help.

Example html i want save data base:

<div class="post-feed-item">
 <a href="/u/HoanKi"><img src="https://images.viblo.asia/avatar/1d0e5458-ad41-4d1c-89db-292dc198b4fa.png" srcset="https://images.viblo.asia/avatar/1d0e5458-ad41-4d1c-89db-292dc198b4fa.png 1x, https://images.viblo.asia/avatar-retina/1d0e5458-ad41-4d1c-89db-292dc198b4fa.png 2x" class="avatar avatar--md mr-05"></a>
 <div class="post-feed-item__info">
  <div class="post-meta--inline">
   <div class="user--inline d-inline-flex">
    <!---->
    <a href="/u/HoanKi" class="mr-05">Hoàn Kì</a>
    <!---->
   </div>
   <div class="post-meta d-inline-flex align-items-center flex-wrap">
    <div class="text-muted mr-05">
     <span class="mr-05">about 3 hours ago</span>
     <button title="Copy URL" class="icon-btn _13z_mK0hRyRB3dPzawysKe_0"><i aria-hidden="true" class="fa fa-link"></i></button>
    </div>
    <!---->
    <!---->
   </div>
  </div>
  <div class="post-title--inline">
   <h3 class="word-break mr-05"><a href="/p/docker-chua-biet-gi-den-biet-dung-phan-3-docker-compose-3P0lPm6p5ox" class="link">Docker: Chưa biết gì đến biết dùng (Phần 3 docker-compose )</a></h3>
   <div class="tags" data-v-cbe11868>
    <a href="/tags/docker" class="el-tag _3wKNDsArij9ZFjXe8k4ryR_0 el-tag--info el-tag--mini" data-v-cbe11868>Docker</a>
   </div>
  </div>
  <!---->
  <div class="d-flex justify-content-between">
   <div class="d-flex">
    <div class="stats">
     <span title="Views" class="stats-item text-muted"><i aria-hidden="true" class="stats-item__icon fa fa-eye"></i> 62 </span>
     <span title="Clips" class="stats-item text-muted"><i aria-hidden="true" class="stats-item__icon fa fa-paperclip"></i> 1 </span>
     <span title="Comments" class="stats-item text-muted"><i aria-hidden="true" class="stats-item__icon fa fa-comments"></i> 0 </span>
    </div>
    <!---->
   </div>
   <div title="Score" class="points">
    <div class="carets">
     <i aria-hidden="true" class="fa fa-caret-up"></i>
     <i aria-hidden="true" class="fa fa-caret-down"></i>
    </div>
    <span class="text-muted">4</span>
   </div>
  </div>
 </div>
</div>

Answer 1

First, modify your logic for fetching post-feed-item like this-

Connection conn = Jsoup.connect("https://viblo.asia");
Document doc = conn.userAgent("Mozilla").get();

Elements elements = doc.getElementsByClass("post-feed-item"); //This will get the whole element.

for (Element element : elements) {
    String postFeeds = "";

    if (element.toString().contains("docker")) {
        postFeeds = postFeeds.concat(element.toString());  
        //save postFeeds to DB
    }
}

Extra

/**
 * Your parsed element may contain single quote ('). 
 * This will cause error while persisting.
 * to avoid this you need to escape single quote (')
 * with double single quote ('')
 */

 if (element.toString().contains("docker")) {
     postFeeds = postFeeds.concat(element.toString().replaceAll("'", "''"));  
     //save postFeeds to DB
 }

Second, What if I want to save each element in a field in a database that is feasible?

You don't need separate columns to store each element at the database. However you can save but the feasibility depends on your use case. If you just want to store the post-feed-items only for writing it back to your web page then it is not feasible.

Third, How can I convert element to html and save?

You don't need to convert the element to html but you need to convert the element to String if you want to save it the database.
All you need is a column type of BLOB data type (you can also save it as VARCHAR but BLOB is safer).

Update

How can I traverse all pages?

By looking at the source code of that page I found this is how you can get the total page number -

Elements pagination = doc.getElementsByAttributeValueMatching("href", "page=\\d");

int totalPageNo = Integer.parseInt(pagination.get(pagination.size() - 2).text());

then loop through each page.

for(int page = 1; page <= totalPageNo; page++) {
    Connection conn = Jsoup.connect("https://viblo.asia/?page=" + page);
    //rest of your code
}

Answer 2

I properly know what's your mean.Here are some views:
First you should clearify what`s your search for and make fields of tables in database. Such as according your ideas, you can make a table_docker table in db and there are field_id,field_content,field_start_time,field_links and so on in it.
Second you should code some utils of classes such as JsoupUtils which is get HTML and parse it , HtmlUtils which is used to handle the html remarks and download these pictures,DBUtils which is used to connect db and save data,POIUtils which is used to show your data,DataUtils which is used to handle your data by your ways.