只获取网站的相关部分

Question

How does Evernote's Web Clipper plugin or Announcify plugin only get relevant article/post/content part of the page? Here is an screenshot from evernote plugin:

No matter which website you visit which is completely different from other layout wises, these are always able to get you article/post/content part of the page.

Each website has different layouts, some have sidebar, some don't, different tags, for main/article/content part, some use <article> or <section> of HTML5 others use <h1> > <p> , some use <h2> > <p> and others don't use at all. So there are different combination of tags as well as layouts of websites.

Can anyone suggest a solution to getting main article/post/content please via Javascript or PHP?

Answer 1

You can do a simple DOM parsing and search for the <div> s and <p> s containing more text ( text! not HTML code! ). However, regardless of the intelligent method you will choose for determining where the content is, you should start from DOM parsing , so let's have a look at DOM parsing PHP libraries.

Anyway, you can start from this:

http://w-shadow.com/blog/2008/01/25/extracting-the-main-content-from-a-webpage/

Looks quite good, and gives technical explanations if you want to write something of your own.

Answer 2

Most blog engines give that div an id of 'content'.

• In javascript you would just do $('#content')
• In php you would do DOMDocument::getElementById('content').