Magento Layered Navigation & SEO

Question

I had a questions about Magento layered navigation & seo.

It appears our site is being indexed with urls that are relevant to attributes for example www.abc.com/exampleproduct?brand=69

This is creating tonnes of issues with duplicate content. Has anyone ever come accross something like this and is there any good solution for it. Inchoo wrote a blog about it here: http://inchoo.net/online-marketing/magento-seo-how-to-handle-problems-caused-by-layered-navigation/ but it did not really come to a solid solution.

Thanks in advance, cm.

Answer 1

You can copy your Head.php file (/app/code/core/Mage/Page/Block/Html/Head.php) to the local directory (/app/code/local/Mage/Page/Block/Html/Head.php)

Here is how to implement modification of the new file:

public function getRobots()
    {
        if (empty($this->_data['robots'])) {
            $this->_data['robots'] = Mage::getStoreConfig('design/head/default_robots');
        }

        //Added NOINDEX, FOLLOW for category page with filter(s)
        if(Mage::app()->getFrontController()->getAction()->getFullActionName() == 'catalog_category_view'){
            $appliedFilters = Mage::getSingleton('catalog/layer')->getState()->getFilters();

            //var_dump($appliedFilters);  //<-- uncomment and see filters as array in page source code in meta robots tag.

            if(is_array($appliedFilters) && count($appliedFilters) > 0){
                $this->_data['robots'] = "NOINDEX, FOLLOW";
            }
        }

        return $this->_data['robots'];
    }

PS Please also note that you should add some checks for objects exist.

Mage::app()->getFrontController()->getAction()->getFullActionName()

Answer 2

The solid and elegant solution for hiding layered navigation for crawlers and fix SEO issues caused by the huge number of layered navigation URLs would be using PRG Pattern .

canonical, robots.txt, rel=nofollow etc. do not fully fix this issue or have at least some downsides or limitations.

The PRG Pattern solution works like a charm, ie not changing the UX regarding Layered Navigation and 100% reliable in terms of preventing crawlers from wasting crawl budget on useless duplicate content URLs.

Simply said, it's about replacing the GET request to a layered navigation/filter URL with a POST request (which search engine crawlers do not follow) before redirecting the user to the original layered navigation/filter URL.

For further details and reading, please see

1. Detailed explanation incl. sample request flow
2. Why robots.txt, rel=nofollow etc. are no satisfying solutions here
3. PRG Pattern Magento 2 Extension
4. PRG Pattern Demo

Answer 3

Try to use canonical url meta tag, and google, yahoo and other major search engine(s) will index only url specified by that meta tag. For this purpose i recommend: Yoast extension

Answer 4

In the video, there are some solutions like nofollow, Robots.txt and more. You can take a look at the following suggestions too.

1. You can use the canonical of respective product page/category page on all the dynamic/filter pages.

2. If you are facing the issue, Google webmaster is reporting duplicate meta tags because all pages are indexed and they are containing meta tags of the main page. Then you can go for the dynamic meta tags.

But using canonical of the main page is the best option. Hope these suggestions will help you! :)