Sub-folders not being indexed by Search Engines
Question
Early this year I registered a domain and installed Joomla and Wordpress to try out difference themes and features. Less than days later the main domain and a sub-folder got indexed by google as following:
- www.example.com (Wordpress installed in the public_html folder)
- www.emample.com/portal (Joomla installation files)
About two weeks ago I cleaned up everything and did the following:
- Created a simple HTML5 document (index.html) and placed in the public_html folder
- Created a sob folder called "wiki" (Dokuwiki installation files) www.emample.com/wiki
- Created a sob folder called "blog" (Wordpress installation files) www.emample.com/blog
Since than, only main page index.html has been indexed by google even though its has links to "wiki" and "blog". Both Wordpress and Dokuwiki have friendly URL (thanks to .httacess)
How could a get my sub-folders indexed by Google? Note: I do not have sub-domains.
Update 1:
The website: http://goo.gl/vVvyr9
1 answers
solution1
1 ACCPTED 2014-09-26 10:35:29
I've listed some topics relevant to your question:
robots.txt
This file instructs the search engine crawler what folders or files it should not index. So make sure your subfolders are not listed here. More information on: http://www.robotstxt.org/
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~joe/
nofollow & noindex
NoFollow attributes can be listed in the page header or on the links themselves. It tells the search engine crawler that certain links or whole pages should not be indexes.
<html>
<head>
<title>...</title>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>
More info: http://www.robotstxt.org/meta.html
<a href="signin.php" rel="nofollow">sign in</a>
More info: https://support.google.com/webmasters/answer/96569?hl=en
Sitemaps
Sitemaps are files containing location links to all your pages, making it easier for the search engines to index them. Consider submitting a sitemap in Google Webmaster Tools .
Content:
Google has some great guidelines for content and webmasters, make sure you are as compliant as you can be to maximise your search visibility.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.