简体   繁体   中英

Allow google to access just one folder and block the rest of site

EDIT: Doing this with robots.txt isn't good enough for me.

This code block entire site from google. I do not know how to keep this code functional but allow google to access one folder on the site.

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteRule ^(.*)$ - [F,L]

Does this works if i had 2 conditions?

Options +FollowSymLinks -MultiViews
# Turn mod_rewrite on
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} yahoo [OR]
RewriteCond %{REQUEST_URI} !^/allowed-folder/ [NC]
RewriteRule ^ - [F,L]

As suggested in comments create a file robots.txt under DOCUMENT_ROOT with following content:

User-Agent: googlebot
Disallow: /
Allow: /allowed-folder/

If you really want to handle this via mod_rewrite then enable mod_rewrite and .htaccess through httpd.conf and then put this code in your .htaccess under DOCUMENT_ROOT directory:

Options +FollowSymLinks -MultiViews
# Turn mod_rewrite on
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} Googlebot [NC]
RewriteCond %{REQUEST_URI} !^/allowed-folder/ [NC]
RewriteRule ^ - [F,L]

create file with name robots.txt

insert

User-agent: *
Disallow: /folder you dont wont google inside in/
Disallow: /other folder you dont wont google inside in/

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM