Disallow robots can be bypassed with htaccess?

Question

I have a simple question. Let's say that I have this in robots.txt:

User-agent: *
Disallow: /

And something like this in .htaccess:

RewriteRule ^somepage/.*$ index.php?section=ubberpage&parameter=$0

And of course in index.php something like:

$imbaVar = $_GET['section']
// Some splits some whatever to get a specific page

include("pages/theImbaPage.html") // Or php or whatever

Will the robots be able to see what's in that html included by the script ( site.com/somepage )? I mean... the URL points to an inaccessible place... (the /somepage is disallowed) but still it is redirected to a valid place ( index.php ).

Answer 1

不能。禁止机器人访问，机器人不允许浏览您网站上的任何网页，并且他们遵守您的规则

Answer 2

Assuming the robots will respect the robots.txt , then it wouldn't be able to see any page in the site at all (you stated you used Disallow: / .

If the robots however do not respect your robots.txt file, then they would be able to see the content, as the redirection is made server side.