[英]Search Disallow: / in robots.txt
I want to search Disallow: /
in robots.txt of domains. 我想在域的robots.txt中搜索“
Disallow: /
。
I have wrote regex, but it is not working. 我已经写了正则表达式,但是不起作用。
if(preg_match("!Disallow:\s*/\s\r\n!i",$string,$disallow_char))
{
print_r($disallow_char);
}
Following are two test cases. 以下是两个测试案例。
1) 1)
User-agent: *
Disallow: /
2) 2)
User-agent: *
Disallow: /product/generate_pdf/40
Disallow: /news/
Disallow: /news/bollards
Disallow: /product/generate_pdf/44
Disallow: /
Disallow: /page_management/insert
Disallow: /glossary/ajax_call/update_words
It should output true for both the cases. 在两种情况下均应输出true。
You need to assert that either a newline sequence or the end of the string follows: 您需要断言换行序列或字符串的结尾如下:
echo preg_match('~Disallow:\h*/(?:\R|$)~i', $string)
Explanation : 说明 :
Disallow: # 'Disallow:'
\h* # horizontal whitespace (0 or more times)
/ # '/'
(?: # group, but do not capture:
\R # '\R' (any Unicode newline sequence)
| # OR
$ # before an optional \n, and the end of the string
) # end of grouping
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.