How can I restrict the search engine not to read our .php url because we want our clean urls to appear in search engines.
I am using htaccess file for removing .php from all the urls but the search engine is taking both the urls ie with .php and without .php.
I want to make the search engines read only the urls having NO .php extension.
Create a robots.txt file in your website root and include the following to avoid crawling .php extensions.
User-agent: *
Disallow: /*.php$
Ref: https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt?hl=en
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.