Apache responds with 404 instead of 403, ignoring F flag

Question

I have this simple project hierarchy:

The contents of the .htaccess file located at the /www root folder are:

RewriteEngine On

RewriteRule ^index\.php$ - [L]

RewriteCond %{REQUEST_FILENAME} "\.php$"
RewriteRule .* - [F]

The rule in .htaccess should return a 403 when requesting any .php file but index.php . The contents of the .php templates are not relevant.

If i'm not wrong, if the last part of the request is the name of an existing .php file and rewrite engine is on, Apache serves it whether you specify the .php part or not. In my case, if i comment out the last 2 lines of .htaccess , any of these urls:

localhost/pages/page 
localhost/pages/page/

return the contents of /pages/page.php

My issue is that, given the above .htaccess , if i write this url:

localhost/pages/page.php

i get a 403 as expected. But if i write any of:

localhost/pages/page
localhost/pages/page/

i get a 404 (not found) instead of the expected 403 . Why am i getting the 404 ?

Answer 1

Interesting... I can reproduce this behaviour on Apache 2.4, but cannot explain it - it looks like a bug in my opinion, since replacing F with R=403 produces the desired result (there should be no difference) - see below.

To clarify, for /pages/page to resolve to /pages/page.php in the first place is dependent on MultiViews being enabled (as already stated in comments). Or, you can do this with mod_rewrite - but there is no evidence of that here. MultiViews is disabled by default on Apache, but some shared hosts do enable it for some reason (it makes extensionless URLs "just work"). However, MultiViews is a common cause of unexpected conflicts with mod_rewrite. And the fact that extensionless URLs "just (unexpectedly) work" can cause additional problems with canonicalisation.

The reason why I think there is a "bug" is that if you simply change the F flag to R=403 then you do get the "expected" 403 response:

 RewriteCond %{REQUEST_FILENAME} "\.php$" RewriteRule.* - [R=403]

There should be no difference between F and R=403 . AFAIK, F is simply a shortcut.

---

I've tried with Options -MultiViews but i get same results

However, with MultiViews disabled then a 404 is the expected response.

With MultiViews disabled then if you request /pages/page , REQUEST_FILENAME will be of the form /abs/path/to/pages/page (no .php extension). Your condition does not match and so the request falls through to a 404.

You are better off working with MultiViews disabled. It is not "normal" for /pages/page and /pages/page/ to "magically" resolve to /pages/page.php . It is "normal" for /pages/page and /pages/page/ to result in a 404 in this scenario. If you are doing much with mod_rewrite, where the requested URL maps to the filesystem then you will likely get conflicts.

Note that MultiViews doesn't just apply to .php files. With MultiViews enabled you've got extensionless everything . Images, scripts, HTML, stylesheets etc. If you have multiple files with a common basename and mime-type, which is going to be served? eg. foo.php or foo.html ?

Of course, you can create mod_rewrite directives to enable this behaviour - but this is not automatic.

---

UPDATE: A more concise example...

Given a file /page.php (in the document root) and MultiViews is enabled .
(eg. Options +MultiViews )

Requesting /page (or /page/ ) results in MultiViews/mod_negotiation serving /page.php via than internal subrequest.

Now, lets block all access with mod_rewrite and serve a 403 Forbidden:

RewriteRule ^ - [F]

This should block every request with a 403. However, requesting /page or /page/ - a URL that is successfully rewritten by MultiViews - results in a 404 Not Found response instead!

However, change the above rule to use the R flag instead:

RewriteRule ^ - [R=403]

Now, everything returns a 403, including the two URLs mentioned above! But F and R=403 should result in the same behaviour.