.htaccess & error pages - Stay on page and don't redirect

Question

This seems very simple. ErrorDocument redirection to another page:

RewriteEngine On

ErrorDocument 400 /err/e=400
ErrorDocument 401 /err/e=401
ErrorDocument 403 /err/e=403
ErrorDocument 404 /err/e=404
ErrorDocument 500 /err/e=500

But the problem with these custom error pages is that the browser actually redirects to the page /err/e=404 instead of just showing its content. When not using custom error pages, the URL in the browser stays the same, thus allowing the user to correct the URL without having to type it again.

Question: How to create custom error pages that do not alter the URL and leave it like domain.tld/nonexistantfile ?

Background: Debian Wheezy, Apache2 and PHP are used.

Update posting the entire .htaccess file

RewriteEngine On

ErrorDocument 400 /err/e=400
ErrorDocument 401 /err/e=401
ErrorDocument 403 /err/e=403
ErrorDocument 404 /err/e=404
ErrorDocument 500 /err/e=500

RewriteRule ^favicon.ico$ favicon.ico [L]
RewriteRule ^css/(.*) css/$1 [L]
RewriteRule ^downloads/(.*) downloads/$1 [L]
RewriteRule ^images/(.*) images/$1 [L]
RewriteRule ^js/(.*) js/$1 [L]

RewriteRule index.php(.*) index.php$1 [L]
RewriteRule ^(.*)/(.*=.*) index.php?controller=$1&$2 [L]
RewriteRule ^(.*=.*) index.php?$1 [L]
RewriteRule ^(.*) index.php?controller=$1 [L]

The last part enables URL's like /home or page1/page2/param=123

Answer 1

You must have a rule to avoid rewrite for real files/directories like this:

ErrorDocument 400 /err/e=400
ErrorDocument 401 /err/e=401
ErrorDocument 403 /err/e=403
ErrorDocument 404 /err/e=404
ErrorDocument 500 /err/e=500

RewriteEngine On

RewriteCond %{REQUEST_FILENAME} -d [OR]
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^ - [L]

RewriteRule ^([^/]+)/([^=]+=.*)$ index.php?controller=$1&$2 [L,QSA]
RewriteRule ^([^=]+=.*)$ index.php?$1 [L,QSA]
RewriteRule ^(.+)$ index.php?controller=$1 [L,QSA]

I also removed some redundant rules.