Check url is a file or directory

Question

Hi there.

I have a URL list. I do not know how to check is this address to a file or directory.

examples:

url = "http://example.com/path/to/file.html"

if '.' in url.split('/')[-1]:
    return True

but if url is

url = "http://example.com/path/domains/domain.com"

domain.com is a directory not a file. How to detect it?

Checking the file extension is not good, maybe some headers? But I want to do as little as possible internet transfer usage.

Edit:

I need to download a large number of links and map their path to the location in my operating system. eg example.com/path/to/file.html

~/Downloads/example.com/path/to/

and here download file.html. eg:

example.com/directory/  
create ~/Downlods/example.com/directory/ 
next url: example.com/directory/dir2  
create ~/Downloads/example.com/directory/dir2  
next url: example.com/directory/file.html  
Download file.html in too ~/Downloads/example.com/directory/
not too create file.html directory

Answer 1

In short, you can't. Accessing the URL http://example.com/path/domains/domain.com would send a 302 redirect (if I remember correctly) to http://example.com/path/domains/domain.com/ by default. There are no headers in the response that indicates if a URL points to a directory. May I ask why you need to know this? I suppose you can add a slash to a URL and see what happens from there. That might get you the results you are looking for.

Answer 2

On HTTP servers, there is no such "file" or "directory" things. You just send an URI to the server which identify a specific resource that depends of the server's configuration. By default, most of the HTTP servers use the files and directories of your system, but it can be configured (URL Rewriting, ...).