服务器端urllib2.urlopen和用户的url的安全注意事项

Question

I'd like users to be able to upload images from the web by providing a URL. I don't think I can get the client to fetch the image and upload it due to possible CORS issues and hotlink prevention, so I'm getting the server to do it.

The biggest concern is the user simply entering file:///home/user/secret_image.jpg . This URL gets sent to the server and django happily fetches it, a local server side file, hosting it for the world to see. Is there a way to limit requests to external resources? How can this approach be made safe (at least -ish)?

Some other concerns may be the user providing a hostname that resolves to a local address or providing a public URL redirect to a local address. Even other devices on the LAN which wouldn't normally be accessible.

Filtering the URL text is not an option. I could perhaps check that the IP gets routed to a gateway and the destination is outside my subnet before allowing urllib to continue. I could block any redirects, but this may be a useful feature in some cases, so I could write a redirect handler to re-check the IP. Now this is starting to feel like a patch job and not a nice robust takes-care-of-all-cases solution.

I'm also doing basic stuff like read(max_size) in case the file is huuge, using python-magic to check the mimetype and pick an extension, using django-ratelimit and setting a timeout on the call to urlopen() .

EDIT: related answer

Answer 1

Give that this can involve significant network I/O, I'd move the work to a "file download worker", eg with Celery , but any other solution even homegrown will do for this part

Filesystem

Next part is, once you separate the main Django deployment from its workers, you can use a different and unprivileged OS user to run the worker, which you can be ran inside a chroot or another jail equivalent. This should limit what the worker can see on its filesystem.

Network Interfaces

As for the network interfaces, you can setting firewall rules, eg I believe on a Linux OS, iptables can set rules per user and per network interface. So that should allow you to limit what the worker's user can do on its network.

Bottom Line

Bottom line is, I'd move the task into a separate worker, with unprivileged and restricted access to the computer resources, instead of inheriting the permissions of the main Django app. The only privilege given to the worker will be to reach out to the internet, grab some files, write it back to disk, nothing more.