How to get a url as a param in Node.js?

Question

CODE:

app.get("/:url", function(req,res) {
    var url = req.params.url;
    res.send(url);
});

---

PROBLEM:

This does not work.

If I try:

http://localhost:3000/https://www.google.com

I get:

Cannot GET /https://www.google.com

Answer 1

You can try this, using a regex:

var app = require('express')();

app.get(/^\/(.*)/, function (req, res) {

    var url = req.params[0];
    res.send(url);

});

app.listen(3000, () => console.log('Listening on 3000'));

When you run:

curl http://localhost:3000/https://www.google.com

the server should return:

https://www.google.com

Update

There is some controversy whether or not colons are legal in URLs.

See this question for details:

• Are colons allowed in URLs?

According to RFC 3986 this is a legal URL:

http://localhost:3000/https://tools.ietf.org/html/rfc3986

But note that while this is legal as well:

http://localhost:3000/https://tools.ietf.org/html/rfc3986#section-3.3

if you entered that URL in the browser, your server would only get:

/https://tools.ietf.org/html/rfc3986

in the request. For that reason while not strictly necessary I would still recommend URL-encoding the URL that you put in other URLs - see the answer by Zac Delventhal .

Experiment

Using the above code example, this command:

curl http://localhost:3000/https://www.google.com/

will output this:

https://www.google.com/

But this command:

curl 'http://localhost:3000/https://www.google.com/#fragment'

will output this:

https://www.google.com/

Note that I used single quotes above not because they are necessary here - they're not, see this:

echo http://localhost:3000/https://www.google.com/#fragment

but to show that the hash fragment doesn't disappear because it is treated as a comment by the shell in case anyone thought that that could be the reason. It is not sent even when the quotes are used, and what happens can be demonstrated with the -v switch of curl :

* Connected to localhost (127.0.0.1) port 3000 (#0)
> GET /https://www.google.com/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:3000
> Accept: */*

As you can see the hash fragment is not even sent with HTTP so it is impossible for your server to know even that it existed.

Incidentally, this also demonstrates that using un-encoded URL within other URLs would not mess up any proxy, because HTTP requests to the proxy servers send this:

GET https://www.google.com/ HTTP/1.1

and not this:

GET /https://www.google.com/ HTTP/1.1

so they cannot be confused. (Note the slash.)

Answer 2

That's because : and / are special characters used in constructing a URL. In other words, they are are not url-safe . If you want to send those characters as part of a URL path, using the default Express param parser, you have to percent-encode them.

Try this with your existing code:

curl http://localhost:3000/https%3A%2F%2Fwww.google.com

You should get back:

https://www.google.com

Another option would be to use a query parameter instead of a path variable. Modify your code snippet slightly to this:

app.get("*", function(req, res) {
    var url = req.query.url;
    res.send(url);
});

Then, you can test it with this command:

curl http://localhost:3000?url=https://www.google.com

And you should get back:

https://www.google.com

Though I would say it is still probably a good idea to percent-encode those characters, even though Express can parse them fine this way. It could potentially lead to weird behavior down the road.

Still another option would be to send the url as a string in the body of a POST request, but that may not be RESTful depending on your use case.