提取 Ruby 中字符串内的所有 url

Question

I have some text content with a list of URLs contained in it.

I am trying to grab all the URLs out and put them in an array.

I have this code

content = "Here is the list of URLs: http://www.google.com http://www.google.com/index.html"

urls = content.scan(/^(http|https):\/\/[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(([0-9]{1,5})?\/.*)?$/ix)

I am trying to get the end results to be:

['http://www.google.com', 'http://www.google.com/index.html']

The above code does not seem to be working correctly. Does anyone know what I am doing wrong?

Thanks

Answer 1

Easy:

ruby-1.9.2-p136 :006 > require 'uri'
ruby-1.9.2-p136 :006 > URI.extract(content, ['http', 'https'])
  => ["http://www.google.com", "http://www.google.com/index.html"] 

Answer 2

I haven't checked the syntax of your regex, but String.scan will produce an array, each of whose members is an array of the groups matched by your regex. So I'd expect the result to be:

[['http', '.google.com'], ...]

You'll need non-matching groups /(?:stuff)/ if you want the format you've given.

Edit (looking at regex): Also, your regex does look a bit wrong. You don't want the start and end anchors ( ^ and $ ), since you don't expect the matches to be at start and end of content . Secondly, if your ([0-9]{1,5})? is trying to capture a port number, I think you're missing a colon to separate the domain from the port.

Further edit, after playing: I think you want something like this:

content = "Here is the list of URLs: http://www.google.com http://www.google.com/index.html http://example.com:3000/foo"
urls = content.scan(/(?:http|https):\/\/[a-z0-9]+(?:[\-\.]{1}[a-z0-9]+)*\.[a-z]{2,5}(?:(?::[0-9]{1,5})?\/[^\s]*)?/ix)
# => ["http://www.google.com", "http://www.google.com/index.html", "http://example.com:3000/foo"]

... but note that it won't match pure IP-address URLs (like http://127.0.0.1 ), because of the [az]{2,5} for the TLD.

Answer 3

一种不同的方法，从完美是好的敌人的思想流派：

urls = content.split(/\s+/).find_all { |u| u =~ /^https?:/ }

Answer 4

just for your interest:

Ruby has an URI Module, which has a regex implemented to do such things:

require "uri"

uris_you_want_to_grap = ['ftp','http','https','ftp','mailto','see']

html_string.scan(URI.regexp(uris_you_want_to_grap)) do |*matches|
  urls << $&
end

For more information visit the Ruby Ref: URI

Answer 5

The most upvoted answer was causing issues with Markdown URLs for me, so I had to figure out a regex to extract URLs. Below is what I use:

URL_REGEX = /(https?:\/\/\S+?)(?:[\s)]|$)/i
content.scan(URL_REGEX).flatten

The last part here (?:[\s)]|$) is used to identify the end of the URL and you can add characters there as per your need and content. Right now it looks for any space characters, closing bracket or end of string.

content = "link in text [link1](http://www.example.com/test) and [link2](http://www.example.com/test2)

http://www.example.com/test3

http://www.example.com/test4"

returns ["http://www.example.com/test", "http://www.example.com/test2", "http://www.example.com/test3", "http://www.example.com/test4"] .