正则表达式从字符串中提取所有URL

Question

I have a string like this

http://example.com/path/topage.htmlhttp://twitter.com/p/xyanhshttp://httpget.org/get.zipwww.google.com/privacy.htmlhttps://goodurl.net/

I would like to extract all url / webaddress into a Array. for example

urls = ['http://example.com/path/topage.html','http://twitter.com/p/xyan',.....]

Here is my approach which didn't work.

import re
strings = "http://example.com/path/topage.htmlhttp://twitter.com/p/xyanhshttp://httpget.org/get.zipwww.google.com/privacy.htmlhttps://goodurl.net/"
links = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', strings)

print links
// result always same as strings 

Answer 1

The problem is that your regex pattern is too inclusive. It includes all urls. You can use lookahead by using (?=)

Try this:

re.findall("((www\.|http://|https://)(www\.)*.*?(?=(www\.|http://|https://|$)))", strings)

Answer 2

Your problem is that http:// is being accepted as a valid part of a url. This is because of this token right here:

[$-_@.&+]

or more specifically:

$-_

This matches all characters with the range from $ to _ , which includes a lot more characters than you probably intended to do.

You can change this to [$\\-_@.&+] but this causes problems since now, / characters will not match. So add it by using [$\\-_@.&+/] . However, this will again cause problems since http://example.com/path/topage.htmlhttp would be considered a valid match.

The final addition is to add a lookahead to ensure that you are not matching http:// or https:// , which just so happens to be the first part of your regex!

http[s]?://(?:(?!http[s]?://)[a-zA-Z]|[0-9]|[$\-_@.&+/]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+

tested here

Answer 3

A simple answer without getting into much complication:

import re
url_list = []

for x in re.split("http://", l):
    url_list.append(re.split("https://",x))

url_list = [item for sublist in url_list for item in sublist]

In case you want to append the string http:// and https:// back to the urls, do appropriate changes to the code. Hope i convey the idea.

Answer 4

这是我的

(r’http[s]?://[a-zA-Z]{3}\.[a-zA-Z0-9]+\.[a-zA-Z]+')