Python 正则表达式捕获各种 url 模式组

Question

I have dataset containing the string like this and I want to remove the all the urls from the it

http://google.com having trouble finding regex https://google.com for this case http // google com / test some gibberish https // google. com / test / test1 great http.//google.org

Now, I am using this regex pattern to find the all urls:

https?:?\s?\/\/\s?\S+

Now, ideally, it should capture all the urls such as in this case,

• http://google.com

• https://google.com

• http // google com / test

• https // google. com / test / test1

• http.//google.org

but with the regex pattern I have, it is capturing only

• http://google.com

• https://google.com

• http // google

• https // google

Link to Regex .

Answer 1

You may use

https?[:.]?\s?\/\/(?:\s*[^\/\s.]+)+(?:\s*\.\s*[^\/\s.]+)*(?:\s*\/\s*[^\/\s]+)*

See the regex demo .

Details

• https? - http or https
• [:.]? - an optional : or .
• \s? - an optional whitespace - \/\/ - // char sequence
• (?:\s*[^\/\s.]+)+ - (to match all domain name parts till the last . before TLD) 1 or more occurrences of
  • \s* - 0 or more whitespaces
  • [^\/\s.]+ - 1 or more chars other than / , . and whitespace
• (?:\s*\.\s*[^\/\s.]+)* - 0 or more sequences of
  • \s*\.\s* - a dot enclosed with 0+ whitespaces
  • [^\/\s.]+ - 1 or more chars other than / , . and whitespace
• (?:\s*\/\s*[^\/\s]+)* - 0 or more sequences of
  • \s*\/\s* - a / enclosed with 0+ whitespaces
  • [^\/\s]+ - 1 or more chars other than / and whitespace

Answer 2

The joy of Python: practically anything you can imagine already exists as a library.

See https://github.com/madisonmay/CommonRegex

This package provides many regex formats (including URL and many others) so you don't have to rediscover how to.