如何正确使用python中的URL列表（.txt）？

Question

我需要检查无限数量的网站的可用性，你能告诉我如何实现一个 URL 列表，以便脚本从 the.txt 读取 URL 并循环每个 URL 吗？

我的近似非工作代码：

import requests
urls_list = open('G:\\urls_list.txt', 'r+')
for url in urls_list:
    response = requests.get(url)
    if response.status_code != 200:
        print('Not active'.format(url))

Answer 1

这是因为for url in urls_list: ...时， url字符串的末尾将包含一个换行符\n 。

您需要使用str.rstrip来删除它们。

此外，您最好在读取文件时使用上下文管理器（ with open("urls.txt") as f ），它会为您关闭文件。

import requests

with open("urls.txt") as f:
    for url in map(str.rstrip, f):

        print("-" * 34)
        print(f"{url = }")

        try:
            response = requests.get(url)
        except requests.ConnectionError as e:
            print(e)
            continue

        print(f"{response.status_code = }")

----------------------------------
url = 'https://github.com/'
response.status_code = 200
----------------------------------
url = 'https://githubb.com/'
HTTPSConnectionPool(host='githubb.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f26a6758f70>: Failed to establish a new connection: [Errno -2] Name or service not known'))
----------------------------------
url = 'https://www.gnu.org/software/bash/manual/bash.html'
response.status_code = 200
----------------------------------
url = 'https://www.gnu.org/software/bash/manual/bashh.html'
response.status_code = 404
----------------------------------
url = 'https://stackoverflow.com/'
response.status_code = 200
----------------------------------
url = 'https://stackoverfloww.com/'
response.status_code = 200