为什么我在尝试使用 Python Flask 时收到“内部服务器错误”？

Question

我一直在关注本教程： https://kb.objectrocket.com/postgresql/scrape-a-website-to-postgres-with-python-938

我的 app.py 文件看起来像这样（取自上面的教程）：

from flask import Flask  # needed for flask-dependent libraries below
from flask import render_template  # to render the error page
from selenium import webdriver  # to grab source from URL
from bs4 import BeautifulSoup  # for searching through HTML
import psycopg2  # for database access

# set up Postgres database connection and cursor.
t_host = "localhost" # either "localhost", a domain name, or an IP address.
t_port = "5432" # default postgres port
t_dbname = "scrape"
t_user = "postgres"
t_pw = "********?"
db_conn = psycopg2.connect(host=t_host, port=t_port, dbname=t_dbname, user=t_user, password=t_pw)
db_cursor = db_conn.cursor()

app = Flask(__name__)


@app.route("/")
@app.route('/import_temp')
def import_temp():
    # set up your webdriver to use Chrome web browser
    my_web_driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")

    # designate the URL we want to scrape
    #   NOTE: the long string of characters at the end of this URL below is a clue that
    #   maybe this page is so dynamic, like maybe refers to a specific web session and/or day/time,
    #   that we can't necessarily count on it to be the same more than one time.
    #   Which means... we may want to find another source for our data; one that is more
    #   dependable. That said, whatever URL you use, the methodology in this lesson stands.
    t_url = "https://weather.com/weather/today/l/7ebb344012f0c5ff88820d763da89ed94306a86c770fda50c983bf01a0f55c0d"
    # initiate scrape of website page data
    my_web_driver.get("<a href='" + t_url + "'>" + t_url + "</a>")
    # return entire page into "t_content"
    t_content = my_web_driver.page_source
    # use soup to make page content easily searchable
    soup_in_bowl = BeautifulSoup(t_content)
    # search for the UNIQUE span and class for the data we are looking for:
    o_temp = soup_in_bowl.find('span', attrs={'class': 'deg-feels'})
    # from the resulting object, "o_temp", get the text parameter and assign it to "n_temp"
    n_temp = o_temp.text

    # Build SQL for purpose of:
    #    saving the temperature data to a new row
    s = ""
    s += "INSERT INTO tbl_temperatures"
    s += "("
    s += "n_temp"
    s += ") VALUES ("
    s += "(%n_temp)"
    s += ")"

    # Trap errors for opening the file
    try:
        db_cursor.execute(s, [n_temp, n_temp])
        db_conn.commit()
    except psycopg2.Error as e:
        t_msg = "Database error: " + e + "/n open() SQL: " + s
        return render_template("error_page.html", t_msg = t_msg)

    # Success!
    # Show a message to user.
    t_msg = "Successful scrape!"
    return render_template("progress.html", t_msg = t_msg)

    # Clean up the cursor and connection objects
    db_cursor.close()
    db_conn.close()

Python Shell 错误日志：

FLASK_APP = app.py
FLASK_ENV = development
FLASK_DEBUG = 0
In folder /home/lloyd/PycharmProjects/flaskProject
/home/lloyd/PycharmProjects/flaskProject/venv/bin/python -m flask run
 * Serving Flask app 'app.py' (lazy loading)
 * Environment: development
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
[2021-12-28 09:57:24,823] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 76, in start
    stdin=PIPE)
  File "/usr/lib64/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/chromium-browser/chromedriver': '/usr/lib/chromium-browser/chromedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/lloyd/PycharmProjects/flaskProject/app.py", line 23, in import_temp
    my_web_driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/home/lloyd/PycharmProjects/flaskProject/venv/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 83, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

127.0.0.1 - - [28/Dec/2021 09:57:24] "GET / HTTP/1.1" 500 -
127.0.0.1 - - [28/Dec/2021 09:57:25] "GET /favicon.ico HTTP/1.1" 404 -

Web 控制台错误：

Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

我之前确实在这里发布了一个类似的问题，即 404 错误： 为什么我收到错误 404 - 尝试使用 Python Flask 时

任何帮助将不胜感激。 干杯

Answer 1

当您请求“/”时，代码中有异常。 因为您的 selenium 设置存在问题。

您是否考虑过使用requests （ https://docs.python-requests.org/en/latest/ ）。 requests会给你的是网页的内容，而不需要定义一个 webdriver。

from requests import get

req = get() # put in your own url
content = req.text() # gets the content of the site, which will be HTML
bs = BeautifulSoup(content, 'html.parser')