如何使用 BeautifulSoup 和 Python 调用 JavaScript 函数
[英]How to call JavaScript function using BeautifulSoup and Python
问题描述
I am performing web scraping to grab data from a website as part of my project.
作为项目的一部分，我正在执行网络抓取以从网站获取数据。 I can make the request and grab the data which is present in the dom.我可以发出请求并获取dom中存在的数据。 However, some data is getting rendered on javascript onClick function.但是，一些数据在 javascript onClick 函数上呈现。
One way could be, using the selenium to click on the link (which calls the javascript function) and grab the rendered data, but this process is time-consuming, and I don't want to open the browser.一种方法可能是，使用 selenium 单击链接（调用 javascript 函数）并获取呈现的数据，但这个过程很耗时，我不想打开浏览器。
Is there any way other than selenium to achieve this?除了硒之外还有其他方法可以实现这一目标吗？
Website: http://catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849网址： http : //catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849
In the courses section of this webpage, all the courses are hyperlinks, and as soon as someone clicks on the courses, a javascript method gets called.在本网页的课程部分，所有课程都是超链接，只要有人点击课程，就会调用一个 javascript 方法。 I need the data which gets rendered after the javascript function call.我需要在 javascript 函数调用后呈现的数据。
3 个解决方案
解决方案1
5 已采纳 2018-02-04 00:04:53
I am performing web scraping to grab data from a website as part of my project.作为项目的一部分，我正在执行网络抓取以从网站获取数据。 I can make the request and grab the data which is present in the dom.我可以发出请求并获取dom中存在的数据。 However, some data is getting rendered on javascript onClick function.但是，一些数据在 javascript onClick 函数上呈现。
One way could be, using the selenium to click on the link (which calls the javascript function) and grab the rendered data, but this process is time-consuming, and I don't want to open the browser.一种方法可能是，使用 selenium 单击链接（调用 javascript 函数）并获取呈现的数据，但这个过程很耗时，我不想打开浏览器。
Is there any way other than selenium to achieve this?除了硒之外还有其他方法可以实现这一目标吗？
Website: http://catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849网址： http : //catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849
In the courses section of this webpage, all the courses are hyperlinks, and as soon as someone clicks on the courses, a javascript method gets called.在本网页的课程部分，所有课程都是超链接，只要有人点击课程，就会调用一个 javascript 方法。 I need the data which gets rendered after the javascript function call.我需要在 javascript 函数调用后呈现的数据。
解决方案2
4 2020-02-01 16:36:10
I am performing web scraping to grab data from a website as part of my project.作为项目的一部分，我正在执行网络抓取以从网站获取数据。 I can make the request and grab the data which is present in the dom.我可以发出请求并获取dom中存在的数据。 However, some data is getting rendered on javascript onClick function.但是，一些数据在 javascript onClick 函数上呈现。
One way could be, using the selenium to click on the link (which calls the javascript function) and grab the rendered data, but this process is time-consuming, and I don't want to open the browser.一种方法可能是，使用 selenium 单击链接（调用 javascript 函数）并获取呈现的数据，但这个过程很耗时，我不想打开浏览器。
Is there any way other than selenium to achieve this?除了硒之外还有其他方法可以实现这一目标吗？
Website: http://catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849网址： http : //catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849
In the courses section of this webpage, all the courses are hyperlinks, and as soon as someone clicks on the courses, a javascript method gets called.在本网页的课程部分，所有课程都是超链接，只要有人点击课程，就会调用一个 javascript 方法。 I need the data which gets rendered after the javascript function call.我需要在 javascript 函数调用后呈现的数据。
解决方案3
1 2018-02-04 00:15:36
I am performing web scraping to grab data from a website as part of my project.作为项目的一部分，我正在执行网络抓取以从网站获取数据。 I can make the request and grab the data which is present in the dom.我可以发出请求并获取dom中存在的数据。 However, some data is getting rendered on javascript onClick function.但是，一些数据在 javascript onClick 函数上呈现。
One way could be, using the selenium to click on the link (which calls the javascript function) and grab the rendered data, but this process is time-consuming, and I don't want to open the browser.一种方法可能是，使用 selenium 单击链接（调用 javascript 函数）并获取呈现的数据，但这个过程很耗时，我不想打开浏览器。
Is there any way other than selenium to achieve this?除了硒之外还有其他方法可以实现这一目标吗？
Website: http://catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849网址： http : //catalog.fullerton.edu/preview_entity.php?catoid=16&ent_oid=1849
In the courses section of this webpage, all the courses are hyperlinks, and as soon as someone clicks on the courses, a javascript method gets called.在本网页的课程部分，所有课程都是超链接，只要有人点击课程，就会调用一个 javascript 方法。 I need the data which gets rendered after the javascript function call.我需要在 javascript 函数调用后呈现的数据。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.