简体繁体 English

从网站抓取和获取不同项目的最快方法是什么

[英]what's the fastest way to web-scraping and get different items from a website

原文 2019-11-02 23:12:50 5 1 javascript/ python/ selenium/ web-scraping/ request

I've been working on a web-scraping project written in Python using Selenium and requests.我一直在使用 Selenium 和请求编写一个用 Python 编写的网络抓取项目。 Each time when I need data I send a request to get it (or using Selenium) and each request takes time.每次我需要数据时，我都会发送一个请求来获取它（或使用 Selenium），每个请求都需要时间。

My question is, is there any option to get a lot of different data ( such as product's name+price+shipping) in 1 request in Python or even in javascript?我的问题是，是否有任何选项可以在 Python 甚至 javascript 的 1 个请求中获取大量不同的数据（例如产品名称+价格+运费）？

Given a url for example: https://www.amazon.com/Dell-Inspiron-7573-i7-8550U-Windows/dp/B07NRC8ZXC/ref=sr_1_1_sspa?keywords=laptop+i7&qid=1572590892&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEyNjFPSFdTOEpVODdQJmVuY3J5cHRlZElkPUEwNTU5Nzk1MlBGWFkxU0JKOVlLNiZlbmNyeXB0ZWRBZElkPUEwNjU0MzYwM0NRT01ER1oxSDdMOCZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU= Given a url for example: https://www.amazon.com/Dell-Inspiron-7573-i7-8550U-Windows/dp/B07NRC8ZXC/ref=sr_1_1_sspa?keywords=laptop+i7&qid=1572590892&sr=8-1-spons&psc= 1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEyNjFPSFdTOEpVODdQJmVuY3J5cHRlZElkPUEwNTU5Nzk1MlBGWFkxU0JKOVlLNiZlbmNyeXB0ZWRBZElkPUEwNjU0MzYwM0NRT01ER1oxSDdMOCZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=

NOTE: you marked this question as duplicated with this question - How to click on Load More button within Google Trends and print all the titles through Selenium and Python PLEASE, those are different questions, I'm asking about getting this product's price+name+shipping who are under different "scopes", she asked about getting the same "objects" (such as names),.. so if you know any other way to get those things instead of sending 3 different requests to get them.注意：您将此问题标记为与此问题重复 - 如何在 Google 趋势中单击加载更多按钮并通过 Selenium 和 Python 打印所有标题，请，这些是不同的问题，我问的是有关获取此产品的价格+名称+运送在不同“范围”下的人，她询问是否获得相同的“对象”（例如名称），..所以如果您知道任何其他方式来获取这些东西，而不是发送 3 个不同的请求来获取它们。 I would love to hear about them,?.我很想听听他们，？ I've heard that there is a way to combine a few requests as one big request in javascript, is it true?我听说有一种方法可以在 javascript 中将几个请求合并为一个大请求，是真的吗？