[英]Can i use selenium with Scrapy without actual browser opening with python
I want to do some web crawling with scrapy and python. 我想用scrapy和python进行一些网络爬行。 I have found few code examples from internet where they use selenium with scrapy.
我发现很少有来自互联网的代码示例,他们使用selenium和scrapy。
I don't know much about selenium
but only knows that it automates some web tasks. 我对
selenium
但只知道它会自动化一些网络任务。 and browser actually opens and do stuff. 和浏览器实际上打开并执行操作。 but i don't want the actual browser to open but i want everything to happen from command line.
但我不希望实际的浏览器打开,但我想从命令行发生一切。
Can i do that in selenium and scrapy 我可以在硒和scrapy中做到这一点
Updated: PhantomJS is abandoned, and you can use headless browsers directly now, such like Firefox and Chrome! 更新:PhantomJS已被放弃,您现在可以直接使用无头浏览器,例如Firefox和Chrome!
Use PhantomJS instead. 请改用PhantomJS。
You can do browser = webdriver.PhantomJS()
in selenium v2.32.0. 您可以在selenium v2.32.0中执行
browser = webdriver.PhantomJS()
。
You can use selenium with PyVirtualDisplay , at least on linux. 您可以将selenium与PyVirtualDisplay一起使用 ,至少在linux上使用。
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(1024, 768))
display.start()
browser = webdriver.Chrome()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.