Storm Crawler-搜寻需要认证的网站
[英]Storm Crawler- Crawling the websites which require authentication
问题描述
I would like to crawl websites which require authorization (I already have credentials) in intranet with Storm Crawler. 
我想使用Storm Crawler在Intranet中对需要授权（我已经有凭据）的网站进行爬网。 Is it possible to do that by simply modifying the crawler configuration or should I alter the classes in the source code, if so, which classes? 是否可以通过简单地修改搜寻器配置来做到这一点，或者我应该更改源代码中的类（如果这样），请选择哪些类？
1 个解决方案
解决方案1
0 已采纳 2017-02-23 07:10:30
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
正在搜寻要求认证的网站 - Crawling websites which ask for authentication
使用Storm Crawler进行爬网 - Crawling using Storm Crawler
$ crawler-> filter php脚本位置 - $crawler->filter php script position
可以在爬网过程中配置风暴搜寻器以将主机网址添加到网址路由的开头吗？ - Can I configure storm crawler to add the host url to the front of the url route during crawling?
在 Storm Crawler 中抓取特定基本 URL 的所有子 URL 的完成事件 - completion event of crawling all of the sub URLs for specific base URL in Storm Crawler
使用Storm爬虫对每个域的不同设置（例如速度）进行特定于域的爬网 - Domain-specific crawling with different settings for each domain (e.g. speed) using Storm crawler
调试Storm Crawler - Debugging Storm Crawler
使用“ npm爬行器”爬行 - Crawling with “npm crawler”
Web搜寻器搜寻网页时，如何通过输入用户名和密码来进行表单身份验证 - How to do form authentication by entering username and password while web crawler is crawling pages
多个销售网站中的履带 - crawler in multiple websites of sales
相关标签