简体繁体 English

JAVA 中抓取

[英]Meta Search Engine/ Web Scraping in Android Studio/JAVA

原文 2020-06-18 14:44:45 7 1 c#/ android/ web-scraping/ search-engine/ meta-search

I want to create an application that basically search for something with some filters from various websites (I don't require to login to those third party websites so the data available is open to public) and show it on my application.我想创建一个应用程序，该应用程序基本上使用来自各个网站的一些过滤器搜索某些内容（我不需要登录到那些第三方网站，因此可用的数据对公众开放）并将其显示在我的应用程序上。 I have a few questions:我有几个问题：

1. Is It Legal? 1. 合法吗？
2. Is this web scraping or Meta Search Engine? 2. 这是 web 抓取还是元搜索引擎？
3. Can I get more information (any web links/articles) to know more about it? 3. 我可以获得更多信息（任何 web 链接/文章）以了解更多信息吗？ How to achieve it technically?如何在技术上实现？ One way I know that we can use the XPath technique to scrape but I am wondering if there are more ways.我知道我们可以使用 XPath 技术进行刮擦的一种方法，但我想知道是否还有更多方法。 I am NOT asking for the entire code.我不是要完整的代码。 Just how to start / Any guidance?只是如何开始/任何指导？

Thank You in Advance !先感谢您！

1 个解决方案

Firstly you need to understand how search engines work.首先，您需要了解搜索引擎的工作原理。 -Our so called search engines like google have special programs designed to mine out information from the web they are called "Spiders" what a spider does is basically scroll over all web pages within the search query and find matching information however that's a really complex thing to work on, it takes really good code and algorithm expertise to develop a spider for yourself. -我们所谓的搜索引擎（如谷歌）有特殊的程序，旨在从 web 中挖掘信息，它们被称为“蜘蛛”，蜘蛛所做的基本上是滚动搜索查询中的所有 web 页面并找到匹配信息，但这是一件非常复杂的事情要继续工作，需要非常好的代码和算法专业知识才能为自己开发蜘蛛。 However if you can master that you'll be earning a smooth sum of money, but it's really rare unless you're a blatant genius!不过，如果你能掌握这一点，你就会赚到一笔不菲的钱，但除非你是一个明目张胆的天才，否则这真的很少见！