使用Android从网络上获取数据？

Question

When using Eclipse for Java, I'm able to access data from websites and fill out online forms using Selenium. All I have to do is do WebDriver driver = new HtmlUnitDriver(); and driver.get("wwww.google.com"); and driver.findElement() . In order to accomplish this, I would go into the Java Build Path, access Libraries, and then add the external JAR file: selenium-server-standalone-2.39.0.jar .

I'd like to do the same for Android but am having difficulty. I understand there was something called Selenium for Android, but it's no longer being supported. Now there's Selendroid. But while the code is vaguely familiar to that of Eclipse for Java (ie, SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.12.0"); , WebDriver driver = new SelendroidDriver(capa); , WebElement inputField = driver.findElement(By.id("my_text_field")); ), I don't think this is actually the same as what I am looking for. I even tried to add selendroid-standalone-0.12.0-with-dependencies.jar to the Android library and all I got back was this error in the console:

Dx warning: Ignoring InnerClasses attribute for an anonymous inner class
(org.apache.xalan.lib.sql.SecuritySupport12$8) that doesn't come with an
associated EnclosingMethod attribute. This class was probably produced by a
compiler that did not target the modern .class file format. The recommended
solution is to recompile the class from source, using an up-to-date compiler
and without specifying any "-target" type options. The consequence of ignoring
this warning is that reflective operations on this class will incorrectly
indicate that it is *not* an inner class.

So my question is: Where can I go to learn about using Android to go to a web page and retrieve some data (but not actually open a web page on the screen, this is strictly background stuff)? Or, what are the steps to getting data from a website via Android using identifiers such as id , name , or Xpath , etc.?

Answer 1

Use JSOUP for the same. I think thats what you loking for.

jsoup is a Java library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods.

Download jar and include in project.

Simple example :

Document doc = Jsoup.connect("http://example.com/").get();
String title = doc.title();

Read apidocs for more info.

Also make sure to put network calls in an AsyncTask and not on main UI thread.

Answer 2

I eventually found something that is exactly what I wanted: HtmlCleaner. There's a good guide here .

Download the JAR file here and include it in the project's library.

Then use the following code to get your element from the XPath:

public class Main extends Activity {

    // HTML page
    static final String URL = "https://www.yourpage.com/";
    // XPath query
    static final String XPATH = "//some/path/here";

    @Override
    public void onCreate(Bundle savedInstanceState) {
        // init view layout
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        // decide output
        String value = getData();
    }

    public String getData() {
        String data = "";

        // config cleaner properties
        HtmlCleaner htmlCleaner = new HtmlCleaner();
        CleanerProperties props = htmlCleaner.getProperties();
        props.setAllowHtmlInsideAttributes(false);
        props.setAllowMultiWordAttributes(true);
        props.setRecognizeUnicodeChars(true);
        props.setOmitComments(true);

        // create URL object
        URL url = new URL(URL);
        // get HTML page root node
        TagNode root = htmlCleaner.clean(url);

        // query XPath
        Object[] statsNode = root.evaluateXPath(XPATH);
        // process data if found any node
        if(statsNode.length > 0) {
            // I already know there's only one node, so pick index at 0.
            TagNode resultNode = (TagNode)statsNode[0];
            // get text data from HTML node
            stats = resultNode.getText().toString();
        }

        // return value
        return data;
    }
}