How to integrate python with a Xamarin Android App for web-scraping

Question

Me and a friend had an idea to build some kind of web-scraping software and eventually we settled for a mobile app (Android). The main idea is: We provide the user with a list of music festivals, the user selects one and when he does, a list of the current confirmed artists for that festival is displayed (we want it to display an up-to-date list). The list is retrieved via python scripts (note that the goal is not deployment or profit (at least right now), thus this very summarized explanation).

My question is, how would I go about integrating the python scripts with the mobile application? At this moment I know you can call them with C# but I'm not sure if that would work on a mobile environment. Plus I saw someone saying one should host the scripts on a web server and then execute them via API calls but the person didn't expand and I'm not sure how to do that.

Any insight on the matter or useful references would be helpful. Thanks!

Answer 1

You can do it in C# using HttpClient and HtmlAgilityPack; You would need:

using System.Net.Http;
using HtmlAgilityPack;

Make methods like this:

public static HtmlNode GetNodeById(HttpClient client, string url, string divId)
{
    string pageHtml = "";
    using (HttpResponseMessage response = client.GetAsync(url).Result)
    {
        using (HttpContent content = response.Content)
        {
            pageHtml =  content.ReadAsStringAsync().Result;
        }
    }

    var doc = new HtmlDocument();
    doc.LoadHtml(pageHtml);
    HtmlNode node = doc.GetElementbyId(divId);
    return node;
}

And call like this (where "musiciansDiv" is the div that contains the info your app needs):

static HttpClient client = new HttpClient();
var musicians = GetNodeById(client, "http://gigs.example.com", "musiciansDiv");