How to Download .XML File From Website Through JavaScript Button Using Python

Question

I am trying to download a customized XML file from this website: http://data.un.org/Data.aspx?d=CLINO&f=ElementCode:11;StatisticCode:01&c=1,2,5,17,18,44&s=CountryName:asc,WmoStationNumber:asc,StatisticCode:asc&v=1

The method I am most familiar with is using pd.read_csv, but in this case right-clicking the download link and copying the link address generates:

javascript:Download('xml','CLINO','ElementCode:11;StatisticCode:01','s=CountryName:asc,WmoStationNumber:asc,StatisticCode:asc','c=1,2,5,17,18,44','');

I tried the solution posted here but unfortunately the process deviated at Step 4.

Using python, how do I access the.xml file to download and save?

Answer 1

import requests
import pandas as pd

params = {
    'Service': 'page',
    'DataFilter': 'ElementCode:11;StatisticCode:01',
    'DataMartId': 'CLINO',
    'UserQuery': '',
    'c': '1,2,5,17,18,44'
}


def main(url, params):
    with requests.Session() as req:
        allin = []
        for item in range(1, 23):
            print(f"Extracting Page# {item}")
            params['Page'] = item
            r = req.get(url, params=params)
            df = pd.read_html(r.content)[0]
            allin.append(df)
        new = pd.concat(allin)
        print(new)
        new.to_csv("Data.csv", index=False)


main("http://data.un.org/Handlers/DataHandler.ashx", params)