Pass file contents to a function instead of downloading the file

Question

I am using Nightmare in order to automatically download a user-specific generated csv file from a web service as follows (rough workflow):

1. .goto() website
2. .insert() credentials and .click() login button
3. .wait() for desired DOM element to be loaded
4. scrape the landing page to extract some request-specific identifier which is needed to generate the download link for the file I am interested in
5. call .goto() on an export.php endpoint with some other arguments to download the csv file. Since Nightmare's Electron Window is displayed, I am asked for a path to store the file to.

Currently, the general overall workflow looks like this:

1. download (and save) the file using Nightmare
2. read file using readFileSync() into a string
3. analyze data using data-forge-js , pandasjs or d3-dsv

By passing the file's contents directly to the data analysis I want to get rid of saving the file in step 1 and reading that local file in step 2. I do not want to store the original csv file locally. Since I do not want to enable Nightmares Electron Window later in production I am looking for an approach to pass the file's contents to the data handling function directly and get rid of those unnecessary steps.

Answer 1

You can use the window.URL object methods to store your data. The following retrieves a binary blob but you could use content-types and response-types to return strings.

        xmlhttp.open('GET', PHOTO_URL, true);
        xmlhttp.responseType = 'blob';

        xmlhttp.onreadystatechange = function () {
            if (xmlhttp.readyState == 4) {
                if (xmlhttp.status == 200) {
                    var blob = xmlhttp.response;
                    photoImg.src = window.URL.createObjectURL(blob);
                    setTimeout(setOnCampusImage,0);     
                } else {
                    photoImg.src = 'unknown.png';
                }
            }
        }