Fetching data from a web site

Question

My company uses an online CRM software. I need to obtain all the customer mail addresses via CRM. Normally i login to CRM and meke several clicks and type "*" into the customer name field and finally click the "Search" button. That brings me all the customers and their mail addresses. How can i automate all thee steps and fetch the mail addresses? I thought Javascript and DOM could work. But with all the customers listed, on the site's source code i dont see the mail data. Any other ideas?

Thanks in advance.

Answer 1

If you want to automate this task (ie data-scraping) I would suggest to use something like PhantomJS in addition with CasperJS . Both are tools for testing WebUI - but who says, that's the only use-case. This would be the brute-force way to do it. With Casper, you could navigate to the Site and get the data from the HTML.

A better way, dependent on your CRM, is it to use some kind of Webservice or a REST-API to get the data, you are interested in. Check the documentation/ WIKI of the CRM. Perhaps there is an easier way (eg $.getJSON() on a specified URL should get you the mailadresses) than manually extracting the information.