如何通过CasperJS从具有特定类的元素中获取hrefs？

Question

I've got a web page with information in this format:

<p>
    <a class="class1" href="href1">text1</a>
    text2
</p>

<p>
    <a class="class1" href="href2">text1a</a>
    text2a
</p>

Using CasperJS, I need to get an array of all the information contained in just in elements with class1 in this format:

href1
text1
text2

href2
text1a
text2a

I've tried using this code:

var casper = require('casper').create();
casper.start('url', function() {
    require('utils').dump(this.getElementsAttribute('div[class="class1"]', 
          'class'));
});
casper.run();

However, I just got a '[ ]' as an answer.

Can anybody help me find the error in my code?

Answer 1

div[class="class1"] as a selector cannot work, because you don't have any <div> elements in your markup that have the class1 class. You can try the following, but it won't get you far:

this.getElementsAttribute('a.class1', 'href');

Building an array of objects in the page context

It is hard and may be error prone to do this only with CasperJS functions. It is much easier to do this by iterating over all the links and fetching the parts that you need.

casper.then(function(){
    var info = this.evaluate(function(){
        var links = document.querySelectorAll(".class1");
        // iterate over links and collect stuff
        return Array.prototype.map.call(links, function(link){
            return {
                href: link.href,
                hrefText: link.textContent.trim(),
                afterText: link.parentNode.childNodes[2].textContent.trim()
            };
        });
    });
    require('utils').dump(info);
});

How this works:

You can get all the links by querying all for all the elements with class1 . Since the result of querySelectorAll() is not an array, but an array-like NodeList, you can't directly use .map() on it.

Each link has a href property and a textContent property. The text after the link is a little tricky. You first need to get the parent of the link ( <p> ) and then try to get the TextNode after the link by accessing the childNodes property.

childNodes[2] must probably be used instead of childNodes[1] , because the first ( childNodes[0] ) is probably a TextNode containing whitespace, so everything after it shifts.

Building a single string in the page context

You can also iterate over it to get it in a textual representation:

casper.then(function(){
    var info = this.evaluate(function(){
        var links = document.querySelectorAll(".class1");
        // iterate over links and collect stuff
        return Array.prototype.map.call(links, function(link){
            return [
                link.href,
                link.textContent.trim(),
                link.parentNode.childNodes[2].textContent.trim()
            ].join('\n');
        }).join('\n\n');
    });
    this.echo(info);
});

How it works:

JavaScript arrays have a join() function . It can joins every element using the specified separator.

---

Keep in mind that the page context ( evaluate() ) is sandboxed. The documentation says:

Note: The arguments and the return value to the evaluate function must be a simple primitive object. The rule of thumb: if it can be serialized via JSON, then it is fine.

Closures, functions, DOM nodes, etc. will not work!