网络抓取返回 URI 而不是图像的 URL。 （Javascript Cheerio）

Question

I'm using Cheerio and request to web scrape image url's. I keep getting the URI when i want to get the URL. What can i change to fix this?

const request = require('request-promise');
const cheerio = require ('cheerio');

(async () => {

    const webUrl = 'https://www.redbubble.com/lists/9747201/favorites';

    const response = await request(webUrl)

    const $ = cheerio.load(response);

    let sticker = $('img[class = "styles__image--2CwxX styles__rounded--1lyoH styles__fluid--3dxe-" ]').attr('src');

    console.log(sticker);
})();

It keeps returning

"data:image/gif;base64,R0lGODdhFQAXAPAAANba3wAAACwAAAAAFQAXAAACFISPqcvtD6OctNqLs968+w+GolUAADs="

When it should return

https://ih1.redbubble.net/image.479946364.2928/st,medium,507x507-pad,600x600,f8f8f8.u7.jpg

Answer 1

This is because the page source contains images that returns that value. It seems that they've encrypted the value with an output of base64 and then decrypted the src once the page has been loaded.

You have a better chance of scraping the contents using puppeteer which provides a high level API to control the browser (headless or not). You can simply wait for the browser to finish loading the page, and then scrape the contents you wish to obtain.

Another alternative is to read through the uglified JS source code that page you're trying to scrape, and look for the portion where it performs the decryption.

UPDATE:

You may not need cheerio or puppeteer at all. Upon checking the XHR requests in the page itself, I found that it uses a graphql API to get all those images and contents. Kindly investigate the request to get the proper results that you require.