[英]Java: How to Scrape Images from Amazon with Selenium?
我正在嘗試使用Selenium WebDriver從Amazon上的此URL刮取頁面左側的6張圖像:
http://www.amazon.com/EasyAcc%C2%AE-10000mAh-Brilliant-Smartphone-Bluetooth/dp/B00H9BEC8E
但是,無論我怎么嘗試都會導致錯誤。 到目前為止,我已經嘗試過:
我嘗試直接使用XPATH刮取圖像,然后使用“ getAttributes”方法提取src。 例如,對於頁面上的第一個圖像,XPATH為:
.//*[@id='a-autoid-2']/span/input
所以我嘗試了以下方法:
String path1 = ".//*[@id='a-autoid-2']/span/input";
String url = "http://www.amazon.com/EasyAcc%C2%AE-10000mAh-Brilliant-Smartphone-Bluetooth/dp/B00H9BEC8E";
WebDriver driver = new FirefoxDriver();
driver.get(url);
WebElement s;
s = driver.findElement(By.xpath(path1));
String src;
src = s.getAttribute("src");
System.out.println(src);
但是我找不到來源。
注意:僅當從某些類型的產品中刮取圖像時,才會出現此問題。 例如,我可以使用Selenium輕松從該產品刮取圖像:
http://www.amazon.com/zh-CN/Ultimate-Unification-Diet-Health-Disease/dp/0615797806/
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class mytest {
public static void main(String[] args) {
// TODO Auto-generated method stub
String path = ".//*[@id='imgThumbs']/div[2]/img";
String url = "http://www.amazon.com/Ultimate-Unification-Diet-Health-Disease/dp/0615797806/";
WebDriver driver = new FirefoxDriver();
driver.get(url);
WebElement s;
s = driver.findElement(By.xpath(path));
String src;
src = s.getAttribute("src");
System.out.println(src);
driver.close();
}
}
該代碼可以完美地工作。 僅當刮擦某些產品時,似乎無法解決它。
我嘗試單擊導致iframe打開的圖像,但是即使使用以下方法切換到iframe,也無法從該iframe抓取圖像:
driver.switchTo()。frame(IFRAMEID);
我知道我可以使用“屏幕截圖”方法,但是我想知道是否有一種直接刮取圖像的方法嗎?
謝謝
試試這個代碼
String path = "//div[@id='imageBlock_feature_div']//span/img";
String url = "http://rads.stackoverflow.com/amzn/click/0615797806";
WebDriver driver = new FirefoxDriver();
driver.get(url);
List<WebElement> srcs;
srcs = driver.findElements(By.xpath(path));
for(WebElement src : srcs) {
System.out.println(src.getAttribute("src"));
}
driver.close();
結果
2015-01-23 12:36:14 [main]-[INFO] Opened url: http://rads.stackoverflow.com/amzn/click/B00H9BEC8E
http://ecx.images-amazon.com/images/I/41cOP3mFX3L._SX38_SY50_CR,0,0,38,50_.jpg
http://ecx.images-amazon.com/images/I/51YkMhRXqcL._SX38_SY50_CR,0,0,38,50_.jpg
http://ecx.images-amazon.com/images/I/51nSbXF%2BCTL._SX38_SY50_CR,0,0,38,50_.jpg
http://ecx.images-amazon.com/images/I/31s%2B31F%2BQmL._SX38_SY50_CR,0,0,38,50_.jpg
http://ecx.images-amazon.com/images/I/41FmTOJEOOL._SX38_SY50_CR,0,0,38,50_.jpg
http://ecx.images-amazon.com/images/I/41U6qpLJ07L._SX38_SY50_CR,0,0,38,50_.jpg
但是,要獲取Amazon Images,建議您嘗試使用Amazon API https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html
好多了
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.