![](/img/trans.png)
[英]Can MSXML2.XMLHTTP retrieve ALL of the HTML data for a given webpage?
[英]Using MSXML2.XMLHTTP in access VBA not extracting all page data
目前,我們使用下面提到的代碼進行數據提取,但是代碼沒有從網頁中提取完整的數據,代碼忽略了當我在Internet Explorer上啟用java腳本和DOM存儲時可見的數據。
到目前為止我使用下面提到的代碼,尾隨代碼是從網頁中提取所有接受圖像的東西。
我的代碼受到了打擊。
Set http = CreateObject("MSXML2.XMLHTTP")
http.Send
html.body.innerHTML = http.ResponseText
On Error GoTo 0
html1 = html.body.innerHTML
brand5 = html.documentElement.innerHTML
If html1 Like "*media__thumb*" Then
other_img = html.getElementsByClassName("media__thumb")(0).innerText
'other_img = other_img.innerHTML
End If
在網頁上有多個圖像html代碼如下(請注意我的上述代碼不是從下面提到的html代碼中提取數據。
<a class="media__thumbnail" data-media_type="IMAGE" data-media_id="orbit-bagged-53017-64" data-target="IMAGE" data-has-index="true">
<img src="https://images.yourweb/_145.jpg">
</a>
<a class="media__thumbnail media__thumbnail--selected" data-media_type="IMAGE" data-media_id="orbit-bagged-53017-e1" data-target="IMAGE" data-has-index="true">
<img src="https://images.yourweb1_145.jpg">
</a>
</span></a>
http.response如下
<div id="thumbnails" class="media__thumbnails" data-component="thumbnails"></div>
<script type="text/template" id="media__thumbnails">
{{#thumbnails}}
<a class="media__thumbnail" data-media_type="{{type}}" data-media_id="{{id}}" data-target="{{type}}" data-has-index="true">
<img src="{{{thumb}}}"/>
{{# hasIcon}}
{{# threeSixtyIcon}} <div class="whitespace"><span class="threesixtyIcon"></span></div>{{/ threeSixtyIcon}}
{{^ threeSixtyIcon}} <span class="videoIcon"></span>{{/ threeSixtyIcon}}
{{/ hasIcon}}
</a>
{{/thumbnails}}
{{#additionalThumbnailsThumbnail}}
<a class="media__thumbnail media__thumbnail-additional-count" data-media_type="{{type}}" data-media_id="{{id}}" data-target="{{type}}" data-has-index="true">
<img src="{{{thumb}}}"/>
{{# hasIcon}}
{{# threeSixtyIcon}} <div class="whitespace"><span class="threesixtyIcon"></span></div>{{/ threeSixtyIcon}}
{{^ threeSixtyIcon}} <span class="videoIcon"></span>{{/ threeSixtyIcon}}
{{/ hasIcon}}
{{#additionalImagesCount}}
<div class="media__thumbnail-overlay"></div>
<span class="media__thumbnail-count">+{{additionalImagesCount}}</span>
{{/additionalImagesCount}}
</a>
該內容需要在頁面上運行javascript,因此您需要:
您可以從以下內容中看到內容是動態加載的:
<script type="text/template" id="media__thumbnails"> {{#thumbnails}} <a class="media__thumbnail" data-media_type="{{type}}" data-media_id="{{id}}" data-target="{{type}}" data-has-index="true"> <img src="{{{thumb}}}"/> {{# hasIcon}} {{# threeSixtyIcon}} <div class="whitespace"><span class="threesixtyIcon"></span></div>{{/ threeSixtyIcon}} {{^ threeSixtyIcon}} <span class="videoIcon"></span>{{/ threeSixtyIcon}} {{/ hasIcon}} </a> {{/thumbnails}} {{#additionalThumbnailsThumbnail}} ...... {{/additionalThumbnails}} </script> <script type="text/template"
1)網絡選項卡 - 不同的網址
使用網絡選項卡中找到的不同URL返回包含鏈接的json。 響應是json所以需要一個jsonparser 。
'VBE>Tools> References> Add reference to Microsoft Scripting Runtime
'Download and add in jsonconverter.bas from https://github.com/VBA-tools/VBA-JSON/blob/master/JsonConverter.bas
將converter.bas代碼放在module2中並注釋掉該行: Attribute VB_Name = "JsonConverter"
在module1中放置GetInfo
子。
Option Compare Database
Option Explicit
Public Sub GetInfo()
Dim json As Object, url1 As String, url2 As String, url3 As String
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://www.homedepot.com/p/svcs/frontEndModel/100001020?_=1556447908065", False
.send
Set json = Module2.ParseJson(.responseText)
End With
'Parse json object (see paths shown below for example)
url1 = json("primaryItemData")("media")("mediaList")(2)("location")
url2 = json("primaryItemData")("media")("mediaList")(3)("location")
url3 = json("primaryItemData")("media")("mediaList")(4)("location") 'example
Stop '<==delete me later
End Sub
前3個縮略圖的路徑:
json►primaryItemData►media►mediaList►2►location
json►primaryItemData►media►mediaList►3►location
json►primaryItemData►media►mediaList►4►location
在這里探索json。
2)自動瀏覽器(IE版):
'VBE > Tools > References:
' Microsoft Internet Controls
Public Sub GetImageLinks()
Dim ie As New InternetExplorer, images As Object, i As Long
With ie
.Visible = True
.Navigate2 "https://www.homedepot.com/p/Orbit-Sandstone-Rock-Valve-Box-Cover-53017/100001020"
While .Busy Or .readyState < 4: DoEvents: Wend
Set images = .document.querySelectorAll(".media__thumbnail img")
For i = 0 To images.Length - 1
Debug.Print images.item(i).src
Next
Stop
.Quit
End With
End Sub
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.