如何提取網頁文本標簽后面的鏈接（使用curl，wget或userscript）

Question

我正在嘗試提取標簽下的href鏈接。 請參閱附件。 我想將所有鏈接保存在“ PDF”標簽下。 http://tinypic.com/r/2n9erdj/8對不起，我還不能更新圖片。

具體來說，href詳細信息顯示為arnumber = 60940cc，如紅色圓圈所示。

有人可以建議如何實現這一點。 我打算使用用戶腳本或bash命令。

與單個pdf相關的html元素詳細信息如下所示。

<a aria-label="Download or View the PDF:  IEEE Transactions on Power Electronics publication information" href="/stampPDF/getPDF.jsp?tp=&amp;arnumber=6094072"><img class="button" src="http://staticieeexplore.ieee.org/assets/img/iconPdf.png" alt="PDF file icon" title="Download or View the PDF">PDF</a>

我正在測試的網頁是

http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6088512

目的是過濾名為“ pdf”的內容及其URL。

Answer 1

嘗試以下操作：如果您不希望在戳記字等之前添加“ http://ieeexplorer.ieee.org/ ”，請調整sed部分

wget http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6088512 -O file.html
grep -o "href.*stamp.*\"><" file.html |sed 's#"#"http://ieeexplorer.ieee.org#;s#><##'

href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094070"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094072"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094110"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6088513"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5680978"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5985544"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5723758"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5716681"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5936741"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5934597"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5734858"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5756244"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5759746"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5958614"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5999721"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6021380"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5961632"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5951783"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5983448"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5934423"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5957306"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5898425"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5959991"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5776690"
href="http://ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5953525"

要么

$ grep -o "href.*stamp.*\"><" file.html |sed 's#href="#ieeexplorer.ieee.org#;s#"><##'

ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094070
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094072
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6094110
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6088513
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5680978
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5985544
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5723758
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5716681
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5936741
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5934597
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5734858
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5756244
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5759746
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5958614
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5999721
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=6021380
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5961632
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5951783
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5983448
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5934423
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5957306
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5898425
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5959991
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5776690
ieeexplorer.ieee.org/stamp/stamp.jsp?tp=&arnumber=5953525