[英]shell script to extract multiline data out of a text file with regex
我試圖使用帶有shell腳本的正則表達式從文本文件中提取一些特定數據
這是使用多行grep ..我正在使用的工具是pcregrep,這樣我就可以兼容perl的正則表達式
[58]Walid Chamoun Architects WLL
* [59]Map
* [60]Website
* [61]Email
* [62]Profile
* [63]Display Ad
Walid Chamoun Architects WLL
PO Box:
55803, Doha, Qatar
Location:
D-Ring Road, New Salata Shamail 40, Villa 340, Doha, Qatar
Tel:
(00974) 44568833
Fax:
(00974) 44568811
Mob:
(00974) 44568822
* Accurate Budget Costing
* Eco-Friendly Structural Design
* Exclusive & Unique Design
* Quality Architecture & Design
Company Profile
Walid Chamoun Architects (WCA) was founded in Beirut, Lebanon, in 1992,
committed to the concept of fully integrated design-build delivery of
projects. In late '90s, company established in-house architectural and
engineering services. As a full service provider, WCA expanded from
multi-family projects to industrial and office construction, which
added development services, including site acquisition and financing.
In 2001, WCA had opportunity and facilities to experience European
market and establish office in Puerto Banus, Marbella, Spain. By 2005,
WCA refined its structure to focus on specific market segments and new
office was opened in Doha, state of Qatar. From a solid foundation and
reputation built over eighteen years, WCA continually to provide
leadership in design-build through promotion of benefits and education
to its practitioners.
Project Planning: Project planning and investigation occurs before
design begins has greatest impact on cost, schedule and ultimately the
success of project. Creativity in Design: You can rely on our in-house
designers for design excellence in all aspects of the project. Our
designs have received recommendations and appreciations on national and
international levels. Creativity in Execution: Experienced in close
collaboration with the designers as part of the integrated team, our
construction managers, superintendents and field staff create value
throughout the project. Post Completion Services: Your needs can be
served through our skills and experience long after the last
construction crew has left the site. Performance: Corporate and
institutional clients, developers and public agencies repeatedly select
WCA on the basis of its consistent record of performance excellence.
Serving clients throughout the Middle East and GCC, WCA provides
complete planning for architectural, interior design and construction
on a single-responsibility basis. Our expertise spans industrial,
commercial, institutional, public and residential projects. Benefits of
Design-Build: Design-build is a system of contracting under which one
entity performs both design and construction. Benefits of design-build
project delivery include: Single point responsibility Early knowledge
of cost Time and Cost savings
Classification:
Architects - [64]Architects
[65]Al Ali Consulting & Engineering
* [66]Map
* Website
* Email
* Profile
* Display Ad
Is this your company?
[67]Upgrade this free listing here
PO Box:
467, Doha, Qatar
Tel:
(00974) 44360011
Company Profile
Classification:
Architects - [68]Architects
[69]Al Gazeerah Consulting Engineering
* [70]Map
* Website
* Email
* Profile
* Display Ad
Is this your company?
[71]Upgrade this free listing here
PO Box:
22414, Doha, Qatar
Tel:
(00974) 44352126
Company Profile
Classification:
Architects - [72]Architects
[73]Al Murgab Consulting Engineering
* [74]Map
* Website
* Email
* Profile
* Display Ad
Is this your company?
[75]Upgrade this free listing here
PO Box:
2856, Doha, Qatar
Tel:
(00974) 44448623
Company Profile
Classification:
Architects - [76]Architects
References
Visible links
1. http://www.qatcom.com/useraccounts/login
2. http://www.qatcom.com/useraccounts/register
3. http://www.qatcom.com/
4. http://www.qatcom.com/
5. http://www.qatcom.com/qataryellowpages/map-of-doha
6. http://www.qatcom.com/qataryellowpages/about-qatcom
7. http://www.qatcom.com/qataryellowpages/advertise-with-qatcom
8. http://www.qatcom.com/qataryellowpages/advertiser_testimonials
9. http://www.qatcom.com/useraccounts/login
10. http://www.qatcom.com/useraccounts/register
11. http://www.qatcom.com/contact-qatcom
12. http://www.qatcom.com/qataryellowpages/companies
13. http://www.qatcom.com/classifications/index/A
14. http://www.qatcom.com/classifications/index/B
15. http://www.qatcom.com/classifications/index/C
16. http://www.qatcom.com/classifications/index/D
17. http://www.qatcom.com/classifications/index/E
18. http://www.qatcom.com/classifications/index/F
19. http://www.qatcom.com/classifications/index/G
20. http://www.qatcom.com/classifications/index/H
21. http://www.qatcom.com/classifications/index/I
22. http://www.qatcom.com/classifications/index/J
23. http://www.qatcom.com/classifications/index/K
24. http://www.qatcom.com/classifications/index/L
25. http://www.qatcom.com/classifications/index/M
26. http://www.qatcom.com/classifications/index/N
27. http://www.qatcom.com/classifications/index/O
28. http://www.qatcom.com/classifications/index/P
對於像這樣的樣本數據,我試圖抓住公司的細節
company name
po box
Tel
fax
mobile
company profile
進入.csv文件我是正則表達式和Linux的新手..所有我能設法獲得的是這樣的
\[\d*\][^\.]*[\(\d*\)\s\d*)]
請有人幫我解決這個問題..
改進:
我想出了這樣的事情
$ awk '/^\[/ && ! /Upgrade this free listing/ {print $0} /:$/ && ! /Classification/ {printf $0 ; getline x ; print x}' file
但那仍然不是我想要的......
你可以在awk中做到這一點,但你最好還是解析HTML。 使用Beautiful Soup模塊的Python就是一個很好的工具。 但這不是很令人興奮,所以這里是如何做到這一點尷尬(哈!)的方式:
#!/usr/bin/awk -f
function trim(s) {
gsub(/(^ +)|( +$)/, "", s)
return s
}
BEGIN {
count = 0
fields[0] = "company"
fields[1] = "pobox"
fields[2] = "tel"
fields[3] = "fax"
fields[4] = "mob"
fields[5] = "profile"
}
# company name
/^ +\[[0-9]+\].*$/ {
sub(/^ +\[[0-9]+\]/, "") # get rid of the Lynx reference
# this is a bit naughty: our regex also matches this other link, but there's only one of them, so we just filter it
if ($0 != "Upgrade this free listing here") data[count,"company"]=$0
}
# two line fields, easy!
/ +PO Box:$/ { getline; data[count,"pobox"]=$0 }
/ +Tel:$/ { getline; data[count,"tel"]=$0 }
/ +Fax:$/ { getline; data[count,"fax"]=$0 }
/ +Mob:$/ { getline; data[count,"mob"]=$0 }
# multi-line field, tricky because it can be empty
/^Company Profile$/ {
getline # skip empty line
# process lines until encountering Classification field
s = ""
do {
s = s $0
getline
} while ($0 !~ / +Classification:$/)
data[count,"profile"]=s
count++ # the Classification field denotes the end of the company record
}
END {
OFS=","
# output CSV header row
for ( key in fields ) {
printf "\"" fields[key] "\","
}
printf "\n"
# output data
for ( i=0; i<count; i++ ) {
for ( key in fields ) {
printf "\"" trim(data[i,fields[key]]) "\","
}
printf "\n"
}
}
保存為parse.awk ,然后使用./parse.awk < sample.txt
調用。 出來一個CSV,像這樣:
"tel","fax","mob","profile","company","pobox",
"(00974) 44568833","(00974) 44568811","(00974) 44568822","Walid Chamoun Architects (WCA) was founded in Beirut, Lebanon, in 1992, committed to the blablabla","Walid Chamoun Architects WLL","55803, Doha, Qatar",
"(00974) 44360011","","","","Al Ali Consulting & Engineering","467, Doha, Qatar",
"(00974) 44352126","","","","Al Gazeerah Consulting Engineering","22414, Doha, Qatar",
"(00974) 44448623","","","","Al Murgab Consulting Engineering","2856, Doha, Qatar",
有評論應該有希望解釋發生了什么。 這將在普通的舊awk中運行,並且不需要花哨的gawk功能。 請記住,awk數組是任意排序的。 這很容易打破一大堆不同的輸入數據,這只是你真正應該解析HTML而不是這樣的lynx -dump
惡作劇的眾多原因之一。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.