[英]Parsing HTML tags using BeautifulSoup
我正在解析一个有很多超链接的网页。 通过查找<a>标签中的所有'href'，能够使用beautifulsoup提取超链接。 接下来，我试图获取所有这些链接的“标题”，这些链接显示为<a>标记的标记内容。 [HTML新手，我认为这个尾随描述被称为标签内容吗？]下面是一个例子。 字符串'哪个课堂设置适合你？' 需要提取。 感谢帮助。
<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html">
Which Classroom Setting Suits You?
</a>
码
import requests
from bs4 import BeautifulSoup
import sys
import codecs
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
url = 'https://www.nytimes.com'
r = requests.get(url)
html_as_string = r.text
soup = BeautifulSoup(html_as_string, 'html.parser')
for link in soup.find_all('a'):
print soup.a.string ##Prints "LEARN MORE"
print soup.a.text ##Prints "LEARN MORE"
print (link.get('text'))##prints "None"
码：
from bs4 import BeautifulSoup
data = '''
<a href="https://www.nytimes.com/video/education/100000005108366/classroom-waldorf-montessori-public.html">
Which Classroom Setting Suits You?
</a>
'''
bs = BeautifulSoup(data, 'html.parser')
href = bs.find('a').text
print(href)
输出：
Which Classroom Setting Suits You?
这是我第二次编辑问题后的代码：目前，脚本从网站中提取所有href链接并打印其文本。 您需要优化脚本以专注于要解析的特定href链接。
这是我第二次编辑问题后的代码：
import requests
from bs4 import BeautifulSoup
url = 'https://www.nytimes.com'
r = requests.get(url)
html_as_string = r.text
soup = BeautifulSoup(html_as_string, 'html.parser')
for link in soup.find_all('a'):
print link.text
输出：
LEARN MORE »
Skip to content
Skip to navigation
中文 (Chinese)
Español
Today’s Paper
Video
World
U.S.
Politics
N.Y.
Business
Business
Opinion
Opinion
Tech
Science
Health
Sports
Sports
Arts
Arts
Style
Style
Food
Food
Travel
Magazine
T Magazine
Real Estate
Trump Allows for Cheaper, Less Regulated Health Plans
Comments
1
Open the door to relaxing A.C.A. rules on coverage.
2
Stop sending department officials to enrollment events.
3
Shut down the A.C.A. website for periods of time.
4
Cut funding to groups that help people enroll.
5
Slash spending on advertising and promotion.
Here Are 11 Ways Trump Is Undermining Obamacare
Trump Warns Puerto Rico: Federal Help Can’t Stay ‘Forever’
Comments
Obliterated: One of the Few Tropical Rain Forests in the U.S.
U.S. Will Withdraw From Unesco, Citing ‘Anti-Israel Bias’
What’s It Like to Cover Trump? Our Reporters Answer Questions at 6:30 P.M. Eastern
California Fires Burn ‘Faster Than Firefighters Can Run’
Comments
‘Thank God We Left’: Californians Describe Escapes
Satellite Images Show 1,800 Buildings Destroyed by Fire in Santa Rosa
The Daily 360
1:32
Sifting Through the Charred Ruins
More in
The Daily 360 »
Weinstein Company Has Been Aware of Payouts Since 2015
Comments
Facing Down the Network That Protected Weinstein
Comments
How Did NBC Miss Out on a Weinstein Exposé?
Palestinian Factions Sign Deal Paving Way for Joint Rule
Comments
The Bronx’s Quiet, Brutal War With Opioids
Comments
California Today: A Rising Death Toll From Wildfires
Comments
Your Friday Briefing
Audio
Listen to ‘The Daily’
Apple Podcasts
Radio Public
Stitcher
Smarter Living
What to Know About Having a Microwedding
How Would That Couch Look at Home? Check Your Phone
Comments
Journalism that matters. More essential than ever.
Subscribe to the Times
From the Magazine
Philip Pullman Returns to His Fantasy World
Comments
U.S. Woman and Family Freed From Afghan Militants
Comments
South Carolina Sidesteps the Law, and the Poor Go to Jail
Comments
Protesters in N.F.L. May Have an Ally: Labor Law
Comments
Different Ads, Different Ethnicities, Same Car
Genes for Skin Color Rebut Notions of Race, Study Finds
Comments
Citing Supreme Court, Judge Questions Menendez Case
Comments
Leader of Philippines Threatens to Expel E.U. Diplomats
Qatari Businessman Accused of Bribing Ex-FIFA Chief
Yankees Complete Comeback From 0-2 to Reach A.L.C.S.
Seychelles Reports a Case of Plague
Fed Still Puzzled by Inflation, but Rate Increase Is on Track
Morning Briefing: Australia Edition
Morning Briefing: Asia Edition
Morning Briefing: Europe Edition
California Today
Morning Briefing
Got a confidential news tip?
Learn more
Aussie Rules: Readers Respond
Vegemite, the Beloved Australian Spread, Goes Upmarket
Australia’s New Offer to Refugees: A Different Detention Center
She Is New Yorkier Than You
Olympic Tales From a 105-Year-Old Baron
Art Review: Collecting Strokes of Genius
Opinion
Mr. Trump, I Live in South Korea. You’re Scaring Me.
The Trumps, The Poodle, The Sex Scandal
Comments
Blow: Trump’s War Games
Kristof: North Korea Fears
Leonhardt: Falling Behind
Rosenthal: Trump’s Latest Outrage Against Puerto Rico
Stephens: Weinstein Enablers
Texas Should Not Execute Robert Pruett Tonight
Editorial: Trump Alone Can Order a Nuclear Strike
Edsall: Democrats Play Checkers, Trump Plays Chess
Greenhouse: Church Over State on Contraception
Has America Forgotten the Virgin Islands?
Congress’s Un-American, Bad-for-Israel Agenda
From Amazon, a Change That Hurts Authors
Ben Shapiro’s Hollow Bravery
What Makes Eminem’s Anti-Trump Rap Different
Getting Help Where It’s Needed in Puerto Rico
New York Times Recipe Commenters (Politely) Spill Their Guts
Smelly and Dirty and, in Bombing Trial, Suddenly a Star
Times Insider »
New York Times Recipe Commenters (Politely) Spill Their Guts
The Crossword »
Play Today’s Puzzle
The Crossword »
Play Today’s Puzzle
Wordplay »
timesvideo
explore all videos »
Television
‘Dynasty’ or ‘Dallas’? Squaring Off
Opinion
You Are Special! Now Stop Being Different
Science
Mountain Lions Sometimes Share Food
Food
Our Best Pizza Recipes
Opinion
Getting Help Where It’s Needed in Puerto Rico
Mind
High Blood Pressure Tied to Later Dementia
Travel
What Martha Stewart Can’t Travel Without
Opinion
Publishing’s Unfair Gray Market
Live
How to Stop Bleeding and Save a Life
Television
Review: Netflix’s ‘Mindhunter’
Opinion
Has America Forgotten the Virgin Islands?
N.Y. / Region
Lawsuits Seek Access to College Websites
World »
For Dignity and Development, East Africa Curbs Used Clothes Imports
Tech and Real Estate Lift the Fortunes of China’s Richest
Palestinian Factions, Hamas and Fatah, Sign Unity Deal
Business Day »
Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads
Different Ads, Different Ethnicities, Same Car
Kobe Steel Problems May Be More Widespread, Raising Fears on High-Speed Rail
Opinion »
Linda Greenhouse: On Contraception, It’s Church Over State
Editorial: Mr. Trump Alone Can Order a Nuclear Strike. Congress Can Change That.
Charles M. Blow: Trump’s War Games
U.S. »
We’re Tracking Ways Trump Is Undermining Obamacare. Here Are 11.
Another Victim of Hurricane Maria: Puerto Rico’s Treasured Rainforest
California Wildfires Burn ‘Faster Than Firefighters Can Run’
Technology »
Sheryl Sandberg Says Facebook Will Hand Over More Data on Russia-Linked Ads
State of the Art: The Frightful Five Want to Rule Entertainment. They Are Hitting Limits.
The Shift: We Asked Facebook 12 Questions About the Election, and Got 5 Answers
Arts »
Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.
Rose McGowan’s Twitter Account Locked After Posts About Weinstein
‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond
Politics »
Foiled in Congress, Trump Signs Order to Undermine Obamacare
U.S. Will Withdraw From Unesco, Citing Its ‘Anti-Israel Bias’
Trump’s Warning on Limits to U.S. Aid in Puerto Rico Adds ‘Insult to Injury,’ Mayor Says
Fashion & Style »
Unbuttoned: Fashion’s Gossip Addiction
With...Tom Hanks: Hollywood’s Most Decent Fella on Weinstein, Trump and History
Social Q’s: What Happens When a Will is Unsigned? (It’s Not Fun)
Movies »
All the Films of Studio Ghibli, Ranked
Critic’s Notebook: Harvey Weinstein Is Gone. But Hollywood Still Has a Problem.
Rose McGowan’s Twitter Account Locked After Posts About Weinstein
New York »
The Bronx’s Quiet, Brutal War With Opioids
Cy Vance Defends Decision Not to Pursue Case Against Harvey Weinstein
College Websites Must Accommodate Disabled Students, Lawsuits Say
Sports »
Yankees 5, Indians 2 | Yankees win series, 3-2: Didi Gregorius Leads Yankees Past Indians and Into A.L.C.S.
P.S.G.’s Nasser al-Khelaifi Accused of Bribing Ex-FIFA Chief Valcke
N.F.L. Week 6 Picks: Chiefs Over Steelers; Packers Over Vikings
Theater »
‘Brilliant,’ 41 and Lost to AIDS: The Theater World Asks Why
She Was About to Quit Acting. Then She Got the Role of a Lifetime.
Review: A Richard III Who Steps Straight Out of the Land of Id
Science »
Matter: Genes for Skin Color Rebut Dated Notions of Race, Researchers Say
ScienceTake: How Kangaroo Rats Escape Rattlesnakes
Trilobites: Solitary Pumas Turn Out to Be Mountain Lions Who Lunch
Obituaries »
Vincent La Selva, Dauntless Founder of a People’s Opera Troupe, Dies at 88
Michel Jouvet, Who Unlocked REM Sleep’s Secrets, Dies at 91
Y.A. Tittle, Quarterback Who Led Giants to 3 Title Games, Dies at 90
Television »
‘Dynasty’ or ‘Dallas’? Squaring Off in the Lily Pond
Review: Netflix’s ‘Mindhunter’ Is More Chatter Than Splatter
Ask a Showrunner: ‘Mr. Robot’ Season 3 Premiere: Sam Esmail on Parallel Universes and President Trump
Health »
In New Test for Obamacare, Iowa Seeks to Abandon Marketplace
Seychelles Reports a Case of Plague
Phys Ed: What Half-Marathons Teach Us About Running a Marathon
Travel »
36 Hours: 36 Hours in Aarhus, Denmark
Personal Journeys: Seeking Solitude in Japan’s Mountain Monasteries
Carry-On: What Martha Stewart Can’t Travel Without
Books »
American Beauties: Riding Shotgun on Martha Gellhorn’s Brave and Comic Adventures
Books of The Times: In Ron Chernow’s ‘Grant,’ an American Giant’s Makeover Continues
Feature: Philip Pullman Returns to His Fantasy World
Education »
On Campus: Campus Hate Lives on the Internet. Administrators Need to Catch Up.
Which Classroom Setting Suits You?
Your Money: A Game to Help Students Pay the Right Price for College
Food »
The Pour: Five Common Wine Myths, Debunked
Hungry City: Traditional Ethiopian Food in a Cosmopolitan Setting
Mike Isabella, Washington’s Restless Restaurateur
Magazine »
Feature: Philip Pullman Returns to His Fantasy World
Feature: In Northern Minnesota, Two Economies Square Off: Mining vs. Wilderness
Letter of Recommendation: Letter of Recommendation: Dunking
Real Estate »
The Hunt: Seeking the Peace of the Suburbs
On the Market: Homes for Sale in New York and Connecticut
On the Market: Homes for Sale in New York City
The Upshot »
A Bump Stock Ban Is Popular With the Public. But Experts Have Their Doubts.
The N.F.L. Is Now One of the Most Divisive Brands in the U.S.
Quality of Mercy: The Word May Be Toxic, but Amnesty Is Everywhere
Times Insider »
Why Tech is Starting to Make Me Uneasy
Times Insider: New York Times Recipe Commenters (Politely) Spill Their Guts
Smelly and Dirty and, in Bombing Trial, Suddenly a Star
Automobiles »
Wheels: For Electric Car Owners, ‘Range Anxiety’ Gives Way to ‘Charging Time Trauma’
Kobe Steel’s Falsified Data Is Another Blow to Japan’s Reputation
Different Ads, Different Ethnicities, Same Car
T Magazine »
Home and Work: A Paris Boudoir Where Everything’s Meant to Be Touched
Slinky Dresses, Meet Slouchy Trousers
The Illustrated Interview: Gloria Vanderbilt
Real Estate »
International Rents Per Square Foot
Comments
Search for Homes for Sale or Rent
Mortgage Calculator
Most Emailed
Most Viewed
Trending
Recommended for you
Go to Home Page »
The New York Times
World
U.S.
Politics
N.Y.
Business
Tech
Science
Health
Sports
Education
Obituaries
Today's Paper
Corrections
Today's Opinion
Op-Ed Columnists
Editorials
Op-Ed Contributors
Letters
Sunday Review
Video: Opinion
Today's Arts
Art & Design
Books
Dance
Movies
Music
N.Y.C. Events Guide
Television
Theater
Video: Arts
Automobiles
Crossword
Food
Education
Fashion & Style
Health
Jobs
Magazine
N.Y.C. Events Guide
Real Estate
T Magazine
Travel
Weddings & Celebrations
Reader Center
Classifieds
Tools & Services
N.Y.C. Events Guide
Multimedia
Photography
Video
NYT Store
Times Journeys
Subscribe
Manage My Account
NYTCo
Home Delivery
Digital Subscriptions
Crossword
Email Newsletters
Alerts
Gift Subscriptions
Corporate Subscriptions
Education Rate
Mobile Applications
Replica Edition
© 2017 The New York Times Company
Contact Us
Work With Us
Advertise
Your Ad Choices
Privacy
Terms of Service
Terms of Sale
Site Map
Help
Site Feedback
Subscriptions
View Mobile Version
使用BeautifulSoup解析HTML时缺少特殊字符和标记
[英]Missing special characters and tags while parsing HTML using BeautifulSoup
Python/BeautifulSoup - HTML 解析多个标签和类
[英]Python/BeautifulSoup - HTML Parsing Multiple Tags and Classes
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.