如何从 Python 中的 CSV 文件中抓取特定数据？

Question

Web scraping has been getting the best of me for the past few days. I know the very basics of it, but I've got no clue on how to scrape data properly. Here's the issue: I'm supposed to use a csv link (provided in the code below) to find out which of the Scandinavian countries (Denmark, Sweden and Norway) won the most gold medals in Curling, Skating, Skiing and Ice Hockey, all starting from year 2001. I made a very simple code as shown below (only for Curling though, I'll add the other sports later), but it doesn't return anything. I have no idea of what I'm missing, so I would truly appreciate any kind of input on this matter.

import requests
from bs4 import BeautifulSoup as bs
import operator
from collections import Counter
import re

url = "https://sites.google.com/site/dr2fundamentospython/arquivos/Winter_Olympics_Medals.csv"

csv = requests.get(url).text

lines = csv.splitlines()

for l in range(1, len(lines)):
  columns = lines[l].split(',')
  #print(columns)
  medalsSweden = 0
  medalsNorway = 0
  medalsDenmark = 0
  if columns[0] > '2001' and columns[4] == 'Curling' and columns[5] == 'NOR' and columns[7] == 'Gold':
    medalsNorway += medalsNorway + 1
    print(medalsNorway)
  else:
    if columns[0] > '2001' and columns[4] == 'Curling' and columns[5] == 'SWE' and columns[7] == 'Gold':
      medalsSweden += medalsSweden + 1
      print(medalsSweden)
    else:
      if columns[0] > '2001' and columns[4] == 'Curling' and columns[5] == 'DEN' and columns[7] == 'Gold':
        medalsDenmark += medalsDenmark + 1
        print(medalsDenmark)

Answer 1

The columns you're checking don't match up. Looking at the columns in lines :

>>> lines[0]
'Year,City,Sport,Discipline,NOC,Event,Event gender,Medal'

The "sport" is columns[2] and country is columns[4] . (You're checking 4 & 5 instead.) Also, you were checking against "discipline" which is specific within the sport. For example, "Skiing" is the sport and "Alpine Skiing" is a discipline within it. So each if statement should be changed to:

... and columns[2] == 'Curling' and columns[4] == 'NOR' ...

---

Other improvements:

1. when dealing with numbers like 'Year', convert those to int s rather than doing string comparison:

    if int(columns[0]) > 2001 and...

   In this case, you won't have an issue with it since they're 4-digit years and you're comparing with '2001' , a 4-digit string, so the string comparison works out.

2. Use the CSV module to parse CSV files

3. Re-structure your if statements so that they're not repetitive. Since you're only interested in 2001 onwards and Gold medals, it should be a top-level if . Others checks, if any, should be nested under it.

Answer 2

You can handle that within pandas.read_csv() function easily:

import pandas as pd


df = pd.read_csv(
    "https://sites.google.com/site/dr2fundamentospython/arquivos/Winter_Olympics_Medals.csv")

filt = (df['Year'] > 2001) & (df['Discipline'] == 'Curling') & (
    df['NOC'] == 'NOR') & (df['Medal'] == 'Gold')

print(df[filt])

      Year            City    Sport  ...    Event Event gender Medal
1972  2002  Salt Lake City  Curling  ...  curling            M  Gold        

[1 rows x 8 columns]