美丽的汤返回无

Question

I am trying to scrape the th element, but the result keeps returning None . What am I doing wrong?

This is the code I have tried:

import requests
import bs4
import urllib3


dateList = []
openList = []
closeList = []
highList = []
lowList = []

r = requests.get(
    'https://coinmarketcap.com/currencies/bitcoin/historical-data/')
soup = bs4.BeautifulSoup(r.text, 'lxml')
td = soup.find('th')

print(td)

Answer 1

There's an API endpoint so you can fetch the data from there.

Here's how:

import pandas as pd
import requests
from tabulate import tabulate

api_endpoint = "https://web-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical?id=1&convert=USD&time_start=1609804800&time_end=1614902400"
bitcoin = requests.get(api_endpoint).json()
df = pd.DataFrame([q["quote"]["USD"] for q in bitcoin["data"]["quotes"]])
print(tabulate(df, headers="keys", showindex=False, disable_numparse=True, tablefmt="pretty"))

Output:

+----------------+----------------+----------------+----------------+--------------------+-------------------+--------------------------+
|      open      |      high      |      low       |     close      |       volume       |    market_cap     |        timestamp         |
+----------------+----------------+----------------+----------------+--------------------+-------------------+--------------------------+
|  34013.614533  | 36879.69856854 | 33514.03374162 | 36824.36441009 | 75289433810.59091  | 684671246323.6501 | 2021-01-06T23:59:59.999Z |
| 36833.87435728 | 40180.3679073  | 36491.18981083 | 39371.04235311 | 84762141031.49448  | 732062681138.1346 | 2021-01-07T23:59:59.999Z |
| 39381.76584266 | 41946.73935079 | 36838.63599637 | 40797.61071993 | 88107519479.50471  | 758625941266.7522 | 2021-01-08T23:59:59.999Z |
| 40788.64052286 | 41436.35000639 | 38980.87690625 | 40254.54649816 |  61984162837.0747  | 748563483043.1383 | 2021-01-09T23:59:59.999Z |
| 40254.21779758 | 41420.19103255 | 35984.62712175 | 38356.43950662 | 79980747690.35463  | 713304617760.9486 | 2021-01-10T23:59:59.999Z |
| 38346.52950301 | 38346.52950301 | 30549.59876946 | 35566.65594049 | 123320567398.62296 | 661457321418.0524 | 2021-01-11T23:59:59.999Z |
| 35516.36114084 | 36568.52697414 | 32697.97662163 | 33922.9605815  |  74773277909.4566  | 630920422745.0479 | 2021-01-12T23:59:59.999Z |
| 33915.11958124 | 37599.96059774 | 32584.66767186 | 37316.35939997 | 69364315979.27992  | 694069582193.7559 | 2021-01-13T23:59:59.999Z |
| 37325.10763475 | 39966.40524241 | 36868.5632453  | 39187.32812109 | 63615990033.01017  | 728904366964.3611 | 2021-01-14T23:59:59.999Z |
| 39156.7080858  | 39577.71118833 | 34659.58974449 | 36825.36585131 | 67760757880.723885 | 685005864471.3622 | 2021-01-15T23:59:59.999Z |
| 36821.64873201 | 37864.36887891 | 35633.55401669 | 36178.13890106 | 57706187875.104546 | 673000645230.8221 | 2021-01-16T23:59:59.999Z |
| 36163.64923243 | 36722.34987621 | 34069.32218533 | 35791.27792129 | 52359854336.21185  | 665831621390.9865 | 2021-01-17T23:59:59.999Z |
| 35792.23666766 | 37299.28580604 | 34883.84404829 | 36630.07568284 |  49511702429.3542  | 681470030572.0747 | 2021-01-18T23:59:59.999Z |
| 36642.23272357 | 37755.89185872 | 36069.80639361 | 36069.80639361 | 57244195485.50075  | 671081200699.8711 | 2021-01-19T23:59:59.999Z |


and so on ...

Answer 2

I think the request object doesn't have a text attribute. Try soup = bs4.BeautifulSoup(r.content, 'lxml')