简体   繁体   中英

Scrape html data with beautifulsoup 4

I am trying to scrape data from nanopool to use in my python script, but whenever I attempt to scrape the data, I only get the class, not the text that I am looking for. The link to the specific webpage in question can be found here
Very simplified page source

First Piece of data I am trying to fetch:

<table class="table table-hover table-bordered" style="margin-bottom: 5px;" data-bind="with: calc">
<tbody>
        <tr data-bind="with: hour"><td>Hour</td>
            <td class="text-right" data-bind="text: coins">0.00015</td>
            <td class="text-right" data-bind="text: bitcoins">0.000012</td>
            <td class="text-right" data-bind="text: dollars">0.058</td><td class="text-right" data-bind="text: euros">0.048</td>
            <td class="text-right" data-bind="text: yuan">0.372</td><td class="text-right" data-bind="text: rubles">3.244</td>
        </tr>
</tbody>

Second Piece of data I am trying to fetch:
<div class="panel-body" data-bind="text: balance() + ' ETH'">0.02743061 ETH</div>

Code that I have been trying to use:
Input

r = requests.get('https://eth.nanopool.org/account/0x2a156c6dd3bdf2a0c5b284b45b2396c053c2a63d')
soup = BeautifulSoup(r.text, 'html.parser')
balance = soup.findAll('div', attrs = {'class': 'panel-body'})[3]
calc_box = soup.findAll('table')[3].findAll('tr')[2].find('td', {'class', 'text-right'})
print(calc_box)
print(balance)

Output

<td class="text-right" data-bind="text: coins"></td>
<div class="panel-body" data-bind="text: balance() + ' ETH'"></div>

It gets data asynchronously.

Balance API :

https://eth.nanopool.org/api/v1/balance_hashrate/0x2a156c6dd3bdf2a0c5b284b45b2396c053c2a63d

Day approximated earnings calculates by Average Hashrate for last 6 hours.

The Average Hashrate for last 6 hours API :

https://eth.nanopool.org/api/v1/avghashrate/0x2a156c6dd3bdf2a0c5b284b45b2396c053c2a63d

After got Average Hashrate for last 6 hours :

{"status":true,"data":{"h1":9.916666666666668,"h3":19.833333333333336,"h6":20.77777777777778,"h12":22.54861111111111,"h24":23.67013888888889}}

Using 'h6' value (20.7), send require to Calculator API :

https://eth.nanopool.org/api/v1/approximated_earnings/20.7

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM